when a neat, elegant theory has to deal with messy reality? It gives you sympathy for an Aristotelian view of motion.
I decided to give my Math 5 class (Differential Equations) an end-of-term exam that was a single problem, a simplified population-ecology problem that they were to analyze using the techniques of the course. They were then to check their analysis by the use of independent methods (also from the course). The idea was multifold:
- give a (somewhat simplified) “real-life”-type problem that the course techniques could be used to address
- show that by suitable transformations, one could turn a problem that used one sort of approach into one that used a different one
- show that different techniques provide a way to check one’s answers without reference to knowing or looking up a “right” answer
- unify a number of the course techniques and approaches in a simple elegant setting
A “pre-testing” of the exam question with some students who took the exam early showed that it was a bit long and that senior Honors students in math couldn’t all solve a system of two equations in two unknowns even with a calculator (sigh), so I modified the main exam slightly.
One of the aspects, of course, of “real-world” problems is that messing up early on screws up the results down the line. Sadly, that aspect was brought out in all its realistic glory by several students, perhaps because they hadn’t done the labs carefully and so didn’t remember what key words describing certain processes meant.
The specific case in point: the system of differential equations I gave them to model the population ecology problem on the exam had to be solved for equilibrium points (I did learn from the pre-testing to get them to check their answers to make sure the equilibrium points were correct) and then had to be simplified in a mathematically legitimate way in order to be able to solve them (which is typical of real-world problems–if they could all be easily solved exactly, the world would be a different place). As the simplifying process (called linearization) had been in a fairly recent part of the course I (foolishly, as it turned out) assumed people would remember it. (Also, the meaning of the word “linear” to a mathematician should have provided a clue…)
A number of people did not linearize the system of equations correctly, which meant that most of their subsequent answers were incorrect. I ended up giving them some kind of B on the exam if their subsequent processes showed a reasonable understanding of the processes involved; to get some kind of A, however, they had to get some significant proportion of the answers correct.
I am currently considering what lessons (of the many available) I should learn from this experience. Suggestions?
Got to love when this happens! Having done this kind of assessment for a while (just take a look at the most recent programming assignment that I gave that my students took way to long to finish), there is always a lot to learn from these situations.
First and foremost is that the first time you give a new “real-world” assessment, you need to be prepared for the results you received. Imagine if you had not had a test case of students go through it the first time? I’ve found that the tweaks that go on on the subsequent assessments make the assignments tighter and cleaner. Don’t give up on the assessment – learn from the mistakes and make it better for next year.
Second thing I’ve learned is that the final answer is really just a small piece of the assessment. Learned this from grading AP exams (boy, want to be depressed by results, grade and AP Comp Sci question – yuck). Design your rubric so that you give students from going through the process of building their solution. Build in partial credit that gives them credit for effort. Try not to punish them too much for making a mistake early on if they keep working on it. You did this a bit by grading in tiers, but a detailed rubric will give them more feedback on what you were looking for and where they made mistakes.
Hang in there and keep trying these kinds of assessments. It gets easier to write them the more you make. Having a rubric has really helped me more fairly assess how the students actually did on the project. Hope this helps!
Well, I effectively had a rubric that I developed as I went along. My policy is to go through and correct everything and then come back through a second time to assign grades. The first pass gives me a sense of how the class did, where they stumbled and where they sailed through. By grading the whole class in parts (for instance, reading question 1 on everybody’s test before going back to read question 2 on everybody’s test), it makes it easier to compare responses and assign partial credit consistently.
I find rubrics have to be very general until you actually see what students write–they are capable both of mistakes and insights I would never have considered. Hence, unless I’ve given very similar problems before, I don’t necessarily know the details of a rubric until I’ve read through their answers. And a general, all-purpose one I find only one step away from useless. I first discovered this working in teams: a team can have the same rubric and still grade pretty differently depending on how it’s interpreted or what mistakes kids make that the rubric-makers didn’t anticipate.
Within the tiers you mentioned, students got different scores depending on how many of the subparts they got right, and I did pretty much give them credit for process after the point of the mistake. They could get an A- with really wrong answers if their processes were pretty much right. I didn’t give an A, however, to people who got really wrong answers. It just doesn’t seem to me that if your analysis predicts a population will explode when it actually moves steadily toward equilibrium that you should get a 95…