Saturday, March 21, 2009

Follow up to assesment comment made by Dr. Howell

I was very interested in a particular comment made in passing by Dr. Howell when he visited our Distance Education class. The comment had to do assessment errors being made when teachers calculate final grades. The error come s as the various weighted components are put together. Dr. Howell said this was very common and given that I am prone to common mistakes, I thought that I might benefit from learning more about this. Dr. Howell was gracious enough to send me this article and recommended that I read it to find out if I was making errors. The article talks about four common errors that are made. These errors are the following:

(a) the Average Speed Error – this is named for the classic problem pertaining to how many hours a person drives if they drive for 60 mph for 120 miles and 30 mph for the second 120 miles. People logically make errors by not making sure the problem has the correct numerator and denominator. This error could introduced if tests are worth different amounts and this is not taken into account. I don’t think I have this problem.

(b) the Weight Problem—this is a problem that I did have, although I’m not sure what to do about it or how serious a problem it is. This problem has to do with not taking into account the standard deviation on one test versus another. For example, if one test has a standard deviation of 5 and a second has a standard deviation of 25 then the tests should be weighted differently.

(c) the Natural Variation Violation – this problem is introduced when a teacher calls any grade from 93-100 an “A” and then compiles grades together. Thus a person who got a “93” on each assignment would get the same grade as a person who got “100” on every assignment. This is also a problem I don’t have.

(d) the Mars Climate Orbiter Miscalculation. This is related to the “weight problem” and shows how to correct for it.

My grading metric is something like this:

Scripture Reading – 100 points
Weekly papers – 100 points
Attendance –100 points
Mid-Terms – 200 Points
Final – 200 Points

So there are a total of 700 points, and I add up all the points a student gets, divide it by 700 and that results in a percentage earned of total points.

94-100 A
90-93.9 A-
87-89.9 B+
83-86.9 B
And so forth.

I guess my main objection to the weight and mars climate problem is that they seem too close to grading on a curve. At least for the present I am not as concerned with how well one student does relative to another, but whether each individual can reach the target. In addition the necessary calculations to adjust for the weight problem are somewhat tricky and most students would perceive them to be unfair. Francis, the author of the article, addresses these concerns but his attempts to resolve them were unsatisfactory (at least for me). I still remember 10 years ago taking a religion class and being so angry that teacher insisted on grading on a curve where 20% of students got an A, 20% an A-, and so forth. Although I got an A in that class, I thought to myself, “God doesn’t grade on a curve. He sets criteria for each kingdom and if you meet the criteria, then you get in!”

I could see somebody responding to this statement by saying, “Sure, if you are an omniscient teacher, go ahead and do that, but if not, you should probably try another approach.” And that may have merit. I’m not saying that I completely reject the “weight” problem, but I did not see it as a problem. If I weight the final twice as heavily and a student does really well on the final, then in my view that should compensate for a poor score on the first exam, regardless of what the standard deviations were.

I look forward to learning more from Dr. Howell on this issue, because he doubtless has some insights that would help resolve my concerns on this matter.

1 comment:

Peter Rich said...

I tried to comment on this yesterday, but for some reason it hasn't been posted.

anyhow, I think you've given a great summary here. I'm not completely sure how to answer any questions herein, and I probably need to reconsider how I grade as well. One thing that does stick out is my experience with a Spanish teacher here at BYU. He allowed us to use our score from the final exam as our score for the course (assuming we'd only take it if it were better). While that motivated many to study harder, he also noted that the Final exam score was usually in the same ballpark as their course score (only a little lower), which tells me that he had his course and the different value of things figured out pretty well. He mentioned that hardly anyone passed up their course score with their exam score, though.