Annotated Self-Grading

The past Fall semester I experimented with student annotated “self-grading” in two classes. This was Idea #1 from my Three Fall Ideas over the summer.

In General Chemistry, my students had take-home midterm exams. These would be handed out in class on Friday and students would turn them in on Monday. The following instructions were on the cover page.

·      After you’ve finished studying, find a quiet spot and take the exam (closed-book, closed notes, etc) on your own. It’s just you, the exam, and a calculator! TIME LIMIT: 1 hour.

·      After the hour is up, you are allowed to make annotations to your exam in a different color. This includes corrections, clarifications, and anything you think makes your answer more accurate or easier to understand. You may consult your notes, the textbook, and even your classmates.

To encourage student adherence to the “closed” nature of the first hour, I told the students that if they made a good-faith effort to follow the two-color scheme, I would give them full credit on the exam regardless of how they actually did (pre-annotated or post-annotated). These midterm exams were worth 20% of the total grade in the class. (In previous years, these are worth ~50%). I also stressed the importance of following the instructions with a warning that it they didn’t adhere, it would give them a false sense of their knowledge of the material, and that it would come back and bite them on the higher stakes closed-book final exam, now worth 50% of the grade (instead of ~33%). This caused a slight uptick in stress approaching the final exam, but the students on average did just fine.

I am very pleased that all my students adhered, as far as I could tell, this past Fall. Because I was giving the students more “free points” during the semester, I had rescaled my grading bands into 10% increments per letter grade rather than my typical 15% increments. The end result: The class on average performed the same as the previous year. The slight difference could be attributed to all sorts of factors, but my sample sizes are too small to tell. I’d like to think that the act of annotating forced the students to think carefully about their answers and whether they could improve on it – an act of metacognition, so to speak. But it’s hard to say at this point. I did use the same final exam as last year with minor modifications and student performance was about 1% higher, not significantly different given my small class size. (I can reuse a final exam that I’ve never provided as a sample exam, and in my many years of teaching there has never been a problem.)

Several years ago, when I tried take-home exams without self-annotation, some students didn’t adhere which resulted in them doing terribly on the final exam. I’d like to think that allowing the self-annotation increased adherence. Another factor is that I was teaching a small Honors section composed of only first-year students (I did the same last year but without the take-home self-annotated exams). Honestly, these first-year Honors students tend to be good at following instructions, in my experience – which is why I was willing to try this experiment.

My grading time didn’t decrease significantly because I still went through the student exams carefully and made comments when students were missing something or if I thought the student did a particularly good job. No grade was provided by me on the exam. When I handed the students exam back to them, I included a detailed Answer key so they could make a comparison to one set of “ideal” answers. (There are often multiple ways to answer a question correctly on my exams.)

After reflecting on all this, I’ve decided to repeat this same experiment in my upcoming Spring second semester General Chemistry course. This is also a smaller Honors course, although there are more students (filtered in from different first semester sections) and they might not all be first-year students. One thing I will be adding to the midterm exam instructions is that after annotating their exams, students will estimate the grade on their exam before they put in the annotations. While this sounds counter-intuitive for someone who generally tries to de-emphasize grades, in this case I think it will be helpful as the students approach the final exam and reduce some stress. This is based on feedback from my students from last semester. But I don’t know for sure, so I’ll have to do the experiment.

In my Physical Chemistry I (Quantum) class this past Fall, I had students annotate their problem sets. There are seven problem sets during the semester (each worth 2% of the total grade). The problem sets are long and challenging for the students, but that’s where a lot of the actual learning happens. Students who don’t work at the problem sets tend not to do well. I encourage students to collaborate on problem sets. In their end-of-semester evaluations, students recognize the importance of the problem sets.

In previous years, I would see weaker students ‘copy’ the problem sets of stronger students when they worked together. Yes, they wrote up their own work, but how many of them really grappled with the problem set? My hope is that the self-annotation would take the pressure off having to turn in a “perfect” problem set (to get as many points as possible) and more importantly, encourage the students to look closely on what they understood and what they did not, where they got stuck, and how they can improve.

My problem sets had a “Finish By” date, although in every class I emphasize which problems students should attempt before the next class so they keep up with the material. On the Finish date, I hand out the Answer Key. Students then have to annotate their problem sets in a different color based on the solutions and turn them in on the “Due Date”, typically the next class period. At first some students would simply copy down parts that they got wrong, which required me to emphasize that annotating the exam might require making corrections in minor cases, but then in major cases the student should instead discuss where they went wrong and why, and not copy out large chunks of the Answer Key. I was more interested in seeing what they learned rather than a repeat of my Answer Key. After several reminders, the students by and large followed through.

My incentive to the students was that if they made a good-faith effort to follow the two-color scheme, I would give them full credit on the problem sets regardless of how they actually did pre-annotation. I’m pleased to say that students got better at annotating as the semester progressed. One disadvantage, in my opinion, is that fewer students came to office hours to get help (because they knew they would ‘get the Answer Key’ and not be penalized for an incomplete problem set pre-annotation). Students also have the option of not turning in a problem set at all and the 2% is then lumped in with their next exam. I’d done this for years to discourage students blindly copying other students and just submitting something for the sake of turning it in, thus increasing my grading time for no particular value. Every year a student or two would do this later in the semester; that was also true this semester (one student did this per problem set, although not always the same student).

As in my General Chemistry class, I changed my grade bands to 13% increments rather than 15% increments. This was based on rescaling how the 7 x 2% = 14% on problem sets would now be essentially “free points”. The overall average grade was comparable to previous years, as were the overall average grades in each exam. Actually, the average grades were marginally higher, but I had an unusually small class last semester of only fifteen students and these don’t always adhere closely to the norm. I also had a crop of very strong seniors in that class. Once again, I’d like to think that annotation helped with meta-cognitive aspects of student-learning, but it’s very difficult to measure how and what students are actually thinking. You can ask them in surveys, but that’s a proxy – not always reliable.

This coming Spring semester, my Physical Chemistry II class (Statistical Thermodynamics and Kinetics) will be more than twice as large. I definitely plan to have self-annotated problem sets again  because in P-Chem, the grading time decreases significantly. I no longer have to look for hard-to-find math errors and can concentrate on the conceptual parts. I still make comments on student problem sets as I go through them, but it’s a huge time-saver. I am, however, concerned about the weaker students not coming into office hours for help. (In the past, many of the weaker students didn’t come anyway. Typically, it’s the stronger students and mid-range conscientious ones that show up.) I’m not sure how much of a difference it makes; I didn’t see worse fallout from the lower end – although a small number of students still did quite poorly (which happens every year). I have been considering an assignment encouraging students to group-study for exams by asking them to write potential exam questions. I’m still working on my syllabus and need to give this a bit more thought.

Overall, I’ve been happy with introducing self-annotation in my General Chemistry I and Physical Chemistry I classes. We’ll see how they fare in General Chemistry II and Physical Chemistry II this coming semester.

P.S. As to my Idea #2 for Magic Hour usage, I think I did well. Here are two examples I blogged about: Connecting the Liberal Arts and Magic & Chemistry.

P.P.S. My Idea #3 for a student creative project to replace an exam in Physical Chemistry didn’t work as well. A couple of students attempted this – one attempt was middling at best, the other was poor. And it was a lot of extra reading for me. I’m not sure how much it helped the students consolidate the material they missed. Instead I think I will go to an option I’ve tried several times: If the grade on the final exam is higher than the student’s grade thus far in the class, then the overall grade is replaced by the final exam grade.