First Instinct Fallacy

You’re taking a multiple-choice exam. Looking at one of your answers, you begin to second-guess yourself. But you’re not still not sure. It seems like a 50-50 toss-up. Should you switch? Or should you stick with your first guess?

Most people prefer to stick rather than switch. Even though this is often detrimental to your exam score. Several studies have shown that switching from a wrong answer to a right answer is significantly more probable than switching from a right to wrong answer. Hence, those who switch score better on the exam, on average.

Yet the belief persists that it’s better not to switch. A Barron’s GRE Prep guide (from 2000) even reads: “Exercise great caution if you decide to change an answer. Experience indicates that many students who change answers change to the wrong answer.” This is known as the First Instinct Fallacy.

Why does the first instinct fallacy persist? That’s the subject of a 2005 paper by Kruger, Wirtz and Miller (Journal of Personality and Social Psychology, vol. 88, no. 5, pp. 725-735). The title and abstract are shown below.

The authors present experimental data arguing that this fallacy persists because when you switch from a right to a wrong answer, it is both more regrettable and more memorable, compared to when you didn’t switch (even though your answer was wrong and your second guess was the right answer although you didn’t ultimately choose it). This is one of those explanations that seems obvious after-the-fact, but in reality the causal link is not easy to tease out – hence the need for clever and sneaky experiments. I recommend reading the paper in full if you are interested in all the details.

I have not taken a high-stakes exam in a while that ‘counted’. (I do take the exams I have written to ensure they are of the appropriate length and difficulty.) But I do grade plenty of exams every semester. Very few of my questions are multiple choice or true-false, however I have observed many times the regret of students who seemingly believe the first instinct fallacy. When the graded exams are returned, there is plenty of groaning and ‘if only’ self-recriminations. I have very rarely observed a student bemoaning not changing their answer. Part of this has to do with attending only to what is easily observable in front of you, although there are occasions when a student writes out two solutions and then scratches out the first one.

Reflecting on what I do as a grader, I might be exacerbating the first instinct fallacy by explicitly writing comments about the student’s crossed-out first answer being correct. Perhaps I should stop doing so. I’m surprised I had not heard of the first instinct fallacy until alerted to it from a recent blog post by Tim Harford. I suspect that if I had taken an exam, I would respond with similar groaning as my students. More importantly, when taking an exam, I would likely also resist making a change in a second-guess situation – probably to my detriment.

I think I need to alert my students to this fallacy and add it to the list of learning tips and strategies in my syllabus. Since I am trying to improve student metacognition, I could couple it with the importance of looking over your exam after you think you’ve finished it. Would such an alert be of any use? The authors question this in a sobering final paragraph:

“The most obvious implication of this research, however, is that test takers should be warned that sticking with one’s first instinct is an ill-advised strategy. Even with such exhortations, however, people may be reluctant to switch as often as they should. Students who have been explicitly instructed to switch as often as they should. Students who have been explicitly instructed as to the invalidity of the first instinct theory are no more likely to change their answers – nor receive higher test scores – than test takers who do not receive instruction [many references included]. The (misleading) personal evidence garnered from a lifetime of test taking may be difficult for test takers to overcome.”

Human beings are not robots. Things that affect us more significantly, and lodge in our memory, can be blind spots that we’ve learned only too well.