Productive Failure

Designing an educational activity aimed at failure seems counter-productive. Unless, of course, it can somehow lead to improved learning that may not easily be elicited otherwise. That initial failure leads to productive learning in the long term, thus Productive Failure. No, I am not talking about making your first exam really, really, really hard so that the students do very poorly to thus motivate them to work really, really, really hard the rest of the term. While a tiny fraction might be so motivated, for the majority of the class I think this would be Unproductive Failure. If there can be Productive and Unproductive Failure, could there also be Productive and Unproductive Success?

Yes, according to Manu Kapur, a professor of cognitive studies who has been developing a framework for these four categories. A good summary article can be found in his 2016 article in Educational Psychologist (see picture below for citation and abstract).

Let’s take the extreme categories first. Productive Success, according to Kapur, involves designing learning activities that “maximize performance in the shorter term and maximize learning in the longer term.” Whether or not one can maximize both arenas independently is an open question. Unproductive Failure results in the opposite and the oft-cited example is pure “discovery learning” or “unguided problem solving” at least for novices. There is evidence that for experts, learning is enhanced by reducing structural elements that tend to hinder by constraint.

Unproductive Success is an interesting category. Performance is maximized in the short-term, but long-term learning remains elusive. Cramming for an exam is a common student strategy in this category. (For a summary of evidence-based student strategies, see this previous blog post.) Teaching approaches in this category might include drilling techniques. Some might consider ‘lecturing’ in the college classroom to also be in this category, but the evidence is not so clear cut. The cartoonish example of the droning professor who pays no attention to whether the students are paying attention, perhaps, if the students are there primarily to record the necessary information they need to regurgitate on the exam. But, there are many examples where direct or explicit instruction leads not just to better immediate learning of introductory material, but provides the necessary scaffold for more advanced material. However, its effectiveness is unclear for ‘far transfer’ – being able to abstract deeper principles and apply them to a wide variety of interesting problems. There is evidence both for and against this idea, and the devil is in the details of the experimental setups.

Kapur cites much of the work on Productive Failure carried out mainly in the last five years. Designing an activity involves two phases. According to Kapur: “(First) the problem-solving phase affords opportunities for students to generate and explore the affordances and constraints of multiple solutions to novel, complex problems. (Then) the consolidation phase affords opportunities for comparing and contrasting, organizing, and assembling the relevant student-generated solutions into canonical solutions.” The students may flail and generate incorrect or inconsistent solutions in phase one, but the consolidation phase somehow solidifies deeper conceptual learning that may also improve ‘far transfer’ at least in the examples given. There is clearly more work to be done in this area, but it is nevertheless intriguing.

Like many other instructors, I have mainly designed my classes around what I think lead to Productive Success. This includes a combination of lecturing, in-class group work, quizzes, class discussions, homework, reflections, exams, projects. Depending on the class, the students, the topic of the day, the classroom physical arrangements, I might do different things. While I’m good at assessing short-term knowledge learned by my students in my class, I’m not sure how to assess ‘far transfer’ effectively and thereby iteratively improve my classes for Productive Success. I do, however, have one prominent example of inadvertently designing an activity for Productive Failure.

Four years ago, I designed an Alien Periodic Table activity. (Former students reading this who experienced this in Fall 2013, thank you for being the guinea pigs!) It covers a week’s worth of in-class activity with four hours of in-class time. Basically, I invented a bunch of new elements and their physical and chemical properties, and dreamed up an underlying Periodic Table structure, similar in some ways to the actual Periodic Table in our known universe but different in other ways. The activities were designed to cram the experience of fifty years of historical floundering into a week-long experience. The names of the elements and their properties (including judiciously introduced “errors” and in some cases simply lacking in information) mirrored the actual messy history.

In the first 90-minute session (after a pre-class activity on how one would measure the relevant physical properties), students were given Element Cards and had to try to organize them in some fashion. There was no communication between students in different groups. Midway through class, we held a “science conference” where each group presented what they knew so far. They realized at this point that while they had a common set of cards, each group had some cards that others did not. I then doled out the new information (new cards for new discoveries!), and they went back to work in their groups. This time they were allowed to send one representative to another group to learn what was going on, share information, and bring that back to the home group.

Not surprisingly, by the end of the class session, no group had come anywhere close to the underlying structure, but there were multiple solutions and some clever (but wrong) ideas. The subsequent class was an hour lecture (with some interactive discussion) on the history of how our Periodic Table was discovered, leading up to the modern version found in textbooks. We also discussed the power of the table in displaying trends and ‘explaining’ properties. The pre-reading included selections from Eric Scerri’s The Periodic Table: Its Story and Its Significance. (Here’s my review of his more recent book.) Before the lecture started I showed the students the slide below (for fun).

In the second 90-minute session, the students were given photoelectron spectroscopy (PES) data. After figuring out how to correlate the PES data to a Bohr shell model of the atom, they were ready to take another stab at the Alien Periodic Table with the PES data for the Alien Elements. After the activity, each group had to turn in a final report presenting their periodic table arrangement, suggesting any missing elements (and coming up with names and physical properties), and reflecting on the experience. It was a lot of work for the students (inside and outside of class). It was a lot of work for me. It took me 80-100 hours to prepare a week’s worth of activity. Since then I’ve tried several pared-down versions on subsequent groups of students, making minor tweaks here and there. (I used it as a scaffold in my general chemistry New Elements project last year.) But I’ve never assessed the activity for ‘far transfer’ or whether these students better understood the deeper underlying concepts surrounding the Periodic Table compared to other students who did not do this activity. I did however inadvertently follow Kapur’s two-phase structure for Productive Failure.

Reading Niels Bohr and the Quantum Atom gave me an idea for a more tractable activity that might be more straightforward to assess. Here’s what I learned from the book. When J. J. Thomson first discovered the electron, he had to puzzle over the fact that the relative mass of the electron was roughly two thousand times less than the hydrogen atom. This led to the suggestion that the hydrogen atom was made up of a thousand negatively-charged electrons paired up with a thousand corresponding positively charged particles, each the same mass as an electron. It’s rather strange that the electron is 1/1840 times less massive than the proton that makes up most of the mass of the hydrogen atom. Why is that? We have no idea.

I’m envisioning an Alien Discovery of the fundamental particles that make up matter, the so-called ‘atoms’ of the Alien Universe. I can modify the quantities, names and particle types, and expand the set of ‘experiments’ to cover a range of work by Thomson and other colleagues working on the problem. Hopefully this will emphasize the strangeness of the subatomic particles. Every year, I try to impress upon the students this very strangeness. I think they sort-of-hear what I’m saying, but because most of them have had high school chemistry or physics, and have had the ‘information’ drilled-in, they’ve lost an appreciation for the creativity of the experiments and the strangeness of our fundamental knowledge of atoms, the building blocks of matter. I’m looking forward to fleshing this out and trying it on my general chemistry class next year.

One thing I’ve learned from reading cognitive science and education research, and trying out different things in my classes, is that there is no surefire best practice in the art and science of teaching and learning. So much depends on the instructor, the students, the relationships, the materials, the facilities, the resources, etc. It does keep things interesting, and I enjoy the flexibility to be creative and try new things.