To give your pet theory some heft, precede its name with the phrase evidence-based. Evidence-based anything is all the rage. And it’s broader than using the phrase data-driven which has its own problems. Given that educating students is what I do for a living, and that I teach in the sciences, I pay attention to theories about science education and how those might translate into suggested pedagogy. Taking a broader lens, applying a pedagogy on a larger-scale might result in sweeping educational policy.
Last month, the title of a paper in Educational Psychology Review caught my attention: “There is an Evidence Crisis in Science Educational Policy”. The authors include Paul Kirschner (who has in the past made the careful and important distinction between scientific practice and science education practice) and John Sweller (originator and champion of Cognitive Load Theory). I’ve found their overall arguments compelling over the years, even when I have occasional quibbles with a minor point of two. Here’s a snapshot of the abstract with the DOI reference.
The most useful thing about this paper is how it groups studies into three types: Program-based, Controlled, and Correlational. For practicing scientists, the gold standard are controlled experiments. These sorts of studies are key to figure out if a vaccine or a therapeutic drug is going to actually be effective; you’ve likely heard of RCTs or randomized controlled trials in the news related to whether something will be efficacious against Covid-19.
It’s not so easy to control most of the variables when you’re trying to figure out if a pedagogy you are experimenting with is favorable compared to “business as usual”. There are good experimental designs and there are poor ones. You typically don’t generate very large data sets, but a well-designed study can allow you to elicit causative factors and eliminate things that don’t work. In contrast to these, big-data correlational studies may show you connections between one factor and another, but do not pinpoint causes. One might say that these two types complement each other.
However, most of the education literature discusses program-based studies, which is neither of these two but a sort-of hybrid. As an instructor, you likely perform these even if you don’t write them up as a paper. You want to know if your students will do better if you try a different approach for a particular topic. So you design your new approach, try it out, and then anecdotally compare it to your previous approach. Or you might teach two sections of the same class where you use your new approach in one, and the old approach in another. While you might think of the “business as usual” approach as your control, that’s rarely the case. You’ve likely not designed your comparison to consider all sorts of confounding variables. You just want to know if what you’re trying seems like it works better according to some arbitrary measurement (quantitative or qualitative).
It’s good that Sweller and co-workers highlight these differences for the unaware among us who don’t often think about these different categories. What constitutes evidence? How strong is that evidence for a particular pedagogy “working well”? Why has there been no silver bullet in education – no one method that is the “best practice”? I think it’s because the process of education is complex, and therefore cannot be boiled down to a single best practice. But I also think it’s because we simply don’t have as much strong evidence about what works and what doesn’t, and in what situation… and there are a variety of confounding variables to throw us off!
I think the authors rightly state that all three types are important. And although program-based studies are the most prevalent and in some sense the easiest to design and carry out, we should be cautious about how widely applicable the “results” are, and be circumspect about supposed “causative” factors that are proposed in these papers. These sorts of studies are generally underpowered statistically and have too many confounding variables. But that doesn’t mean they’re unimportant. I continue to tweak my classes every semester using this approach. I think that’s a good thing to do to improve learning and the student experience. And it’s practical and useful. But I should be careful in making pronouncements that my pet pedagogy is particularly effective for reasons I have come up with anecdotally.
No comments:
Post a Comment