Teaching Evaluations: Yet Another Proposal

Now that the semester is over, I’ve been catching up on some of my reading. Today’s blog is on the dreaded Teaching Evaluations filled out by students usually in the last week of classes. The ineffectiveness of such evaluations is common fodder for articles and blogs in the higher education world. There is much griping about what it can measure (if anything), how it can be manipulated, and the evils of using such a device to evaluate teaching effectiveness. They are rarely accompanied by solid proposals of how to improve.

In the Jan-Feb 2015 issue of Change, Nobel Prize Winner and oft-quoted physics education “specialist” Carl Wieman has an article titled “A Better Way to Evaluate Undergraduate Teaching”. Wieman goes through the current common methods and critiques about their limitations. He then goes on to outline his Teaching Practices Inventory approach. The idea is to quickly and efficiently “measure” a set of teaching practices used by the instructor with somewhat objective criteria. (Wieman acknowledges its limits and which parts of the inventory may be more prone to subjectivity.)

An analogy is drawn to how research is evaluated. Wieman writes: “While having a relatively large number of grants and publications does not guarantee substantial research contributions, they tend to be well correlated. Correspondingly, a faculty member in the sciences who does not get research money and does not publish papers is very unlikely to be making significant research contributions. Using effective, research-based teaching practices as a proxy for the desired student outcomes is based on much the same concept.”

Wieman claims that there is now sufficient research showing a strong correlation between teaching practices in the STEM fields and student learning outcomes, to justify the use of a practice inventory as a proxy. He also refers to such practices as “consistent with empirically grounded theoretical principles for the acquisition of complex expertise.” The article references a paper by Wieman and his collaborator Sarah Gilbert who have refined the inventory over a six-year period. I haven’t read the paper yet, but it is on my list.

The approach is fast – one goes through the inventory checklist, and the data is self-reported which in itself encourages self-reflection. However the checklist is easy enough to use for the most part if reported by someone else looking at the materials (although there are some limitations, and there is some subjectivity involved). A classroom observation protocol helps to get at some of the in-class activities and interactions. A rubric is used to convert “raw data” from the inventory into numerical “scores”.

Wieman goes through some of the potential pitfalls with the approach. He in fact had anticipated my main concern: “The most obvious concern with the inventory data and scoring rubric is that they measure the use of particular practices, not how well those practices are being used. But the important comparison here is not with perfection, but rather with alternative methods of evaluation.” I’m inclined to agree with him that there is no perfect method, and there will always be valid critiques of any method proposed. However his method might actually be workable, certainly in the sciences, and possibly to other fields. It is expected that some amount of tweaking will be required, and Wieman acknowledges this.

Overall, I think this might be something worth trying (from my point of view as a department chair). I might pull together a small task force to see what might be tweaked for some of our courses and work out an implementation strategy for a test run. But first I need to balance our budget as we near the end of the fiscal year. And I need to gear up for student summer research. There’s always something else to do. Perhaps writing about this in a blog will motivate me to actually do something, instead of just reading something and forgetting about it.