Teaching Evaluations: Yet Another Proposal

Now that the semester is over, I’ve been catching up on some of my reading. Today’s blog is on the dreaded Teaching Evaluations filled out by students usually in the last week of classes. The ineffectiveness of such evaluations is common fodder for articles and blogs in the higher education world. There is much griping about what it can measure (if anything), how it can be manipulated, and the evils of using such a device to evaluate teaching effectiveness. They are rarely accompanied by solid proposals of how to improve.

In the Jan-Feb 2015 issue of Change, Nobel Prize Winner and oft-quoted physics education “specialist” Carl Wieman has an article titled “A Better Way to Evaluate Undergraduate Teaching”. Wieman goes through the current common methods and critiques about their limitations. He then goes on to outline his Teaching Practices Inventory approach. The idea is to quickly and efficiently “measure” a set of teaching practices used by the instructor with somewhat objective criteria. (Wieman acknowledges its limits and which parts of the inventory may be more prone to subjectivity.)

An analogy is drawn to how research is evaluated. Wieman writes: “While having a relatively large number of grants and publications does not guarantee substantial research contributions, they tend to be well correlated. Correspondingly, a faculty member in the sciences who does not get research money and does not publish papers is very unlikely to be making significant research contributions. Using effective, research-based teaching practices as a proxy for the desired student outcomes is based on much the same concept.”

Wieman claims that there is now sufficient research showing a strong correlation between teaching practices in the STEM fields and student learning outcomes, to justify the use of a practice inventory as a proxy. He also refers to such practices as “consistent with empirically grounded theoretical principles for the acquisition of complex expertise.” The article references a paper by Wieman and his collaborator Sarah Gilbert who have refined the inventory over a six-year period. I haven’t read the paper yet, but it is on my list.

The approach is fast – one goes through the inventory checklist, and the data is self-reported which in itself encourages self-reflection. However the checklist is easy enough to use for the most part if reported by someone else looking at the materials (although there are some limitations, and there is some subjectivity involved). A classroom observation protocol helps to get at some of the in-class activities and interactions. A rubric is used to convert “raw data” from the inventory into numerical “scores”.

Wieman goes through some of the potential pitfalls with the approach. He in fact had anticipated my main concern: “The most obvious concern with the inventory data and scoring rubric is that they measure the use of particular practices, not how well those practices are being used. But the important comparison here is not with perfection, but rather with alternative methods of evaluation.” I’m inclined to agree with him that there is no perfect method, and there will always be valid critiques of any method proposed. However his method might actually be workable, certainly in the sciences, and possibly to other fields. It is expected that some amount of tweaking will be required, and Wieman acknowledges this.

Overall, I think this might be something worth trying (from my point of view as a department chair). I might pull together a small task force to see what might be tweaked for some of our courses and work out an implementation strategy for a test run. But first I need to balance our budget as we near the end of the fiscal year. And I need to gear up for student summer research. There’s always something else to do. Perhaps writing about this in a blog will motivate me to actually do something, instead of just reading something and forgetting about it.

What did my students learn?

Finally, I’m done grading! Overall the students did well with a solid B average. This, after all, was a small Honors class. When I’ve taught the regular section, the average is somewhat lower.

On the Final Exam of my class, I had the following 1-point question (on a 50-point exam) at the end of the exam: “What was the most important thing you learned from this class?”

The answers can be grouped in three categories and I had roughly equal numbers of responses in all three.

One group of students talked about specific topics from class. The majority of responses were about energy, entropy and thermodynamics although a few other topics were mentioned. Here are my two favorite ones:

·      Entropy and energy in general is a lot more complicated than it sounds.

·      The most important thing I learned in this class came from the lectures and readings on fuel sources and renewable energy sources.

Another group of students enjoyed the connections made between what we learning in class with their life outside of class. In particular, I was gratified that the blog posts were credited with helping them see this. Here is a selection:

·      How to observe the surroundings and to formulate ideas about science (blog posts).

·      How to connect what we learning in class to the outside world. This was one of the first times where I could see how what I learn can have an impact on the world.

·      How to apply what I’ve learned so far this semester to real world problems (in order to come up with possible solutions to those problems?)

·      In order to be great at anything, you really need to be interested in it. I think the blog posts really created a better sense of chemistry and made it a lot more interesting.

·      I have learned how to articulate my thoughts better, especially through writing due to the blogs. I am very grateful for this because it is an important life skill.

·      To try and apply the concepts we’ve learned to our every day lives (encouraged by the blog requirement) because it makes what we learn more interesting.

The final group talked about learning how to learn, and I’m glad that some of the metacognitive aspects and reflection I had been emphasizing seems to have borne some fruit. Here’s a selection:

·      Study technique: I learned that you have to study a little every day and quiz yourself on material.

·      How to prepare and study prior to lectures on my own.

·      Besides all the chemistry and the real life applications of chemistry, the most important thing I learned in this class is that I always know more than I think I do. You just have to sit and think about the problem.

·      I learned how to get my grade up. I worked really hard on getting my grade to a B+ after I found that I had a C in the beginning of the semester.

·      The most important thing I learned is to never give up. I also liked galvanic cells.

·      I learned that it is very important not only to know the concepts, but to know how to apply them to problems quickly and efficiently by practicing.

And my favorite comment which was short and to the point:

·      I learned that in a strenuous course, sleep is extremely important.

Amazingly, I am already looking forward to the classes I will be teaching in the Fall semester. I’ve already started thinking about some of the activities. Over the summer I will continue reflecting on what worked well in my “experiment” this semester and what didn’t go so well. After all, that’s how we learn to improve. I’d like to thank my students for sticking through this semester with me

Molecular Machines

I just finished reading Life’s Ratchet: How Molecular Machines Extract Order From Chaos by Peter M Hoffmann, a professor of physics and material science at Wayne State University. While the book purports to explain the mystery of life by examining molecular machines, it does well on the latter but not the former. Hoffmann’s prose is relatively easy to follow and he has uses simple and clear everyday examples to illustrate his main points. In some cases I feel the examples are a bit simplistic, and obscures the more complex and interesting questions. However the book overall does well in giving the non-biologist a good idea of how interesting and complex the molecular machines in our body can be.

The punchline of the book is that Life is based on both Chance and Necessity, a play on Monod’s famous book of the same name. The chance comes from statistical mechanics and thermal fluctuations at the molecular level. The necessity comes from physical law, the physics of chemical interactions at the molecular level. The book does well when the author focuses on describing phenomena at the nanoscale. It gets shaky or simplistic when he slides from physics into metaphysics. A philosopher of science might cringe at some of the metaphysical pronouncements made. Since I’ve started to think about my classes next semester, and in particular how our core curriculum revision will incorporate "scientific inquiry" into our introductory classes, my warning sensor alerts me when I see scientists slide into metaphysical claims.

I almost stopped reading the book after the first couple of chapters but I’m glad I persevered through Chapter 4. Right at the end of the chapter titled “On a Very Small Scale”, there is a section on how magnitudes of energies of a rather wide range of physical processes and interactions converge at the nanoscale and coincide with thermal energy at room or body temperature. I had never thought about this before (and I really should have as a physical chemist). Hoffmann's book led me to a superb 2006 article by Rob Phillips and Steve Quake titled “The Biological Frontier of Physics”. I’ve reproduced the key figure here, but the article can be found at Physics Today. (I highly recommend it!) Hoffmann discusses the examples used by Phillips and Quake, and in my opinion owes much of his book to their work, but in addition he adds a detailed description of myosin “walking” and does a nice job tying this to the Ratchet of his book.

In any case, learning about the interesting convergence of energies at the nanoscale was sufficient motivation for me to read through Hoffmann’s entire book. In all fairness, he does temper the strongly reductionist slant in the early parts of the book with the need to understand emergence, and how both approaches are two sides of the same coin. As scientists who study complex systems, we need to use both approaches, and Hoffmann acknowledges that we are indeed far from the “mystery of mysteries” that is Life. Hoffmann does not go as far as others who have delved into how exactly the second law of thermodynamics and statistical mechanics might work to support systems that maintain themselves away from equilibrium (one definition of life). But perhaps that is neither his intention nor his audience.

I’m not sure how I will use the idea of convergent energies at the nanoscale in my own research projects, but summer is approaching, and I’m looking forward to pondering this issue. But first I need to get through Finals this coming week!

The Knowable Futures of Life

I recently finished reading A New History of Life by Peter Ward and Joe Kirschvink. The book traces the history of earth from 4.5 billion years ago to the present day. It was a good refresher for me to go through the different eras in the Geological Time Scale. Much of what was written was familiar to me from other reading, but there were a few new things I did not know about. I did not realize the Neo-Proterozoic era had been subdivided such that there is now a Cryogenian era from 850 to 635 Mya. I also got up to speed in the recent Ediacaran discoveries. Although I had read Nick Lane’s book Oxygen a while back, I found it interesting to reminded of the close correlation between CO₂ and O₂ levels, and extinctions/evolutionary “explosions”. My knowledge of plant evolution was also rather scant and I found it fascinating to read about the effect of CO₂ and O₂ on the evolution of grasses, trees, flowering plants, C₃ and C₄ species, and more.

Given that I asked my students about the “faint young sun hypothesis” in a recent quiz, it was interesting to read the final chapter in Ward and Kirschvink’s book titled “The Knowable Futures of Earth Life”. The luminosity of the sun has increased by some 30 percent since the Earth’s formation, but it is only going to get brighter and more intense. The result, according to Ward and Kirschvink, will be “loss of oceans [and] hellish conditions, similar to those that exist on Venus.” While Earth is currently well-positioned in a “Goldilocks” habitable zone, this zone will move further out. In fact, Mars will move into such a zone.

Much has been speculated about life on Mars. As the intensity of the sun increases, and temperatures rise, we would expect the melting of the polar ice caps. Unfortunately there is hardly any atmosphere and it is unclear that Mars will be able to support one so all that water may simply evaporate. Could Mars be terraformed? That could prove very difficult. And it’s unclear if there is sufficient geothermal energy to support an underground complex (like Zion in the Matrix movies). It might be easier to build an Elysium-like space-station if our technology actually gets sufficiently advanced.

One of the authors, Peter Ward, is credited as the originator of the Medea hypothesis – that life essentially destroys itself over time, as evidenced by the series of mass extinctions in the past. The authors comment that “it seems appropriate to us to end this book with some comments about the most Medean of all species ever evolved: our own. What will the future be for our own species?” There are a number of speculations made and questions posed in this section, ending with “Is humanity but the builder of the next dominant intelligence on Earth – the machines?” For some reason this makes me want to watch the Terminator movies again. The future may be knowable unless we can go back in time to change it!

Transfer and Teaching Metacognitive Strategies

I recently finished reading Minds Online by Michelle Miller, a professor in the psychological sciences at Northern Arizona University. The book discusses much more than online learning. First the basis for learning is laid out given what we know from research on learning and cognition. Three of the nine chapters in the book are devoted to Attention, Memory and Thinking. In each chapter, the author suggests practical strategies of incorporating the principles discussed into an online learning context. I found that many of the suggestions translate well (with minor modification) into face-to-face classes too. Overall, this is a book I will probably want on my bookshelf. So I’m likely to purchase it after I return my library copy.

One of the things I have been thinking about this semester is how to teach metacognitive strategies to my students. This, I think, is really important for students to get the most out of their own learning. Are there best-practice strategies? Over the years I spend some amount of time in my office hours teaching students how to study effectively, particularly if they are first-year students. (In my first year teaching I was a little shocked by how ineffective strategies employed by the average student.) I have been trying to add a dose of self-reflection strategies to my students’ thinking, but it seems harder for them to “catch”, or I’m not doing a very good job trying to convey how one actually goes about doing this.

In the chapter on Thinking, there is a section on Transfer, i.e., getting students to apply what they have learned to new domains. Prof. Miller writes: “Teaching metacognition does not have to be highly technical or theoretical in nature… [Having faculty] introspect about how they approach material, as disciplinary experts, [is] a way to uncover what they should teach their students about thinking in a discipline… Infusing metacognition into teaching means focusing on the ‘process as well as the product’… and maximize transfer between [old and new problems]”.

Five suggestions are provided by the author. I do some of these things, but perhaps not well. Here’s my self-analysis.

1. Emphasize how knowledge is organized when the original material is being taught. In class, I’m decent at making connections and describing the organizational structure, but I don’t do it as consistently I should. There’s room for improvement.

2. Go for depth rather than breadth. One of the problems teaching General Chemistry is that it is a pre-requisite for subsequent science courses that assume a certain body of knowledge. Therefore I find that I’m often pressed for time just to get “coverage”. I have made some progress, i.e., I’ve cut out a few peripheral things and drilled down a little further in some areas. However what I really need to do is sit down with my colleagues and really revisit our curricular structure and content.

3. Emphasize underlying principals and conceptual structure. The author suggests drawing students’ attention to abstract, general aspects rather than on surface features. In class I’m good at posing questions, listening to students answer, and posing follow-up “why” and “why not” questions. However, sometimes I don’t do this as consistently if I’m trying to cover a certain amount of material, so I’ll need to work more on this.

4. Frequent quizzing. Finally, this is something I do consistently!

5. Tell students why wrong answers are wrong. I get the students to help figure this out for themselves in class and in my office hours. However I’m not sure how much reflection they do when they’re working on a homework assignment outside of class.

My semester is drawing to an end, but I am looking forward to being a better teacher in my classes next semester!