Teaching Evaluations: Yet Another Proposal

Now that the semester is over, I’ve been catching up on some of my reading. Today’s blog is on the dreaded Teaching Evaluations filled out by students usually in the last week of classes. The ineffectiveness of such evaluations is common fodder for articles and blogs in the higher education world. There is much griping about what it can measure (if anything), how it can be manipulated, and the evils of using such a device to evaluate teaching effectiveness. They are rarely accompanied by solid proposals of how to improve.

In the Jan-Feb 2015 issue of Change, Nobel Prize Winner and oft-quoted physics education “specialist” Carl Wieman has an article titled “A Better Way to Evaluate Undergraduate Teaching”. Wieman goes through the current common methods and critiques about their limitations. He then goes on to outline his Teaching Practices Inventory approach. The idea is to quickly and efficiently “measure” a set of teaching practices used by the instructor with somewhat objective criteria. (Wieman acknowledges its limits and which parts of the inventory may be more prone to subjectivity.)

An analogy is drawn to how research is evaluated. Wieman writes: “While having a relatively large number of grants and publications does not guarantee substantial research contributions, they tend to be well correlated. Correspondingly, a faculty member in the sciences who does not get research money and does not publish papers is very unlikely to be making significant research contributions. Using effective, research-based teaching practices as a proxy for the desired student outcomes is based on much the same concept.”

Wieman claims that there is now sufficient research showing a strong correlation between teaching practices in the STEM fields and student learning outcomes, to justify the use of a practice inventory as a proxy. He also refers to such practices as “consistent with empirically grounded theoretical principles for the acquisition of complex expertise.” The article references a paper by Wieman and his collaborator Sarah Gilbert who have refined the inventory over a six-year period. I haven’t read the paper yet, but it is on my list.

The approach is fast – one goes through the inventory checklist, and the data is self-reported which in itself encourages self-reflection. However the checklist is easy enough to use for the most part if reported by someone else looking at the materials (although there are some limitations, and there is some subjectivity involved). A classroom observation protocol helps to get at some of the in-class activities and interactions. A rubric is used to convert “raw data” from the inventory into numerical “scores”.

Wieman goes through some of the potential pitfalls with the approach. He in fact had anticipated my main concern: “The most obvious concern with the inventory data and scoring rubric is that they measure the use of particular practices, not how well those practices are being used. But the important comparison here is not with perfection, but rather with alternative methods of evaluation.” I’m inclined to agree with him that there is no perfect method, and there will always be valid critiques of any method proposed. However his method might actually be workable, certainly in the sciences, and possibly to other fields. It is expected that some amount of tweaking will be required, and Wieman acknowledges this.

Overall, I think this might be something worth trying (from my point of view as a department chair). I might pull together a small task force to see what might be tweaked for some of our courses and work out an implementation strategy for a test run. But first I need to balance our budget as we near the end of the fiscal year. And I need to gear up for student summer research. There’s always something else to do. Perhaps writing about this in a blog will motivate me to actually do something, instead of just reading something and forgetting about it.

What did my students learn?

Finally, I’m done grading! Overall the students did well with a solid B average. This, after all, was a small Honors class. When I’ve taught the regular section, the average is somewhat lower.

On the Final Exam of my class, I had the following 1-point question (on a 50-point exam) at the end of the exam: “What was the most important thing you learned from this class?”

The answers can be grouped in three categories and I had roughly equal numbers of responses in all three.

One group of students talked about specific topics from class. The majority of responses were about energy, entropy and thermodynamics although a few other topics were mentioned. Here are my two favorite ones:

·      Entropy and energy in general is a lot more complicated than it sounds.

·      The most important thing I learned in this class came from the lectures and readings on fuel sources and renewable energy sources.

Another group of students enjoyed the connections made between what we learning in class with their life outside of class. In particular, I was gratified that the blog posts were credited with helping them see this. Here is a selection:

·      How to observe the surroundings and to formulate ideas about science (blog posts).

·      How to connect what we learning in class to the outside world. This was one of the first times where I could see how what I learn can have an impact on the world.

·      How to apply what I’ve learned so far this semester to real world problems (in order to come up with possible solutions to those problems?)

·      In order to be great at anything, you really need to be interested in it. I think the blog posts really created a better sense of chemistry and made it a lot more interesting.

·      I have learned how to articulate my thoughts better, especially through writing due to the blogs. I am very grateful for this because it is an important life skill.

·      To try and apply the concepts we’ve learned to our every day lives (encouraged by the blog requirement) because it makes what we learn more interesting.

The final group talked about learning how to learn, and I’m glad that some of the metacognitive aspects and reflection I had been emphasizing seems to have borne some fruit. Here’s a selection:

·      Study technique: I learned that you have to study a little every day and quiz yourself on material.

·      How to prepare and study prior to lectures on my own.

·      Besides all the chemistry and the real life applications of chemistry, the most important thing I learned in this class is that I always know more than I think I do. You just have to sit and think about the problem.

·      I learned how to get my grade up. I worked really hard on getting my grade to a B+ after I found that I had a C in the beginning of the semester.

·      The most important thing I learned is to never give up. I also liked galvanic cells.

·      I learned that it is very important not only to know the concepts, but to know how to apply them to problems quickly and efficiently by practicing.

And my favorite comment which was short and to the point:

·      I learned that in a strenuous course, sleep is extremely important.

Amazingly, I am already looking forward to the classes I will be teaching in the Fall semester. I’ve already started thinking about some of the activities. Over the summer I will continue reflecting on what worked well in my “experiment” this semester and what didn’t go so well. After all, that’s how we learn to improve. I’d like to thank my students for sticking through this semester with me

Molecular Machines

I just finished reading Life’s Ratchet: How Molecular Machines Extract Order From Chaos by Peter M Hoffmann, a professor of physics and material science at Wayne State University. While the book purports to explain the mystery of life by examining molecular machines, it does well on the latter but not the former. Hoffmann’s prose is relatively easy to follow and he has uses simple and clear everyday examples to illustrate his main points. In some cases I feel the examples are a bit simplistic, and obscures the more complex and interesting questions. However the book overall does well in giving the non-biologist a good idea of how interesting and complex the molecular machines in our body can be.

The punchline of the book is that Life is based on both Chance and Necessity, a play on Monod’s famous book of the same name. The chance comes from statistical mechanics and thermal fluctuations at the molecular level. The necessity comes from physical law, the physics of chemical interactions at the molecular level. The book does well when the author focuses on describing phenomena at the nanoscale. It gets shaky or simplistic when he slides from physics into metaphysics. A philosopher of science might cringe at some of the metaphysical pronouncements made. Since I’ve started to think about my classes next semester, and in particular how our core curriculum revision will incorporate "scientific inquiry" into our introductory classes, my warning sensor alerts me when I see scientists slide into metaphysical claims.

I almost stopped reading the book after the first couple of chapters but I’m glad I persevered through Chapter 4. Right at the end of the chapter titled “On a Very Small Scale”, there is a section on how magnitudes of energies of a rather wide range of physical processes and interactions converge at the nanoscale and coincide with thermal energy at room or body temperature. I had never thought about this before (and I really should have as a physical chemist). Hoffmann's book led me to a superb 2006 article by Rob Phillips and Steve Quake titled “The Biological Frontier of Physics”. I’ve reproduced the key figure here, but the article can be found at Physics Today. (I highly recommend it!) Hoffmann discusses the examples used by Phillips and Quake, and in my opinion owes much of his book to their work, but in addition he adds a detailed description of myosin “walking” and does a nice job tying this to the Ratchet of his book.

In any case, learning about the interesting convergence of energies at the nanoscale was sufficient motivation for me to read through Hoffmann’s entire book. In all fairness, he does temper the strongly reductionist slant in the early parts of the book with the need to understand emergence, and how both approaches are two sides of the same coin. As scientists who study complex systems, we need to use both approaches, and Hoffmann acknowledges that we are indeed far from the “mystery of mysteries” that is Life. Hoffmann does not go as far as others who have delved into how exactly the second law of thermodynamics and statistical mechanics might work to support systems that maintain themselves away from equilibrium (one definition of life). But perhaps that is neither his intention nor his audience.

I’m not sure how I will use the idea of convergent energies at the nanoscale in my own research projects, but summer is approaching, and I’m looking forward to pondering this issue. But first I need to get through Finals this coming week!

The Knowable Futures of Life

I recently finished reading A New History of Life by Peter Ward and Joe Kirschvink. The book traces the history of earth from 4.5 billion years ago to the present day. It was a good refresher for me to go through the different eras in the Geological Time Scale. Much of what was written was familiar to me from other reading, but there were a few new things I did not know about. I did not realize the Neo-Proterozoic era had been subdivided such that there is now a Cryogenian era from 850 to 635 Mya. I also got up to speed in the recent Ediacaran discoveries. Although I had read Nick Lane’s book Oxygen a while back, I found it interesting to reminded of the close correlation between CO₂ and O₂ levels, and extinctions/evolutionary “explosions”. My knowledge of plant evolution was also rather scant and I found it fascinating to read about the effect of CO₂ and O₂ on the evolution of grasses, trees, flowering plants, C₃ and C₄ species, and more.

Given that I asked my students about the “faint young sun hypothesis” in a recent quiz, it was interesting to read the final chapter in Ward and Kirschvink’s book titled “The Knowable Futures of Earth Life”. The luminosity of the sun has increased by some 30 percent since the Earth’s formation, but it is only going to get brighter and more intense. The result, according to Ward and Kirschvink, will be “loss of oceans [and] hellish conditions, similar to those that exist on Venus.” While Earth is currently well-positioned in a “Goldilocks” habitable zone, this zone will move further out. In fact, Mars will move into such a zone.

Much has been speculated about life on Mars. As the intensity of the sun increases, and temperatures rise, we would expect the melting of the polar ice caps. Unfortunately there is hardly any atmosphere and it is unclear that Mars will be able to support one so all that water may simply evaporate. Could Mars be terraformed? That could prove very difficult. And it’s unclear if there is sufficient geothermal energy to support an underground complex (like Zion in the Matrix movies). It might be easier to build an Elysium-like space-station if our technology actually gets sufficiently advanced.

One of the authors, Peter Ward, is credited as the originator of the Medea hypothesis – that life essentially destroys itself over time, as evidenced by the series of mass extinctions in the past. The authors comment that “it seems appropriate to us to end this book with some comments about the most Medean of all species ever evolved: our own. What will the future be for our own species?” There are a number of speculations made and questions posed in this section, ending with “Is humanity but the builder of the next dominant intelligence on Earth – the machines?” For some reason this makes me want to watch the Terminator movies again. The future may be knowable unless we can go back in time to change it!

Transfer and Teaching Metacognitive Strategies

I recently finished reading Minds Online by Michelle Miller, a professor in the psychological sciences at Northern Arizona University. The book discusses much more than online learning. First the basis for learning is laid out given what we know from research on learning and cognition. Three of the nine chapters in the book are devoted to Attention, Memory and Thinking. In each chapter, the author suggests practical strategies of incorporating the principles discussed into an online learning context. I found that many of the suggestions translate well (with minor modification) into face-to-face classes too. Overall, this is a book I will probably want on my bookshelf. So I’m likely to purchase it after I return my library copy.

One of the things I have been thinking about this semester is how to teach metacognitive strategies to my students. This, I think, is really important for students to get the most out of their own learning. Are there best-practice strategies? Over the years I spend some amount of time in my office hours teaching students how to study effectively, particularly if they are first-year students. (In my first year teaching I was a little shocked by how ineffective strategies employed by the average student.) I have been trying to add a dose of self-reflection strategies to my students’ thinking, but it seems harder for them to “catch”, or I’m not doing a very good job trying to convey how one actually goes about doing this.

In the chapter on Thinking, there is a section on Transfer, i.e., getting students to apply what they have learned to new domains. Prof. Miller writes: “Teaching metacognition does not have to be highly technical or theoretical in nature… [Having faculty] introspect about how they approach material, as disciplinary experts, [is] a way to uncover what they should teach their students about thinking in a discipline… Infusing metacognition into teaching means focusing on the ‘process as well as the product’… and maximize transfer between [old and new problems]”.

Five suggestions are provided by the author. I do some of these things, but perhaps not well. Here’s my self-analysis.

1. Emphasize how knowledge is organized when the original material is being taught. In class, I’m decent at making connections and describing the organizational structure, but I don’t do it as consistently I should. There’s room for improvement.

2. Go for depth rather than breadth. One of the problems teaching General Chemistry is that it is a pre-requisite for subsequent science courses that assume a certain body of knowledge. Therefore I find that I’m often pressed for time just to get “coverage”. I have made some progress, i.e., I’ve cut out a few peripheral things and drilled down a little further in some areas. However what I really need to do is sit down with my colleagues and really revisit our curricular structure and content.

3. Emphasize underlying principals and conceptual structure. The author suggests drawing students’ attention to abstract, general aspects rather than on surface features. In class I’m good at posing questions, listening to students answer, and posing follow-up “why” and “why not” questions. However, sometimes I don’t do this as consistently if I’m trying to cover a certain amount of material, so I’ll need to work more on this.

4. Frequent quizzing. Finally, this is something I do consistently!

5. Tell students why wrong answers are wrong. I get the students to help figure this out for themselves in class and in my office hours. However I’m not sure how much reflection they do when they’re working on a homework assignment outside of class.

My semester is drawing to an end, but I am looking forward to being a better teacher in my classes next semester!

Six-Month Roundup

Now that I’ve been blogging for six months, it seemed like a good idea to take stock of what I have or have not accomplished (or maybe learned) through the process.

I re-read the first six Harry Potter books, thereby averaging one book a month. It’s been a very enlightening experience to re-read the books with a different lens. I learned that when I have an additional motive (blogging), that I do read topics that intersect my blog with a different eye. This also proved to be the case with some of the students in my class. Having an assignment of blogging regularly made them see things and ask questions from the point of view of scientific inquiry. This was eye-opening for some of the students who proceeded to also enjoy writing about their observations. However, for other students, this remained a chore. I’ve found myself flagging through the months of March and April. I’d like to blame it on my bad back problems and the busyness of the semester, but maybe it is just a matter of re-prioritization. I am looking forward to re-reading the seventh Harry Potter book after the semester ends.

Looking back at some of my blog posts, I realized that I read many books about teaching, education and the liberal arts, more so than I have in the past. The blog may have contributed in encouraging me to think more about education systems. Having more involvement in academic administration has probably also contributed significantly to what I spend my time thinking about. I’m very glad that I decided to attempt to overhaul my General Chemistry course this semester because that made me go out and read more about teaching and learning. The three best books about teaching and learning I’ve read in the past six months were Building a Better Teacher, make it stick, and most recently Minds Online.  I need to write a review of this third book, hopefully in an upcoming blog post. It’s sitting right next to my home laptop for that sole purpose!

I was hoping that blogging would help me become a better writer. I’ve learned that I need to be more disciplined in the drafting and editing process. I paid more attention to this in the early stages where I spent time thinking about the best way to craft a blog post. Admittedly I got lazier and was not willing to put in the time to improve my writing. I will have to re-evaluate my time priorities and figure out if I’m willing to put in the time to allow myself the opportunity for significant improvement. I tell my students that they need to put in the time and persevere and then learning isn’t “easy”. I should apply the same principle to myself.

Thinking about the intersection between science and magic, and exploring the world of alchemy, has given me interesting ideas about the intersection of philosophy, history, literature, art and science. To make further advancement in this area would require a significant time investment. I feel as if I’m dancing around the edges thinking about low-hanging fruit, but am unwilling to throw myself into a larger and more significant project. That being said, with the summer approaching, I’ve recently become more excited about my science research projects. It also helps that I have some enthusiastic new research students who will start in late May and early June. I’m enjoying the excitement of potential new “discoveries”. It seems easier to keep doing what I was trained to do (in terms of the science and my methodological expertise) than to branch into a philosophical-historical-scientific project. Maybe I’m just not quite ready, or I haven’t found the right collaborators yet. I certainly don’t have the expertise to pull the different threads together.

Blogging has opened up fun and interesting conversations I’ve had with students, colleagues, and friends & family outside of academia. Some of them read my posts with bemusement. I actually feel that I bring a bit more to a conversation when I’ve spent some time previously thinking about or blogging on a related topic. Maybe it’s just the academic in me!

Finally there is always a time trade-off. I’ve found that what I do in my “leisure” has changed since I started blogging. Some older hobbies lay untouched (but not forgotten). New interests have taken hold. I think that overall this is enriching. Will I continue blogging? I think so. Although I think it is okay if I don’t make my self-imposed requirement of two thoughtful posts per week and not feel guilty about it either.

The Faint Young Sun Paradox

I had been suspecting that the last couple of weeks, students in my Honors General Chemistry class might be slacking off and not preparing adequately for class. I can tell by the increased number of blank stares when I go through something that should have been familiar if students followed the preparation instructions beforehand. There is usually some reading, a few basic problems to try, and occasionally an additional reading or a video to help motivate the topic to be discussed. As the semester draws to a close, students increasingly (as they get busy) try to skip the reading and go straight to the problems only going back to the reading if they can't do the problems.

We had just finished looking at acid-base aqueous equilibria and the plan was to put things together by having the students go through an exercise where they can apply what they learn to study an interesting problem that has both relevance today and in the distant past. It's kind of a neat exercise that I put together (if I do say so myself!) Starting with just four equilibria one can derive a relationship between the partial pressure of CO2 and ocean pH to a reasonably good approximation, and I guide the students through this. They aren't strangers to this sort of exercise. I had a similar guided exercise the week before on deriving O2 saturation curves for myoglobin and hemoglobin the week before after we finished the chapter on chemical equilibrium.

Below are the instructions to my students in preparation for class:
 
To integrate the material you have learned in Chapter [number provided here], you will be figuring out the relationship between the partial pressure of CO2 in the atmosphere and how that affects the pH of the ocean on the early Earth. In preparation for class:

1. Read the wikipedia entry on the "faint young Sun paradox". You should be able to tell me what it is, and how CO2 might help "solve" the paradox.
2. Look up the polyprotic acid dissociation constants for carbonic acid (Ka1 and Ka2), the solubility product equilibrium constant (Ksp) for CaCO3, and the Henry's Law constant for CO2. [Explicit instructions of where to look were provided.]
3. Be sure to bring a device that allows you to make a data table and construct a graph since you will be plotting P(CO2) versus pH for the early ocean.
4. There will be a worksheet in class that takes you through the multiple steps you will need to plot the graph. If you don't finish it in class, it will be turned in as homework at the beginning of next class.
5. Optional: 2-minute animation on the effect of ocean acidification on shell formation of marine life -- we'll be looking at the same chemical reactions, but in much more detail, in class today. [A youtube video link was provided here.]

My class has frequent low-stake quizzes. This semester I am counting the top 20 (for 10% of the total grade) but I give the students 25-30 quizzes throughout the semester. They take place right at the beginning of class and take 3-5 minutes so the students are very used to the drill as they are quizzed 2 out of every 3 class days on average. On Friday morning I decided to check how many of them actually did the reading in #1. My quiz question: What is the faint young sun paradox?

This is a case where if you did the reading you would know exactly what the answer is. If you didn't, you would have no idea. Some students looked completely befuddled, others started writing immediately. I could tell within the first 10 seconds who had done the reading. Grading the quizzes after class was hilarious. (I either give full credit, half credit or zero credit for a quiz so I can grade them very quickly. The students write their answers on an index card.) About half the class easily got full credit. The other half got zero credit with answers ranging from nonsensical to amusing. I got a nice picture of a sun wearing shades, several apologies for not doing the reading, some who honestly said they didn't know, and some that tried to make up scientific sounding but completely wrong answers. However one answer was so amusing it made me laugh out loud and I decided to award half-credit just because it was witty and written in just a couple of minutes at best:

The "faint young sun" paradox originates from ancient Greek philosophy and the concept of existance. The paradox itself questions the ability of our sun to flicker and faint despite being a young sun, son.

[The student spelled existence incorrectly.]

Given I was grading this at the end of the day after a bunch of Friday afternoon admin meetings, this was welcome relief! A nice way to end the week.

And if I successfully get you to look up the faint young sun paradox after reading this blog, I've done my job as a teacher!