Need for Speed

This morning I watched two pre-recorded webinars (on A.I. tools) at 1.5x speed. I did slow down to 1.25x speed at certain portions where I anticipated something I wanted to learn. Why did I speed up the videos? Often there’s dross and filler I am less interested in, so it feels efficient to let those pass by at a higher speed where presumably I don’t do as much cognitive work. I did pause the videos on a couple of occasions to write a note on something I should try out.

 

Post-pandemic, students sometimes ask if my lectures can be recorded (not by them, but by the “system” setup in the classroom). The purported reason for asking is so they can review the lecture again later. The answer thus far has been No. I strongly suspect that if I provided recorded lectures, attendance would drop. While attendance in my G-Chem classes is overall quite good (over 90% most days), the small number of students who consistently miss class on a regular basis do poorly in the course. (Attendance in P-Chem is usually close to 100%, and students fret if they miss class, because when it happens they very quickly realize they have fallen behind.)

 

And in the age of A.I., if I provide recorded lectures, I’m certain the students would watch it at accelerated speeds, and then ask A.I. to provide a written summary. This is unlikely to help their learning although it might give them the illusion of learning. But this isn’t to say that you cannot learn from videos. There are many examples where video is excellent at helping you learn how to do something that physically requires motor skills – if I need to make a minor repair or if I wanted to learn a new cooking technique. But does that extend to learning scientific concepts?

 

With a brief but lazy search, I came across “The effects of lecture speed and note-taking on memory and educational material” (Chen et al, Applied Cognitive Psychology 2024, DOI: 10.1002/acp.4166). Here’s the abstract which also summarizes the work, but the paper is worth reading in full. It’s a small study and thus its extensibility is less clear at the moment, but the results are interesting, and I hope there is further study in these areas. The pre-recorded video materials were on the Paleozoic Era and Microeconomics, and there was a post-test. Interestingly, watching at 2x speed results in a small drop in test results. Taking notes helped.

 

I’m not going to discuss the article details as I’m unlikely to implement video recordings of my classes, but it made me think about the speed of my lectures. My course teaching evaluations have students regularly saying that I go fast. This is true and there are reasons behind it. I won’t hash those out here, but I will say that I tell students how to approach my classes and that pre-reading before class is important to getting the most out of class in-person. While I do pause my speech when I observe the majority of the students writing furiously what I have written on the board, it’s possible I need to provide longer pauses so they can digest the material. The natural punctuating rhythm of in-class student questions helps in this regard. What I don’t know is the quality of the student notes, whether they are taking notes while cognitively engaging the material or just passively copying stuff down. I can’t peer into their minds. For the students who come to drop-in (office) hours, I do get to see the quality of their notes, however this is a small subset and it’s usually the stronger students who stop by. I have had limited success getting the students who need the most help to visit my office and ask me questions.

 

One positive aspect of the speed in my classes is that practically no students are on their phone or surfing the web if they use a tablet to take notes (hardly anyone uses a laptop to take notes during class). The pace means there is less time and opportunity to be distracted, at least visibly so. I can’t peer into their minds so there might be passive, mechanical, note-taking with the mind wandering elsewhere. Students look like they are paying attention, and the lecture classes are interactive. When I break students into groups to work on something and walk around, students do work on my assigned questions, (because they might randomly be called to answer to the whole class) and the time is tightly controlled – they have to work relatively quickly and productively.

 

Actually, knowing the material and learning it well results in a fluency when it comes to taking exams. Thus, my exams are written with time being relatively tight. The strongest students will finish with time to spare; the weakest students will also finish early because they simply didn’t know how to do a problem; but the majority of students might finish with barely time to spare. I’ve read the literature about the supposed “harm” of timed tests, and I’m overall not convinced by the arguments (although I have made modifications to my pedagogical approach after reading some reasonably raised points).

 

I suspect that short-form incessant video watching has changed the way the student today “consumes” information in my classes, certainly differently from the students I had in the previous generation when I started teaching at the university two dozen years ago. Extensive gen-A.I. use among students in the last couple of years has further changed things, possibly drastically. All this keeps my life as an educator interesting as I learn to adapt with the evolving technology that students use and consume. Acceleration, the need for speed, seems to be the new name of the ongoing game.

Discovery Learning: ADOM Edition

This past weekend, I notched my fifth ADOM win with a Drakeling Elementalist who is now #2 on the high-score list. In preparation for today’s blog post, I also replayed a Tutorial game on a new installation to remind myself what tips the system provides to brand new players.

 

What I’ve been musing about is “Discovery Learning”, a buzz-phrase that leverages (in this case simple-minded) “common sense” thinking. In the extreme version, there is no formal schooling for kids. Let them explore and discover the world and learn “naturally” from nature. Natural – good! Artificial – Bad!

 

I don’t disagree that much can be improved about the seemingly artificial settings of today’s classrooms especially for kids who have lots of energy and are bouncing off the walls. But I don’t think Discovery Learning and doing away with formal schooling is the answer. It could work well for some people after they’ve had a decent foundation (acquirable in diverse ways). The media likes highlighting the college dropout who went on to found a tech company and become ridiculously wealthy. They don’t tell you about the tens of thousands of other dropouts who did not become billionaires or even millionaires.

 

In ADOM, the world of Ancardia has rules. The basics are provided in the manual, and I found the tutorial much clearer now that I have 70-80 games under my belt compared to the very first time when I was floundering around. Because I had experience with old-school CRPGs, ADOM wasn’t impenetrable, but many of the rules are “hidden”. I actually did okay getting to my first mid-game character within a dozen games purely through Discovery Learning. Characters die early and often in ADOM. I could sink hundreds or thousands of hours into the game and learn more nuances about staying alive and making further progress towards the end goal, or I could learn from experts who have already traversed the path. I chose the latter, and my enjoyment of the game increased by more efficiently getting over many of the otherwise frustrating barriers that would have killed dozens of characters.

 

The “natural” environment of ADOM is brutal. You might even say they are hostile to learning efficiently. While the tutorial gives you a “warning” when you first enter the Small Cave, you have no idea what that really means. And until you notice or understand how the hostile monsters are generated, you’ll bang your head against the wall trying to get through. You have no sense of how the difficulty level scales. You encounter monsters you know nothing about. You might get cursed or doomed and not realize why it happened and what it means for your character. You don’t know what talents or attributes are helpful and how they might be trained naturally. There is plenty that is hidden unless you know exactly what to look for. I’m not sure how many games it would have taken me to figure out that dropping a potion of water on a co-aligned altar blesses it, and that when you dip a scroll of identify into holy water, you can then read it (if your literacy is high enough) to identify all your items in a single swoop.

 

As a chemistry professor, the natural sciences and math are the areas I am most familiar with. Learning math or chemistry efficiently is very unnatural. If you had to figure it out from scratch, it might take you several lifetimes. (Also, failure is not always productive.) The accumulation of human knowledge has taken lifetimes – small bits of info passed down from teacher to student. The apprenticeship model has been true for a long, long time. It’s far better than having to discover everything from scratch through trial and error, but this one-on-one learning is inefficient and very expensive on a larger scale. I don’t like having forty students in G-Chem; I think I do a better job when I have five or ten or twenty. (At least it’s not four hundred.) But I recognize the efficiency of teaching a group of students. They can also help and encourage each other, which is a plus in my opinion.

 

Becoming an expert requires depth of knowledge and acquiring abstract schemas in long-term memory. Without books and teachers and some very effortful thinking on my part, I would not have the expertise that I now have in chemistry. I can’t imagine getting there through pure discovery. Of course, here I’m caricaturing Discovery Learning, and an advocate would say that no one is promoting pure “throw you into the deep end of the pool and you sink or swim”. They’d say the learning has to be guided. I don’t disagree. But the same advocates caricature current classroom practices, especially what is known as “explicit teaching” as inferior to discovery approaches, or “lecturing” as an artifice and therefore worse than a more “natural” approach. In reality, one balances multiple aspects when considering pedagogical strategies.

 

My current ADOM character is a level 13 gnome druid. I just made it to the High Mountain Village although I was not able to retrieve the waterproof blanket on the way because I understand how the Small Cave works. The fun in ADOM is that the dungeon layouts (and the game is a dungeon-crawler) are randomly procedurally generated, so each game feels quite different. Your character’s inherent skill set provides even more variation. I think this is my third druid (the previous two did not make it past level 10 before succumbing), and I’ve learned how to balance spellcasting with traditional weapons. I also now know that most animals are generated friendly, and switching my alignment to Lawful means that I have a reasonably good chance of completing the Rolf Quest and getting the ring of the master cat, provided I don’t die in the Pyramid or somewhere else. The balance of some discovery and some guide-reading, in my case, has led to maximum enjoyment. I still do bits of both when I encounter something rare (statues and artifacts) or exploring a different aspect of the game, and I wouldn’t do this any other way.

Cybernetics Informing Learning?

I stumbled across an interesting blog post connecting Ross Ashby’s principles of cybernetics to how one designs questions to probe student learning. I have some familiarity with the cybernetics principles for thermostat design; several years ago I was reading papers using this to analyze a complex prebiotic chemistry problem adjacent to one of my research projects. I had not, however, considered how this affects instructional design. Given that A.I. methods are heavily encroaching on education, I think the article highlights some of the potential pitfalls of a computerized system that supposedly personalizes learning.

 

The word “system” is important here. The blogger, Carl Hendricks, has this to say: “An instructional system can only regulate what it can detect and many learning environments rely on a channel of extremely low capacity: correct or incorrect [which] carries almost no information about process. It does not distinguish decoding from guessing, understanding from memorisation, reasoning from elimination.” Prior to the present LLM burst, the computerized learning systems relied on multiple choice questions (MCQs) or True/False questions. A subject matter expert designed these questions as a proxy to probe certain learning goals, usually atomized Taylorian-style. In the last decade, this morphed into “adaptive” systems that mixed-and-matched questions depending on whether a student got this right or wrong.

 

I think that expert-designed questions and answers for the computer-distance-online-learner can be effective to some extent. Writing good questions and answers is time-consuming and challenging. It’s also why lazy me doesn’t use exam MCQs. It’s faster for me to write a short-answer question and then evaluate the student answers, i.e., the time it takes me to grade the student answers is less than the time it would take to design really good MCQs. I’ve tried getting the LLM to help generate good answer-question pairs but right now the results are low quality. I expect they will improve with time; I might even be able to train one on a limited chemistry corpus.

 

But while expertly-designed individual questions may be quite good, stringing them together in an A.I. “adaptive” system degrades that goodness. This is why after trying out some pre-LLM systems, I never selected the “adaptive” option. Hendricks mentions the drawbacks of not coming up with good questions and answers that really get at what you want the student to learn, and the additional problem of having a regulating control system that supposedly personalizes the learning. He writes that measuring such performance is “fragile” because “the system ensured that answers were right often enough, but it never ensured that the right thinking had occurred. [It is] informationally impoverished; and no amount of pedagogical enthusiasm can compensate…”

 

LLMs continue to push the personal tutor aspect; I should say A.I. tech companies are pushing heavily because they need revenue streams. Last month I found that the current LLMs do a better job generating chemistry questions and answers compared to previous ones. I see more nuance and better accuracy overall. And the “voice” of the LLM tutor leans heavily on trying to sound helpful, offering follow-up information and more. One thing I learned is that I can make an effort to sound more helpful when students ask me questions, so the LLM had at least that aspect to help me improve. But the LLM doesn’t always (and maybe not often) offer what might be best in actually improving learning. It’s good at helping the student feel good. But that’s because it was designed to do so. It wasn’t designed to be an expert tutor.

 

A thermostat does just one thing – regulate the temperature by measuring the ambient value and then turning on (or off) the heating/cooling device. Even with its narrow purpose, the mechanics of designing a good thermostat is trickier than it looks at first glance. The business of teaching and learning is more nebulously defined in purpose and certainly much more complex. Cybernetics may be a starting point to think about adaptive tutors but there is far to go before it will replace an actual human expert in terms of quality. Pessimistically, I predict that overhyped adaptive tutors will degrade the desired quality to a low common denominator. Hendricks writes: “Learner ingenuity will always exceed designer foresight; there will always be shortcuts that were not anticipated, strategies that were not mapped, paths that were left open by accident. Requisite variety is an asymptote, not a destination.”

 

I’m reminded how amazing it is to learn something as a human being. I don’t pretend to know exactly how it happens especially in glorious moments of gestalt “aha!” understanding. Present neural networks underlying LLMs are not like our brain or our mind or our sense of self. As a computational scientist, I have some vague and wild ideas of how to improve on this. I’m sure others like me have such thoughts and hence I expect over time that LLMs will continue to improve. Whether they will eventually achieve the quality of the hype remains an open question.

The Feeling of Knowing

To err is human. To admit to erring… well, that’s difficult. Like most people, I don’t like the feeling of realizing that I’m wrong. I always think I’m right (I can’t help it!), but I don’t think I’m always right. Past experience confirms that I do err; it’s consistent but I can’t predict when it will happen. And I feel I’m right… right up to the moment that I’m proven wrong.

 

The subject of Wrong-ology is taken up by Kathryn Schulz in her book, Being Wrong. Our minds are funny things and the way we learn and remember things is much more complex and mysterious than we imagine. We think, as Plato suggested, that memory works like a wax tablet: “Everything you experience, from your own thoughts and sensory impressions to interactions with others, creates an imprint in that wax… an unchanging mental replica of the events of the past, captured at the moment they occurred.” This may contribute to that feeling of knowing, even when in gross error.

 

As a professor, I’m well practiced at professing. Students think you’re more knowledgeable and know what you’re talking about when you present the material confidently. That’s not hard to do because I feel that I know the material. Even when I don’t know it as well as I should, I still present it confidently. Fake it till you make it. Did I do so when I first started teaching? Was I more diffident back then? Honestly, I don’t remember. I’ve learned not to trust my memory even if I feel I can visualize it in my mind’s eye. My spouse provides me a very useful signal when I might be professing with confidence about something I know little about; she says: “You say that so confidently”. That gets me to chuckle, stop, check, and think.

 

Schulz discusses medical cases of brain issues where patients confidently describe or explain something with no correspondence to actual reality. And they seemingly believe it. This is known as confabulation. Here’s how Schulz describes it: “Imagine, by way of analogy, that each of us possesses an inner writer and an inner fact-checker. As soon as the writer begins devising a story, the fact-checker gets busy comparing it with the input from our senses, checking it against our memory, examining it for internal consistencies, thinking through our database of facts about the world, and, once we utter it, gauging other people’s reactions to assess its credibility… When the fact-checker falls asleep on the job, however, our theories about the world can become wholly unmoored from reality. All of us have experienced this, because the one time our fact-checkers reliably fall asleep is when we do, too. Think about dreams again for a moment, and about how weird even just the averagely weird ones can be… Now, two bizarre things are going on here. The first is that your brain is generating representations of the world that are only lightly tethered to the real, or even to the possible. The second is that you are completely untroubled by this fact.”

 

Being surprised is a good wake-up call to discovering error. You’d think that by now I would have gotten used to being surprised every time I err. But I am surprised every single time. Consistently, yet unpredictable in when it will happen. I’m heartened when Schulz writes that saying “I don’t know” is a good sign of brain function because in some forms of dementia, the fact-checker falls asleep and confabulation ensues. It also turns out that being confabulatory is part of how the human brain works. It’s an engine, possibly the engine, of creativity and imagination. I can think about and imagine things that are not real. I can make mental models of things that are abstract or invisible (which I must do frequently in thinking about chemistry). Our minds have adapted to come up with quick instinctive solutions, not always thought through, that do serve us well on many an occasion. The feeling of knowing allows us to act quickly when needed.

 

When grading exams, I still get surprised by the occasional confabulatory explanations of students. A student who has no idea what’s going on is yet able to come up with a fantastical story involving throwing together chemistry concepts completely untethered to reality. It doesn’t happen often, but it’s interesting for me to read these “answers” and try to imagine how a student came up with them. I wonder if that student had the confident feeling of knowing. But actually didn’t. Not knowing what you don’t know isn’t a great situation to be in.

 

Reading Being Wrong has made me a little quicker to say “I was wrong” in my classes when I make a mistake on the board and it is pointed out by a student. I apologize to the class, then thank the student for paying attention and being brave enough to tell me so that I don’t mislead the class any further. While it doesn’t happen often, I feel that as I age it has ticked up in frequency. I’m not as sharp as I used to be, perhaps. Or maybe I am less well prepared because I’m overconfident in the feeling of knowing, having taught the subject matter multiple times over a couple of decades. But if I actually learn something from my errors, that’s a good thing!

Words and Pictures

I’d read several papers by Richard Mayer on the dual-coding model: Learners have two channels for processing incoming information, verbal and visual. Over time, this was combined with insights from cognitive load theory and learning more about the brain and how memory works. Mayer now calls it the cognitive theory of multimedia learning (CTML) and I read a recent review that goes through the history of how they got there and where to next. The citation is Educational Psychology Review (2024) 36:8, DOI: 10.1007/s10648-023-09842-1. I very much enjoyed the personal insights the author shared about his research journey. Each heading is listed in the bullet points below followed by my thoughts.

 

1. Theory Building Depends on Intellectual Curiosity. Mayer became very curious about how to improve teaching for “transfer” – being able to apply something you’ve learned usefully to a new situation. He did this by first narrowing the issue to the effects of multimedia. I am curious about a lot of things, but I haven’t had the discipline to really narrow my focus, and as a result I remain a dilettante on a broad range of topics. As a result, I haven’t made significant theoretical contributions in my field even though I’ve learned a number of interesting things about a number of interesting systems I’ve studied. It seems I scratch the surface, pick the low hanging fruit, and move on. Maybe I need to change my approach.

 

2. Theory Building is Grounded in Old Ideas. Mayer discusses his reading of classic works in his field. I find reading the historical underpinnings of my research and teaching very enjoyable from a learning point of view. I hadn’t thought much about building new theory off the old ideas in a systematic way. Something for me to consider.

 

3. Theory Building is Not a Straight, Planned-Out path. Mayer relates how he usefully breaks down interesting questions into “shorter 2- or 3-year plans targeted on specific research questions”. This led him to the multimedia principle: “people learn better from words and pictures than from words alone”. I’ve known about this, and it’s common in the natural sciences, to have lots of pictures. I’ve also learned that the pictures I project on the screen should not be cluttered with text as I verbalize my way through an explanation (Mayer’s coherence principle). After doing so, I then write things on the board for students to have good notes, at least in G-Chem. (I’m worse at it in upper division classes.) Mayer also writes about pursuing fruitful paths; I also do this research-wise but I likely move too quickly away from something that looks like it would take more work. I’m lazy.

 

4 & 5. Theory Building is an Engineering Problem [and] an Iterative Process Involving the Persistent Interplay Between Research and Theory. By this Mayer means that it requires tinkering, to make something work better, and going through a development cycle where theory leads to research experiments, the results of which feed back into theory. Mayer discusses fostering generative processing: “motivating the learner to actively engage with the material”. This is a weak area for me. I’ve relied on my enthusiasm for my subject area (which students recognize and comment positively on) but this is likely not enough. My activities mostly require the students to do analysis, but few of them ask the students to be generative. This needs more work on my part.

 

6. Theory Building Depends on Persistence in Collecting New Research Evidence. Sounds obvious, but this requires hard work which is not my strong suit.

 

7. Theory Building is a Team Activity. The days of the lone theorist making substantial novel discoveries are long gone. A good and fruitful collaboration requires work to sustain it, and since I’ve already admitted I’m lazy, my collaborations tend to be short-term and specific, and not dedicated to theory building in particular. Maybe I need to change that.

 

In the middle section of his article, Mayer discusses his “inching towards a visual representation of the theory”. This is very appropriate given what he studies. He starts with simple flowcharts that slowly build up to what has become a compact and useful picture. Here’s Figure 8 from this article. You’ll have to read his article to get all the details, but once you know what each of the boxes and arrows represent, it summarizes the theory in a single uncluttered visual representation. 

 

There’s also a useful Table with his fifteen principles of multimedia instructional design along with their effect sizes from experiments. I already follow some of these, given my prior immersion into cognitive load theory. Here are some that I hadn’t thought about much or haven’t incorporated yet.

·      Presenting material in user-paced segments rather than a continuous unit. I don’t do this well and I need to improve how I cue different segments in class.

·      Sometimes I assume students know definitions and terminology that they don’t and/or present them in an order that confuses them.

·      Apparently in multimedia, using a conversational style works better than a formal style. I don’t know where I am on this spectrum and should reflect more on this.

·      If you’re onscreen as an instructor, high embodiment helps. I take this to mean that being a disembodied voice talking though slides is inferior. In our pandemic all-on-Zoom year, my camera showed me writing on a large white board, and I would sometimes step out of frame so that more of the board would be visible. At some point we’ll have another pandemic and I’ll have to think about this.

·      Generative learning activities help. I mentioned this above; I should design more of these.

While I don’t use 3D immersive virtual reality, apparently studies show that students don’t necessarily learn better compared to a corresponding 2D screen presentation. The effect size of this was small.

 

I have a sabbatical coming up that will allow me to think more deeply about some of these issues. A third of my sabbatical proposal had to do with pedagogy but mostly related to adapting machine learning and data science. And there were a whole bunch of other things in my proposal which are dilettantish, so maybe what I should be considering is how to narrow what I’d like to accomplish into specific questions and design specific activities ahead of time instead of my ad hoc muddle-through approach. But meanwhile I should look over my upcoming class materials and think about the words and pictures and whether I can improve optimizing student learning.

Overly Efficient

I was forced to get a new phone; the old one started to precipitously decline and would randomly restart. I’m a creature of habit and I’m easily bewildered by new technology (shocking for a computational chemist, I know!); I’ve only owned two smartphones in thirteen years (an iPhone5 and a first-generation SE). To minimize having to switch again in the next five years, I decided on an iPhone17. I was dreading making the switch.

 

The Apple store at my local mall was quite busy when I arrived; eventually someone was assigned to help me. I said I was interested in getting a new phone. The employee wanted to know which model, assumed I knew all the pros and cons, was curt in her responses, and I felt rushed through the process. I was nervous about moving data from my old phone to my new one, but was told it would be easy and I should “just follow the prompts”. A new phone was given to me in its box, I tapped my credit card, and I was done. The employee moved on to the next customer. Purchase completed, but anxiety heightened.

 

I stopped by the T-Mobile store for help getting the SIM and info in my old phone transferred to the new one. There were no other customers at the store, and the employee was relaxed and friendly. I said I was feeling anxious about the challenge given my old phone had a very old OS which suggested a more arduous process. But the employee helped me through the steps, which turned out to be short and easy. The phones, placed next to each other, did their info-sync dance. He then patiently showed me some basic moves for my new phone (when to use buttons and how to swipe for different options). He empathized with the challenge of switching to a new outlay, and answered my very basic questions without judgment of my ignorance. It was a very positive experience and reduced my anxiety substantially.

 

The epiphany I had after these two contrasting customer experiences is that I want my students to experience the second one when they visit me in office hours. However, more often than is warranted, they get the first one instead. The problem is me; I’m overly efficient, and sometimes the student feels rushed through a process. If the student was feeling anxious by their ignorance, I haven’t allayed the student’s (usually unspoken) concern. I expect a student to have done some background work and have their questions ready, and I answer them efficiently, especially if there is more than one student in my office. I ping-pong amongst the students so they get their answers efficiently while making sure no one student has to wait long in between. Students have busy lives, many busier than mine, and I don’t want them to experience long wait times if they visit. If there’s only one student in my office, I don’t have to ping-pong but I do multitask and work on something else in between their questions, which I suppose is me trying to be efficient with my own time. But from their point of view, the vibe I’m giving out might well be “I’m a busy person, so get on with it.” Overly efficient.

 

In my student evaluations, I get the occasional comment from a student that feels intimidated about asking questions in office hours and feels I was being dismissive in my answers. It’s true I expect the student to come into office prepared with their questions after working through the material; that’s something they should learn to do as they prepare for the working world. I also have a philosophy of not spoon-feeding; I sometimes respond with a question or ask a student to look at their notes to see the definitions or examples we covered in class. I want the student to understand that they have to put in the time and work to chew over what we’ve covered and not just say “I didn’t understand anything in class” and try to get me to go over it all again a second time. The vibe I’m giving off? Like the Apple employee. Quick, curt, efficient, and expected me to know what I was looking for before I came in.

 

Not all students feel this way. Some like the way I do things and say so in the student evaluations. They felt their questions were answered, that I was helpful in office hours, and felt that I cared that they were successful in the class. They said I was accessible, even when other students were in my office. But these are likely the students who were not struggling excessively with the material, had a reasonably good background coming into class, and were stronger academically. They liked the depth, my varied explanations, and felt that the way I organized the class set them up for success. In contrast, the students who had negative experiences comment that I go over the basics too quickly, I’m not good at teaching beginners, and that I assume prior knowledge they don’t have. How I felt at the Apple store likely mirrors how some of them were feeling. So while I have good reasons for the way I structure my class and office hours, my recent customer service epiphany tells me that I need to make some changes so that the anxious students who feel they are clueless in class feel more welcome to ask questions and feel like they were actually aided.

 

My mantra to myself this semester is “don’t be overly efficient”. When you’re old and set in your ways, it’s harder to change. But that’s not an excuse; rather it’s a challenge I need to overcome. I know that I will revert to my efficiency (hopefully not too often) during office hour visits or during my interactions in class, but I need to keep making the effort to help the struggling students feel welcome. They’re the ones that need the most help, and if they’re not coming to my office, that’s a problem and I’m not helping them. I don’t need to be overly efficient with my own time, and maybe my time-log (which was likely excellent to have in my first several years as a professor) has slowly compounded my overly efficient behavior. I’ve been successful partly because I was efficient, and I want students to learn to be efficient in managing their time and learning. But more importantly, they are students, and still learning. I need to make accommodations and at least not give out an “impatient” vibe in my interactions with them. I think I’m being efficient. They think I’m impatient and that they’re not worth my time. I need to work on this because the whole reason I became a professor at a liberal arts college (and not a research-intensive university) is because I want to spend most of my time teaching students.

 

I tell students that office hours (I’ve called them “Drop-in hours” for the last five years) are my favorite time in my workday. It’s when I feel I actually help them individually because it’s harder to do so in the classroom with many students. I tell them I look forward to seeing them, but it’s clear some don’t believe me and my over-efficiency vibe dissuades them. I feel that fewer students come to my office in the age of A.I., where a friendly chatbot is always there to answer their questions as many times as necessary and makes them feel good while doing so. I’m no chatbot; I should be better than a chatbot (I’m certainly more accurate and I actually care about their learning as a human being). My goal this semester is for more students to feel that I am accessible, especially the ones that should be coming to my office to ask questions. Maybe my new phone can be a visible reminder to be more like the helpful T-Mobile employee and not be overly efficient.

Biochem Round 2

Teaching biochemistry for the second time this past semester was not as time-consuming as the first time two years ago. I spent 3-5 hours per week on class prep and updating the materials which was three times less than my first run. This was significantly more manageable given I had two larger sections of G-Chem 1 using a new textbook. For Biochem, I did not make large-scale changes to the course. The topical flow was similar and I mainly updated the slides and study guides. I made some changes to the in-class computational activities, exclusively using the Molstar/PDB viewer (and skipping Pymol). I added a protein-folding-prediction exercise given the ubiquity of AlphaFold-like tools. It also has a nice wow-factor!

 

This second pass, I was able to clear up some errors I made and confusion on my part about some of the more complex enzyme regulation involving kinases, in particular FBPases. I streamlined the enzyme kinetics so it would be less heavy math-wise, and I think I did a better job with carbohydrate nomenclature without getting stuck in the weeds. Those are the positives. The negatives are that I likely went faster and had a little more information on my slides when I should likely have done the opposite. I also went into more chemical detail because a third of my class were chemistry or biochemistry majors; in contrast I had less than fifteen percent of them the first time I taught it.

 

My class was fifty percent larger this time around, simply because there were more students enrolling in the course as numbers have rebounded post-pandemic. This probably made the largest difference because it means I help each student less individually. This was certainly true during in-class activities where the students work in pairs or small groups and I circulate. The majority of students never came to office hours, which didn’t help matters. My end-of-semester grade distribution was much wider and included some D’s and more C’s, and there was a surprising amount of nonsense answers on exams. That being said, many of the students still did well and two-thirds were in the A and B range, unlike the first time around when ninety percent earned A’s and B’s. It was an unusually small class and I was likely paying lots of attention to the students and their learning. By spending less time on the metacognitive aspects of my own teaching and focusing much of my time on G-Chem, I think I did a poorer job overall.

 

The end-of-semester course evaluations were not surprising. On the Likert scale questions, my ratings went down – as expected for a larger class with more students not doing so well in the course. There were the usual comments about the speed at which we went through the material and the density of the material. A couple of students thought the twice-a-week format (with two longer rather than three shorter classes per week) was exhausting, and I see their point even with my three-minute break mid-class. Students found the study guides the most helpful; again not surprising. The chemistry and biochemistry majors liked my chemical emphasis and details. The non-majors did not like it. One made comparisons to the other sections which had “more MCAT applications” and another felt that while the other sections “skimmed through a lot of topics”, our class “felt like we learned the whole damn book”.

 

I don’t know when I will get to teach the class again. Recent staffing changes in my department might preclude my teaching it again anytime soon. If there is a next time, I would consider not using a standard textbook now that I am more comfortable with the material. One problem with following the textbook somewhat closely (which is a reasonable thing to do when you’re teaching something for the first time or two) is that you can get lost in the details and forget the big picture. A couple of students commented that this is how they felt about my class. I think instead of opening with review of G-Chem concepts and launching into amino acids and proteins, maybe I can start with some big picture metabolism (not the weeds) before getting into the building block molecules. There’s a logic to biochemistry and I’d like the students to see this. I thought I was trying to emphasize this, but many students found these details bewildering possibly because I had not spent enough time on the big picture or I was too abstract.

 

My self-rating for Round 2 is that I was overall mediocre; I’m not sure I did a better job teaching the second time around even though I was clearly more comfortable with the material. Perhaps that was the problem; I let the curse of knowledge slip in, and spending less time on thinking about the class showed.

Student A.I. use: Fall 2025

The last two times I taught G-Chem 1, I briefly told students how a generative A.I. such as ChatGPT can be useful and what some of its limitations are. Last semester (Fall 2025), I made no mention of A.I. use in any of my classes until the last week of the term. I surveyed the students asking if they used A.I. in my class, how so, how often, and if they found it helpful. The questions were open-ended and students could answer (or decline to do so) in any way they wish. I prefaced by saying that I had no problem with A.I. use, and that their responses would help me provide guidance to my future classes. I taught two sections of G-Chem 1 and one section of Biochem. My musings on the results are mainly focused on G-Chem because of the larger class sizes.

 

In G-Chem, 12% of students said they did not use any A.I., while 88% did so. ChatGPT was by far the main source, with Gemini a distant second. (Other apps got only one or two mentions.) Only a small proportion of students said they used it a lot. Most used it sparingly or occasionally. A.I. was most often used shortly before exams (in conjunction with getting answers to my study guides) and on the stoichiometry unit where students wanted help on step-by-step calculations. From my limited tests, GPT-4o does noticeably better on stoichiometry than GPT-3 (which wasn’t very good) in providing a correct solution, although typically a verbose one.

 

Interestingly, a few students used the chatbot to recommend youtube videos to help them understand a topic. (Many students just use Google or go straight to youtube to look for such videos.) Most students said they found it helpful in “explaining” concepts or how to solve problems. Several students specifically said they used it to generate practice problems or to quiz themselves. One student said it helped them “decode” their notes and explain it in a simple way. Students said it was particularly helpful when they missed class, one even saying “I didn’t need to go to office hours… it gave me the answer from anywhere I liked.” While the majority of students said they found ChatGPT useful, a handful did not.

 

A number of students provided specific caveats in their usage. A student writes: “I would strongly recommend not heavily depending on it for homework, as it ends up being more harmful than beneficial. You must know how you obtained your answer, not just copy and paste.” Another student: “These models are constructive for learning as long as you use them productively and have them guide you instead of answering for you.” A student notes: “It was helpful, but some ideas it presented contradicted my notes, so I am not sure how accurate it is.” Another student: “While not always correct, I felt that it would usually get me started in the right direction to finish understanding the topic or solving the question on my own.” Interestingly, the students who made these types of comments were almost all students who earned A’s or B’s as their final grade. Also noteworthy, the 12% of students who did not use A.I. also earned A’s or B’s. (The average grade was in the C+ range so slightly less than 50% of the students earn A’s or B’s.) Of the students who used it sparingly or rarely, again these were the A or B students. This is perhaps not surprising. The students who knew the material felt less of a need to use A.I.

 

Since the best use of a generative A.I. is to generate test questions and study guides, I’m glad to see many students mention it in this way. Even more use it for explanations or answers which is more hit-or-miss, but I’m glad that students noticed this. Here’s one thoughtful student comment: “When it comes to studying equations, ChatGPT was very helpful because it showed me step-by-step how to solve it. I also used this model to create practice problems for me. In terms of elaborating the material from class, it was moderately helpful. It mostly gave me vague explanations.” This student also thought it was a limitation of the free version and mused that if they had used a paid version they may have had better results. One student would load the study guide in and then ask ChatGPT to provide timed quiz questions so that the student would feel like they were in an exam.

 

In Biochem, I saw similar trends: 15% of the students did not use A.I. (All three earned A’s and were among the top five.) There aren’t many math-related or calculation questions in Biochem so most of the students used it to clear up things they weren’t sure about, again usually pertaining to the study guides or my lecture slides (which I provide to the students). Since this is a smaller class, I’m not sure if any trends are significant.

 

My takeaways: Students are going to use A.I. in a chemistry class regardless of whether you have a policy or not. The majority of them already do so and feel that it is helpful, so they will keep doing so. The academically stronger students use it less, but likely because they feel they understand the material in class and are able to solve problems without outside help most of the time. Many students leverage the generative capabilities of a Large-Language-Model A.I. to generate test questions although whether they are generating sufficiently complex questions is less clear. Some students notice the weaknesses of A.I. answers yet still find it helpful as a guide. Students think A.I. helps to “simplify” some concept they are struggling with. Whether or not it is over-simplified is less clear. Students still gravitate to video explanations to supplement the text explanations of A.I., and youtube remains a key source for students.

Concreteness

As a theorist, I’m very comfortable thinking abstractly. I believe this has helped me gain expertise in my field, evidenced by being able to “see” the deep features when problem-solving, and not be distracted by surface-level features. I also believe that one of the biggest challenges of being a teacher in my area of expertise is the curse of knowledge. I cannot “unsee” the deep features of chemistry, but neither can I bestow my mind’s-eye-sight to the novice. It’s not a concrete gift I can give.

 

My job is to help move students along the path from novice to expert. I’m trying to help students see chemistry the way I do. I have a limited amount of time to carry out this task. Similarly, my students have a limited amount of time to learn a body of material before they take the final exam that assesses their knowledge. My strategy is to make sure students know definitions and problem-solving protocols. Then we go through several different examples where I try repeatedly to point out the common deep structure of the problem even though the surface features are diverse. One challenge is baseline knowledge. The expert has tons of it; the student has little. When you don’t know much, you can only grasp in vain at the surface features that seem more concrete, even though they are less important for solving a problem.

 

I’ve been wondering if, over the years, my theoretical bent has quietly asserted itself more and more in my pedagogy, favoring the abstract over the concrete. Yes, I do want students to be able to think abstractly. This is particularly important in chemistry where understanding what is happening in the tiny nanoscale regime requires abstract imagination. We can’t see atoms or wavefunctions or chemical bonds or dipole attractions. Chemists are always imagining what is invisible to the naked eye, because the heart of chemistry comes from making and breaking chemical bonds. I talk about balls and sticks and springs and waves. I ask students to imagine such entities. I draw graphs. I write equations. I try to include real-world tidbits of chemistry in the mundane that’s all around us. I find myself excited just thinking about such things. But they are all still in my mind’s eye.

 

Do the students see things the way I do? They can tell that I’m knowledgeable and enthusiastic about what I’m teaching (evidenced by comments in student evaluations). But this doesn’t mean they are learning how to think chemically. Humans haven’t been around for long, and for most of human evolution, learning has been visceral. Concrete. Physical. Not so much in the abstract realm. I think I need to bring more concreteness to the teaching and learning of chemistry, and counter my strong bent towards the abstract. Not that the abstract is unimportant – it’s still vital! – but to do better in helping novice students learn chemistry. The best science writers who are able to convey complex ideas to their novice readers employ the visceral in their language. Blood and guts and more. Your mind’s eye is not the only think activated. You can almost feel, smell, taste even though you’re just scanning words on a page. There’s a concreteness to it.

 

How will I remind myself to keep working on this? Over time I am likely to revert back to emphasizing the theoretical and abstract. Maybe I need a (small) concrete block. A physical brick might be sufficient. I could muse more about this topic, but instead maybe I should take the good first step of getting up off my chair and away from my computer and locate said concrete brick.

Optimizing Learning and Attention

“Learning is the slow, ponderous and beautiful Galapagos tortoise, and online content is the invasive predator which will inevitably drive it to extinction.”

 

This quote from Daisy Christodolou’s article (“Why education can never be fun”) really struck me. Addictive online games and videos, she argues, only need to optimize in one dimension (fun!) while apps which might actually increase learning need to optimize in two dimensions (fun and learning). Sometimes the two parameters oppose each other. For many learners, this will be the case when learning material involving math. Your most engaging learning app will never beat the app that only needs to optimize for holding one’s attention. She writes: “However much fun you make learning, someone else will use the same techniques minus the constraint of learning. You are in an arms race where you have one arm tied behind your back.”

 

The subject that I teach, chemistry, is hard. While there is some math involved in the introductory levels, what is more challenging is the abstraction of having to juggle three aspects simultaneously – known as Johnstone’s Triangle. Chemistry is abstract by nature. We’re trying to explain everything in terms of tiny things that we can’t see. Hence, we have to think about chemistry using models. None are complete-in-itself; expertise in chemistry involves fluidly moving amongst a panoply of such models. This is not easy for the novice learner.

 

As more tasks become facile with the aid of technology, we humans who outsource our thinking to such machines will become less adept at the basics. In many instances, that might be okay. I have no interest in going back to the stone age, and I’m glad I was not born a century ago. I like my technology-aided creature comforts. I even blog to offload some of my cognitive effort. But something is lost in the process. It’s okay if what I’m losing isn’t crucial, but if it’s something important such as basic numeracy, facility with language, concentration skills, or thinking deeply and actively, then this is a problem. It’s possible that humankind is heading towards a general idiocracy with a small number of elites controlling the levers. Or it could be worse – the oligarchs might be just richer and power-wielding members of the idiocracy.

 

Learning science and math is not easy, but I think it is important to understanding the world we live in. It will sometimes be a slog. No pain, no gain. As educators, we should try to make our subject matter interesting and relevant, but there is only so much you can do to gamify your course before running into the hard reality of actually learning difficult material. Passively consuming short-form videos from creators who are optimizing eyeballs may give you a false sense that you know something. But that knowledge might be superficial at best, or possible misleading, or simply wrong. And if you don’t have the basic knowledge, you won’t be able to tell when you’re consuming crap.

 

After thinking about this, I took a moment to think more carefully about some seemingly basic concepts in chemical bonding that are much more than meets the eye. I needed to remind myself of things that I had read a year or two or more, but have since forgotten because I haven’t practiced the effortful cognition needed to retain some of these ideas. But if I don’t keep making the hard effort, I will slowly but surely be joining the idiocracy and not even realize it.

Educating AI

One reason my blog writing has fallen off the past year – I’m ambivalent about bots scraping my data to train AI models. But honestly, I’m not that great a writer, and it’s not like the bots are mining gold. I just need to get over myself and keep sharpening my writing practice, be it on this blog or elsewhere.

 

I just finished reading The Alignment Problem by Brian Christian. While the issue of AI ethics and the dangers posed by advanced AI are the main theme, what I spent time mulling over was comparing the educating of AI with the educating of human students. There are differences between human brains and machine learning neural networks, but the bigger difference is the wetware of the entire human body-organism, which cannot be separated into dry hardware and software.

 

Christian launches the historical story with Skinner’s behaviorism, Turing’s computing machines, and the neuron assembly of McCulloch and Pitts. (I didn’t know Pitts was such an enigmatic character until reading this book!) This is the framework of reinforcement learning. The reward hypothesis states that “all of what we mean by goals and purposes [is essentially] the maximization of the cumulative sum of a received scalar reward”. Shoot for the high score! Not surprisingly, Atari and other early video games were utilized in the training process. (I also learned that Montezuma’s Revenge, a game I played in the 1980s, is particularly tricky for an AI to get good at and represented some sort of gold standard.) What made the world pay attention was when AI beat grandmasters at Chess and Go.

 

I appreciate Christian going through the challenges of any training method. (He also carefully distinguishes reinforcement learning from supervised and unsupervised learning.) These include the problem of the terseness of a scalar reward or punishment, compounded by a delay in knowing that a much earlier blundering move may have cost the game. Turns out “reinforcement learning is less like learning with a teacher than learning with a critic. The critic may be every bit as wise, but is far less helpful.” There’s an interesting story on the “dopamine puzzle” that leads to a learning model (known as temporal difference) that what’s really being valued is the “error in its expectation of future rewards”.

 

The most interesting part for me was Chapter 5 (“Shaping”) on the Problem of Sparsity. Essentially, “if the reward is defined explicitly in terms of the end goal, or something fairly close to it, then one must essentially wait until random button-pressing, or random flailing around, produces the desired effect. The mathematic show that most reinforcement-learning algorithms will, eventually, get there…” but it’s inefficient and takes too darn long. The solution is to put together a Curriculum. That’s what we do as human educators. I break down the learning of chemistry into steps; I set tasks for the students; I try to motivate them; and there’s a rewards system in terms of points and a final grade. But creating the right incentives in AI training turns out to be quite tricky. Specifying certain steps along the pathway often does not have the desired outcome. Evolution has had hundreds of millions of years to shape humans, dolphins, elephants, and octopi, all naturally intelligent creatures among many others.

 

Can you get beyond external reinforcement strategies? Can you build in intrinsic curiosity into a computer? Can you value novelty? There are some clever tricks to do this. OpenAI (now famous for ChatGPT) is profiled for their early efforts working on Atari-arcade-like games. Can we learn from how humans and apes learn? Can computers learn through imitation? Do they learn the same way? I learned that human children in some situations over-imitate compared to chimpanzees; “children are from a very young age, acutely sensitive to whether the grown-up demonstrating something is deliberately teaching them, or just experimenting.” Why does this work? It “allows the student (be it human or machine) to learn things that are hard to describe.” The OpenAI folks managed to get an AI to beat Montezuma’s Revenge by watching YouTube videos of many human players.

 

This may be why taking students through worked examples, then letting them try simpler problems, before adding complexity to a more sophisticated problem is a pedagogical approach that works well, at least for the subject of chemistry. Many of these principles came from folks doing research into teaching and learning math. There’s also a tricky balance between intrinsic and extrinsic motivational approaches. It’s not that one always works better than the other. I’m not sure that final grades, which I assign based on numerical scores, are the best value function that most of my students strive towards. I understand that grades loom large for increasingly stressed students in what they perceive to be a global cutthroat career market. My generation did not experience the pressures they are facing now. With AI chomping at their heels as a competitor, the business of educating AI may be existential for them, even if they don’t realize it yet.