Nations: Rise and Fall

I recently finished a full six-player game of History of the World and found myself pondering the final status of the world. How might history change if certain empires did not arise? The game ends about a hundred years ago at the dawn of the twentieth century. Kaiser Wilhelm II of Germany gets the final turn. In a full six-player game, six empires (randomly drawn) do not make their appearance. Below are photos showing the state of the world at game’s end.

 

In this game, the United States did not arise as a nation-empire in the final epoch. Thus, North America is a patchwork with parts controlled by England, Spain, Germany, and Russia. Interestingly the Mayans still hold Central America in strength! In South America, the Portuguese control the south. The Inca empire is overrun by several groups ending with German control of Peru. England controls Guiana in the northeast.

 

Europe remains a patchwork of states. In the British Isles, the Scots have held out fortified against England. Go William Wallace! Germany controls much of Northern Europe, displacing France which retains the Spanish peninsula and northwest Africa. The Swedes have maintained their neutrality in strength. Southern Europe has remnants of the Romans, the Goths, and the Holy Roman Empire. Eastern Europe and much of western Asia are controlled by the Ottoman Turks. A key missing empire that did not show up were the Arabs.

 

This leaves the Middle East as a patchwork of states. Old Crusader states still hold Palestine and environs. There are remnants of Macedonia in the Levant and the Sassanids occupy modern-day Iraq. Of the old empires, bits of the Egyptian and Carthaginian empires have held out in north Africa since the first two epochs. The sub-Sahara has migrants from India. An ancient Gold Coast empire still holds out against colonization. The Portuguese rather than the Dutch have colonized South Africa, while Spain instead of France occupies Madagascar. In India, there are tiny pockets of the Maurya dynasty in Hindu Kush. The Mughal empire is still ascendant. England has a foothold in Goa. The Guptas still hold Hindu Kush, the northwest, and Ceylon but have managed to forge ties with the Malay States.

 

The Khmers hold the mainland of southeast Asia. China is controlled almost solely by the Manchu dynasty except Germany has forced open some ports. The Japanese empire still hold their ancestral lands, while England has colonized Australia. There were a number of attempted colonial land grabs by the European states, not all successful. Unlike my last game two years ago (with five players), there was more of a mosaic at the end of the game – tiny pockets of isolated nation-states rather than great empires. The no-shows (Epochs II to VII respectively) were: Assyria, the Celts, the Arabs, the Chola dynasty, the Timurid Emirates, and the U.S. Many monuments were destroyed, but a number still stand in China and Europe.

 

I could design an Epoch VIII to bring us to the present age, except others have already done so for the newer edition of the game. But I’d have to bend the idea of how an empire is defined. If anything, we now see more breakup rather than consolidation. The world today is a strange patchwork of nation-states, with layer upon layer accreting on the bones and dust of old empires. It’s no wonder such entities carry so much political baggage, and world peace remains elusive.

Get Lamp

Get Lamp is a documentary by independent filmmaker Jason Scott stitches together some eighty interviews with people from the world of text adventures and interactive fiction. I had encountered text adventure games in the 1980s, but didn’t find them as interesting. However interactive fiction is now on my radar screen since reading Nick Montfort’s Twisty Little Passages. That led me to playing A Dark Room, and I was pleasantly surprised at how immersive the experience I had with no fancy graphics.

 

Get Lamp begins with the game that started it all – Adventure. Created by Will Crowther and expanded by Don Woods, it was followed-up by a bunch of MIT folks with the more well-known Zork and its descendants. While Crowther declined to be interviewed, it was interesting to hear Woods and others discuss those early exciting times in the 1970s that eventually led to the founding of Infocom – a computer game company specializing in text adventures that was very popular in the 1980s. But it wouldn’t last. Leaps and bounds in computer graphics (and memory) meant the demise of text-only games.

 

The interactive fiction community today is small but still active. When asked whether they thought if a commercial comeback was possible, most of them demurred. Some folks remarked that as a hobby or side-gig that you just loved doing personally, it was fine, but you couldn’t leave your day job if you wanted to pay the bills. An annual competition still draws interesting and creative entries, but these have become shorter and with an artistic bent – rather than the long sprawling adventures of old. Mazes, long a staple of such adventures, is now a bugaboo. As one interviewee remarked, the first few are fascinating but after a while it gets old and repetitive – especially since the main reason to include them was to “lengthen playing time” so consumers felt they were “getting their money’s worth”.

 

The wide range of interviewees and Scott’s ability to ask them questions that elicited interesting responses gave me plenty of food for thought. Blind folks talked about how playing the game increased their feel of what it might be like to be sighted. One person made an analogy to what it might feel like to wield magic (in your inner being, not like a limited LARP). Those of us sighted folks who rely on vision for much of our experience and interpretation of the world around us may find it harder to “see” such experiences in a different light, so to speak. A non-blind interviewee commented on how being solely in text mode – reading words and typing words in response – was more immersive than switching between graphics and text. Until we get to the level of Ready Player One, there might be something to be said about immersion without mode-switching. It made me think about my classroom and how students take notes and interact with the material, some on tablets and others with pencil and paper.

 

Old-school gamers talked about the joy of “banging your head against the wall” trying to solve a puzzle and then the elation of coming up with the answer. There is always an answer, and in the best games it’s creative and clever – and you feel creative and clever coming up with it as long as you don’t short-circuit the process by finding it on the internet (today). But the same folks didn’t think most of today’s young gamers had the patience or desire to crunch through such experiences, given the wealth of other possibilities they could spend time (and money) on. I suppose old-school puzzle solving is a little like my experience with research. Try lots of things that don’t work before you eventually hit on a solution – and it’s a great feeling when you do.

 

Listening to the interviewees talk about their personal attraction to interactive fiction made me wonder how I could get my students to find chemistry so fascinating they’d want to immerse themselves in it. I chose to be a chemistry major because I was intrigued by its puzzles and its blend of visual data and mathematical modeling: infinite possibilities with just a few idiosyncratic building blocks! I feel that my students today are much more instrumental and practical about their choices. They’re also much more aware about issues such as social impact and sustainability, things I did not envision and played little role in my choice of major or career. Most students in my class likely see chemistry class as something they have to endure towards some other future goal. The present goal seems to be to get an ‘A’ or in some cases just pass the class. I’m not if sure my own enthusiasm and love for the subject makes much of an impact.

 

Watching a bunch of talking heads in Get Lamp sounds like it might be boring – and for someone who isn’t interested would certainly think so – but I found my mind abuzz. The documentary is fifteen years old now. With the advances in AI Large Language Models, is large-scale immersive interactive fiction around the corner? We’ve trained AI to mimic famous dead historical figures as conversational partners. Could an interactive fictional world of magic so that we muggles could imagine what it would be like? And the key is to use text, not graphics, to fire up the imagination – that could open the door to an immersion experience more powerful than fancy graphics. As one interviewee remarked, the dragon imagined in your mind is so much more real than the one you see through a filter that has decided what a dragon should look like. Could I do the same with conveying chemistry? Could imagination help students see the magic of the molecular level better than pretty pictures on a slide or in an animated movie? Could words evoke that experience? The mind’s eye is a powerful thing and perhaps the conduit to deep and meaningful learning. Now that’s an exciting thing to imagine!

MegaProjects

Why do mega projects constantly have cost overruns and hardly ever finish on time? For an incisive and insightful analysis, I recommend How Big Things Get Done by Bent Flyvbjerg and Dan Gardner. Flyvbjerg is probably the leading authority on mega projects and his organization maintains probably the only useful database that classifies projects into categories such as building hydroelectric dams, airports, pipelines, and even hosting the Olympic Games. The book is short, engaging, and organized according to key takeaways.

 

I learned that nuclear power and storage have the worst overruns. The Olympic Games and large IT projects are also in that mix. The best cases are solar and wind farms. Why is that? The authors argue that a key ingredient to reducing the overrun problem is having lots of modularity; their catchphrase is “build with Lego”. The advantage of a modular approach is that you can quickly and systematically improve the process as you move along. It’s partly why the Empire State building is one of the few skyscrapers that finished on time at lower than the projected cost.

 

But modularity isn’t enough, you need an overseer with a lot of experience in the field – a “master builder”. And there needs to be a great team doing the work. It’s why the best contractors bring in their own experienced teams, people they trust and they’ve worked with on other projects. The planning phase is also essential. It’s so crucial that the authors spend the first chapter hammering this point, and repeating it in subsequent chapters. When you plan and simulate in detail, you can eliminate many (but not all) potential problems that may arise. Then you execute swiftly – because dragging out the time frame when actual physical work is being done increases the problem of a “black swan” occurrence – a global pandemic, for example. The authors’ catchphrase: “Think Slow, Act Fast”.

 

My industry is higher education. I’m not building anything physical so at first glance it doesn’t seem like much of this would apply. I’m trying to help students build abstract knowledge so they can apply it, but I can’t look into their minds so I don’t really know if they’re learning or not until I assess that knowledge. Even then, my assessments may not provide me sufficiently valid inferences. A chapter in the book reminded me how to ask “why?” to any project. I have a tendency to quickly think up solutions and actions to a perceived problem without stepping back and asking the why questions to get at the heart of the matter – like a good detective should (I’ve been watching old TV police procedurals.) The vague “I want my students to learn better” is the ultimate goal but I need to drill down a little more into the details of why a particular activity or curricular goal is appropriate. That’s a lot of planning work!

 

I was once involved in a building project, except I was a partial contributor to the cost overrun. I was part of starting up a new liberal arts college outside the U.S. where such institutions are rare. As a faculty member, my main focus was building the curriculum, but somehow I got dragged into becoming the college’s building liaison to the architects because of poor planning before I arrived. By the time I saw the plans for the woefully inadequate science facilities, foundations were being laid. Every suggested modification made would increase the cost and time overrun. I didn’t understand the politics surrounding the different stakeholders, I didn’t have any experience, but the architects on the ground kept coming to me because I was available. So I learned fast. There were many compromises made. I tried my best on a tight timeline, but I can’t say I did very well.

 

As to the non-physical parts of starting up a new college, I can see many parallels to the good advice given in How Big Things Get Done. It was a big project. The most experienced overseer left before I joined. When I arrived, I found a mixed team – certainly not the best, and most of us lacked experience, myself included. Planning was rushed and inadequate. While initially hired as a faculty member, I quickly found myself becoming an administrator. Much of my time was spent anticipating problems coming down the pipeline so I could get ahead of them. It was exhausting. I left after eighteen months. I knew nothing about big projects and I had little preparation and insufficient experience. Trial by fire. I’ve learned a lot since then. If I ever get the chance to work on another big project, I’d be more wary, less idealistic, and most importantly much better prepared.

Everybody Wins

Boardgames were my primary hobby for some fifteen years. It started around 1996 when I was introduced to Settlers of Catan and Robo Rally. The former, now known simply as CATAN, went on to change boardgames from niche industry to mainstream. I still have my Mayfair first edition of Settlers; it doesn’t mesh with any of the expansions so I don’t own any of those (I highly recommend Seafarers but not Cities & Knights). While Robo Rally is not as well known, I have a coveted first edition including three expansions (Armed & Dangerous, Grand Prix, Radioactive) that I could sell at significant profit on the secondary market.

 

Then my interest waned. I also got busy with a big move and a new job. I was previously the nexus of a game group – hosting game nights, being an evangelist for the hobby, and teaching new games to new people. None of that anymore. I still play games here and there, and I still enjoy them, but I don’t foresee going back to those good old days. I used to be very knowledgeable about the latest releases and what were considered the best games. I even owned many of them, but I’ve slowly downsized my collection to some 125 games (not including expansions).

 

So it was a blast from past to read the beautifully produced book Everybody Wins by James Wallis. It is subtitled “Four Decades of the Greatest Board Games Ever Made” because it goes through the winners of the coveted Spiel des Jahres awarded annually to the supposed “best” game of the year. Wallis, a game designer himself (of the delightful storytelling Once Upon a Time), does an excellent job introducing and critically evaluating the awardees. Some are well-known, some are obscure, some are questionable, some are clear winners. He mixes his narrative with vignettes about designers, game mechanics, and other great games that didn’t win the award. It’s a great coffee-table book if you’re a game evangelist. (I borrowed my copy from the library so I won’t be owning it.)

 

I liked how Wallis divided the 44 awardees into 5 era groups.

·      1979-1985: Opening Moves

·      1986-1995: Settlers & Co

·      1996-2004: The Golden Age

·      2005-2015: Identity Crisis

·      2016-2022: New Purpose, New Direction

The historical arc was helpful in understanding why and how the award came about in Germany and why the first winners weren’t German designs. Then there is a narrowing of choices and we see the rise of the more famous German designers culminating in Settlers of Catan. This kicks off the Golden Age of the new classics from El Grande to Ticket to Ride before yet another divergence.

 

Of the sixteen winners from 1995 to 2010, I owned all but two. But in recent years, I’ve given away some and sold others on the secondary market. I used to have many Renaissance city-building games (quite the fad back in the day) but I’ve trimmed many out of my collection. I’ve also given away many family games to colleagues with pre-college-age kids. There wasn’t much reason to maintain such a large collection when I was hardly playing most of my games. My collection is still fairly diverse and includes old “wargames” from the 1980s, but I don’t have many newer games. My buying habit decreased in 2012 and I doubt I own any new games since 2016. I’ve played newer games, and there are many good ones out there, but it’s hard to justify purchasing more games to only have them sit in my shelf most of the time.

 

Kickstarter has completely upended the board game industry. Many of the stalwart game producing companies have disappeared. The locus of creative inventions has shifted from Germany and Europe to the United States, with Asia also on the rise. The hobby is thriving and I’m glad for it. The Spiel des Jahres helped even as its importance lessened. Once upon a time I could recite all the winners. Now I no longer pay attention to the award (although I would like to try Cascadia!) and that’s okay too. I’m no longer a games guru but I can still introduce some of the oldies – they’re still great games. Everybody Wins is an apt title for this retrospective, but one that also looks forward. And they have great game suggestions if you’re looking for something new to try!

Biochem Roundup

Whew! I made it through my first semester teaching biochemistry. It was a lot of work but I enjoyed the learning experience overall. Preparing for class took the lion’s share of my time this semester. This included reading the textbook, looking up supplementary materials to clarify things that I didn’t find clear in the textbook, preparing lecture slides, creating study guides for the students for each class session, assigning homework (from the accompanying online system or writing my own problem sets), putting together my own lecture notes, and learning some biochemistry software to analyze protein structures.

 

I successfully kept to my rigorous schedule of being three weeks ahead of the students. I managed to maintain this until Thanksgiving when the gap narrowed to two weeks. But by then, we were almost at the end and I only had a final lecture (and associated materials) to prepare after the Thanksgiving break. In the first half of the semester, I was too ambitious and overpreparing. I had too much class material for the allotted time. I had to cut around 5% of the material on average, but after the first month I started to reduce what we would cover in class and found a happy medium in the second half of the semester. On a rare occasion we even finished five minutes early (and there was great rejoicing)!

 

The students were fantastic. I had a small class (my section was the least subscribed) but an engaged one! They asked lots of interesting questions, participated with gusto in all my assigned class activities, and were very forgiving when I didn’t know something or if I exhibited minor confusion over the material. I was upfront with the students that this was my first time teaching biochemistry and that I was not a biochemist by training. I also told them that I would pepper my class with many examples from origin-of-life research – I think students found these interesting based on conversations in class and when students dropped by my office to chat. I won’t see the course evaluations until January but I suspect they will be quite positive. I feel I established a good rapport with the students.

 

While I did cover most of the material in the typical sequence covered by my biochemistry colleagues, I made one significant change. I moved enzyme kinetics earlier so that I covered it immediately after protein structure. I think this worked well because all those equations and graphs can be quite intimidating to the students, but they got to see it earlier rather than close to metabolism – another challenging topic – that is close to the end of the semester. The next time I teach this class I think I will start talking about the basics of phosphates being energy carriers (ATP) earlier. I will also move some of the G-Chem review material out of the first few lectures and sprinkle them where they apply during the semester. I’d like to jump into amino acids the first week (I only got to them at the end of Week 2). I need to make some changes to nomenclature/structure of sugars and lipids; these felt plodding. I had applications, but I need to rearrange the material and spread things out a bit more.

 

I need to write more problem sets and do less with the online homework system so that I can get students to generate written answers of sufficient precision and detail. A number of students gave vague answers – sometimes a sign that they don’t really understand the material that well. I think my first midterm exam was pitched correctly. My second midterm exam focused too much on enzymes and protein function and less on nucleic acids and sugars even though we spent about an equal time on both, i.e., questions on the later topic were a little too superficial. That being said, the average on both midterms was a solid B. Were the exams too easy? I suspect maybe a little. (The students would disagree.)

 

The final exam had the same average score as the midterms, so I kept my promise to the students that the final would be of similar difficulty. But writing the cumulative final, however, was very difficult for me. Since designing a good cumulative summative assessment that has (I hope) reasonably high validity and reliability requires me to ask a spread of questions, I felt constrained in how deep I could go with the questions. I eschewed multiple-choice questions, which meant I couldn’t ask as many questions if I wanted students to generate answers. Some of biochemistry colleagues do not give a cumulative final; rather the final is weighted similarly to a midterm exam and doesn’t aim to cover such a wide spread – thus one can ask questions of similar depth as other midterms. I think I will do this the next time around because I don’t yet have as much experience designing really good exam questions to test cumulative knowledge. After teaching the class a few times, I’m sure to get better at it.

 

Hopefully I will be slotted to teach biochemistry again next academic year. In the meantime, I can look forward to devoting more time to G-Chem and P-Chem next semester. This semester, I did adequately in those classes (which I’ve taught many times) but my teaching mindspace was mostly focused on Biochem. I’m also looking forward to getting more research done, as I did almost zero for the four months of August through November.

 

And that’s the Biochem Roundup! I made it through without screwing things up. I will count that as a win.

Food of a Nation

I am regularly amused by arguments between my home country and its neighbours about who has the better claim to a ‘national dish’. These have taken place more often as countries apply to UNESCO for recognition of a cultural artifact – food apparently qualifies. You’ve likely heard about the argument between Ukraine and Russia about the origins of borsch since the breakout of war last year. Anya von Bremzen covers this incident in the epilogue of her latest book, National Dish, which I just finished reading.

 

National Dish is the second food-travel book I’ve read this year that’s not also a cookbook –  the other being Have You Eaten Yet? by Cheuk Kwan on the Chinese diaspora. Both books are excellent and engaging. Kwan’s book motivated me to try fusion Peruvian-Chinese food three months ago, while von Bremzen’s book had me scouring the internet looking for a specific brand of Japanese instant noodle bowls; I plan to visit a Japanese market in my area to see if they have any.

 

But National Dish isn’t just about food. It’s also about what happens to how food is viewed in the process of nation-building. What we call nation-states today are a rather recent development over the last two hundred years or so. Population explosion following the industrial revolution and advances in medicine are contributing factors. So are wars and territory-grabbing. It was eye-opening for me to read about Franco’s Spain and Mustafa Kemal’s Turkey, and how the stoking of nationalism extended to cuisine in multiple ways. Von Bremzen tries to trace the history of a so-called ‘national dish’ and uncovers varied beliefs from the local populace. She interviews food experts, anthropologists, and chefs. And best of all, she decides to try and cook these dishes with local ingredients in their home countries. You might get hungry reading about it; she’s an evocative writer.

 

Growing up in a multicultural country, I learned to classify certain foods as being the specialty of particular ethnic groups. But these dishes had been adapted to local tastes such that you were not likely to find them from their “home countries” – although none of us (school-aged kids) thought of ourselves as being from some other nation. Where we were born and grew up was our nation. We were proud of it. Everything I ate growing up I consider to be part of my nation’s cuisine. Globalization wasn’t as prevalent, and I had hardly tasted the cuisine of other nations. My only impression of American food growing up was KFC, an occasional treat.

 

These days, when I visit my home country, cuisines of other countries are all the rage. My friends who still live there are more interested in these new foods, while I am interested in the older local delights. But now I do see a trend of ‘new’ restaurants finding a niche by specializing in ‘old’ food – supposedly more authentic, and higher-priced with advertised artisanal labour. National Dish made me aware of this trend, and von Bremzen helps me put words to concepts that were hazy in my mind. Food travels. Cultures meet and exchanges take place that lead to a new synthesis. A fusion of sorts. For me, these exchanges have been friendship-building. Coming to the U.S. was eye-opening for me, and I enjoyed being exposed to national cuisines from all over the world. Food unites rather than divides. As it should be!

 

I close with a picture (grabbed from the internet) of what I suspect most citizens of my country would pick (if only given one choice) of our national dish. I haven’t eaten it in a while. My country’s cuisine is not represented in the city where I live; I’d have to drive an hour and a half to get it, if I didn’t want to make it myself. One thing globalization has done is that I can now find spice mixes of any food that I miss from home. (I’d still have to put in the labour.) And the internet tells me step-by-step exactly how to make anything I want if I want to celebrate the food of my nation!

Elemental

I’m a sucker for movies featuring the four elements of Empedocles (and popularized by Aristotle): Earth, Water, Air, Fire. Thus, despite the less than stellar reviews, I watched Pixar’s latest: Elemental. Not willing to shell out cash, I borrowed the DVD from my local library.

 

What’s the story about? There are four populations of elemental beings: Earth, Water, Air, Fire. While most folks live with other folks of the same element in rural areas, the shiny metropolis brings together the different elements – some well-established, some looking to start a new life. The trouble is that it’s hard for elements to co-exist in the same space without causing physical problems. That’s especially true for Fire. It burns Earth. It boils Water. (Oddly, I don’t recall noticing problems between Fire and Air even though you’d think that Fire would consume the oxygen in air as fuel.)

 

The Earth folks look like plants, seeds, and earth – they’re solid. The Water folks are distinguished by being liquid and transparent-ish. Air folks look like clouds. These three fit into the well-known phases of matter: solid, liquid, gas. In chemistry, a substance can transition between phases at the appropriate pressure and temperature. In fact, the boundary that separates phases define melting and boiling points where two phases co-exist at equilibrium. At the triple point, all three can co-exist. My first-year college students should be able to identify all these on a phase diagram. Fire doesn’t fit into this story. It’s not a substance, it’s chemistry in the midst of transformation.

 

In Elemental, there are no phase changes for the most part. Each elemental being maintains its properties. But there can be problems on physical contact. Water gets other folks wet. Air folks get annoyed when someone walks through them instead of going around. The biggest problem, though, is Fire. These folks have to literally tiptoe around their neighbors, being careful not to touch them physically lest they actually cause a change of phase. Fire can burn earth causing it to be consumed or parts to “drop off”. Water can be boiled into what seems like nothing (should it be Air?) which sorta kills its existence. So what happens when an attraction forms between a goofy Water-based individual and the protagonist from the Fire clan?

 

In the beginning, they can’t touch. And then somehow by the end of the story they can, because they have somehow “changed”. There’s no explanation how or why. I posit that they manage some sort of insulation in between. An air gap perhaps? The movie doesn’t support this – that physical touch seems to be, well, a touchstone. Maybe there’s a thin layer of oil. That allows the fire to keep burning without evaporating the water? I’m coming up empty-handed for reasonable physical explanations. Maybe unknown magic should be invoked. And the point of the movie is the emotional relationships – and somehow those transcend and transform the physical, I don’t know how.

 

In the physical world there are phase boundaries between solids and liquids, liquids and gases, solids and gases. (These also exist between different solid phases of which there can be many, especially at higher pressures.) My students should also be able to locate and tell me the definition of the critical point, where the boundary between liquid and gas seemingly dissolves. There are some videos showing the disappearance of the “horizon” that separates them, but frankly they’re not terribly exciting. I suppose you could call the supercritical fluid a new phase of sorts. I suppose you could also call plasma a separate phase, and maybe that’s where Fire fits.

 

I don’t have much else to say about the mostly forgettable movie, other than I like how Fire has the best food. Hot and spicy! My kind of folks. They also make the best chemists, crafters, and engineers. The manipulation of glass scenes in the movie are cool; I mean hot. What on earth would we do without fire?

Multiple Choice

There are very few multiple-choice questions in my chemistry exams. This is because, at the college level, I think students should be able to generate answers. It’s easier to recognize a better answer or explanation given several choices compared to generating one. And since I think that being able to generate answers is a learning goal, most of my exam questions require students to write out their answers and explanations.

 

I’d like to think that the grades my students earn on those exams provide a fair assessment of their knowledge of chemistry. By fair, I mean that I hope my cumulative final exams, which are summative assessments, are both reliable and valid. But because I don’t use multiple-choice questions on the exam, there might be additional inconsistency from the grader (me!) even though I have a tight grading rubric. Furthermore, my exam would have fewer questions than a multiple-choice equivalent because generating answers takes more time than recognizing them. Thus, it is questionable whether the “coverage” of my final exams allows me to make a reasonable inference that the students’ final grade reflects their knowledge of the semester’s cumulative material. Asking fewer questions provides less coverage and may be a poorer proxy.

 

While I haven’t taken the plunge, I occasionally consider what it would take for me to switch my summative assessment to a multiple-choice format. There are several advantages and disadvantages to this. A big advantage is that I might well be able to construct a final exam that has better validity and reliability. Grading will also be much faster. And analyzing the results per question will allow me to refine the questions on subsequent exams in a systematic way to improve their quality as differentiators. A disadvantage is that I can’t assess student’s ability to generate. Also it takes a lot of time and work to design high-quality multiple-choice questions and answers.

 

I decided to read a decade-old paper by Marcy Towns: “Guide to Developing High-Quality, Reliable, and Valid Multiple-Choice Assessments” (J. Chem. Educ. 2014, 91, 1426-1431) to help me think about all this. Towns provides good guidelines on writing questions and putting together the response set (or answers). I was familiar with the principles, but I found it helpful to see actual examples and be convinced by why the guidelines work well. I was also reminded how hard it is to put together good questions and answers.

 

Eye-opening things I didn’t know: (1) The optimum number of responses in a set is probably three. Towns backs this up with data and examples. I would have guessed four, but I find Towns’ argument convincing. (2) Item order and response order effects can be significant. Towns cites the data that having four challenging stoichiometry or equilibrium problems in a row impacts the student getting the right answer on the last one. I should think about this for my non-multiple-choice exams too. There are also priming effects where the order of questions can aid a student in “setting up” the appropriate cognitive processes to answer a question. This is both good and bad depending on whether you have a summative or formative assessment. (3) Sometimes students “choose an earlier answer without reading the entire response set”. Huh, that did not occur to me.

 

Towns also suggests simple ways to perform item analysis on the results to gauge the quality of the questions and response set. Were they good discriminators of knowledge? These should be easy to set up, and I can even start doing this on non-multiple-choice exams although with a bit more work, maybe with the help of GradeScope. Towns provides rule-of-thumb values to estimate item difficulty and item discrimination. I could easily write these into a program that does the analysis for me.

 

Am I willing to switch to multiple-choice questions for the final exam? I don’t know, although I have considered a possible partial mix. Being at a liberal arts college with small class sizes, my grading is not a chore. But if I want to prepare students for the summative final, my formative assessments should be of similar format so students get used to the format. But I feel that requiring the students to generate answers is of value to their learning the material at a deeper level. Thus providing both formative assessment and summative assessment in my current exam/test/quiz format does what I want it to do. I’m probably sacrificing some validity and reliability within my class. More so if my department was trying to compare grades across sections. We haven’t done so in any formal way, but I can see reasons why it might be a good thing to do. And if we do so, a multiple-choice assessment is likely the path forward.