Smoke in the Leaf

In quantum chemistry this semester, we’ve been solving eigenvalue equations over and over. I think some of my students get what we’re doing, but I still have moments in office hours where a student seems to struggle with the very concept. Perhaps the narrow way in which I introduce this topic isn’t helping some of the students.

 

It was therefore refreshing to read a different take on eigenvalues in Jordan Ellenberg’s Shape, which I recently blogged about. Starting with geometric progressions, he shows examples of how one can extract a specific number that seems to control the rate of geometric growth. This number is the eigenvalue. He then introduces the Fibonacci sequence and shows that it arises from the difference of two geometric progressions, controlled by the eigenvalues -0.618 and +1.618, the latter famously known as the “golden ratio”. I like the way Ellenberg discusses the nature of eigenvalues:

 

[They] capture something deep and global about the system’s behavior. They’re not properties of any individual part of the system, but emerge from interaction among its parts. The algebraist James Joseph Sylvester called these numbers the latent roots – ‘latent,” as he vividly explained, “in a somewhat similar sense as vapour may be said to be latent in water or smoke in tobacco-leaf.” Unfortunately English-speaking mathematicians have preferred to half translate David Hilbert’s word Eigenwert, which is German for “inherent value”.

 

In my class, we talk about using Hermitian operators to extract eigenvalues from a wavefunction. I’ve tried to give my students the sense that you’re pulling out a number from some sort of global system that is captured by a mathematical (wave)function, but sometimes when I talk about math the students seem befuddled. Perhaps Ellenberg’s “smoke in the leaf” will give them a physical picture of what we’re trying to do – perhaps capturing the vapor in a GC-MS and extracting an output number, e.g., the mass of a particular molecule in the vapor.

 

Ellenberg expands the idea with more examples beyond the geometric progression “growth” of a pandemic and R₀ values. There’s an example of Google PageRank searching and Markov (random) walks. There’s a segue into Monopoly the boardgame, where apparently Illinois Ave (in the U.S. version) is where a counter would spend most of its time in the long run (i.e., you’d need to play a very, very long game to notice this) once the system approaches ergodicity. You can capture this in an eigenvector: “something inherent to the long-term behavior of a system that’s not apparent just by looking at it, something latent like the smoke in the leaf.”

 

What’s helpful at the end of the chapter: Ellenberg provides a worked example of two infinite sequences and two operations. One is a shift, where moving all the numbers in a sequence one spot to the left looks like you’ve multiplied the original sequence by a constant to get the new sequence. The other is a pitch where you multiply each term of a sequence by its position in the sequence. This leads to a discussion of eigensequences, akin to eigenfunctions, followed by demonstrating these as examples of non-commuting operations. And yes, this leads to Heisenberg’s Uncertainty Principle. My students are regularly puzzled by this even though they follow the mathematics. I don’t think I’ve done a great job helping to shed light on this, so I’ve scanned some pages of Ellenberg’s book for my students to read. Maybe it will help shed light, maybe it won’t.

 

Next week we’ll be looking at Huckel Molecular Orbital Theory for delocalized pi-systems. One of the examples will be butadiene. Interestingly, the eigenvalues for the “resonance” integral (in units of beta) are -1.618, -0.618, +0.618, +1.618 for the four molecular orbitals – seemingly related to the two eigenvalues controlling the Fibonacci sequence. I don’t know if there’s a connection between the two, or I should say that I haven’t spent the time thinking about it more carefully. My brain is somewhat fried, and I’m just happy to make it to the Thanksgiving holiday. My students are also tired, maybe more so. It’s been a grueling semester.

 

This chapter in Ellenberg’s book ends with the “notes in the chord”. It’s kinda cool. When you hear a chord triad, you can separate out the three notes and their three eigenvalues mathematically using a Fourier transform. But more amazingly, a trained ear can actually hear these notes in the chord “even if you don’t know calculus… because this deeply geometric computation… is also carried out by a curled-up piece of meat in your ear called the cochlea.” It’s amazing what these small body parts can do. An eigenvalue separator in our very ears!

Trial and Error

Sometimes I enjoy reading math. Or I should say I enjoy reading about math when it’s aimed at the non-specialist. Jordan Ellenberg does a great job at this, and I enjoyed reading his book How Not to Be Wrong. I had a feeling I would enjoy his latest book Shape, and so far I’ve not been disappointed. Once again, he wraps math – this time focusing on geometry and number theory – around interesting stories of people and events. Yes, there’s a chapter about Covid-19 and geometric progressions, but I won’t be discussing it today.

 

I particularly enjoyed Chapter 6, “The Mysterious Power of Trial and Error”. It’s about random walks, and features both the Drunkard’s Walk and the Gambler’s Ruin. Ellenberg begins the chapter with a question he often hears in his math class (one that I occasionally hear in my P-Chem office hours): “How do I even start this [problem]?” Ellenberg jumps at the teaching moment: “… it matters much less how you start than that you start. Try something. It might not work. If it doesn’t, try something else. Students often grow up in a world where you solve a math problem by executing a fixed algorithm...”

 

That’s a good description of how my students approach chemistry problems. In my G-Chem classes, we’re in stoichiometry tackling problems of how much of A reacts with B to form some amount of C and D. What is the limiting reactant? What if the reaction yield is less than 100%? How much leftover reactants do you have? There are systematic ways to approach these problems, and I try to model these with worked examples. But there are multiple ways to solve these problems, so I try to show the students the common approaches and their caveats. In most cases, these problems are not as open-ended, so learning algorithmic approaches is helpful.

 

Several weeks ago, we were drawing Lewis Structures in G-Chem. Trying to draw the best structures is a more open-ended problem. I tell my students that the only way to get better is to practice, practice, practice. As you draw more structures and evaluate them (using general guidelines about the octet rule, formal charges, resonance), you get better at the task. I show the students my method which is more intuitive and diagrammatic, involving some trial and error. But some of my students have learned a more algorithmic method from their high school chemistry class. I tell students that they don’t have to use my approach if they prefer something else they’ve learned. (My approach also differs from the textbook.) Students don’t like this open-endedness. They want a surefire algorithm. But real chemistry doesn’t work that way. Neither does real math, according to Ellenberg.

 

Research is a good example of trial and error. Sure, there’s intuition involved, and I’ve built up some amount of it over the years. But as I branch into areas new-to-me, I become a novice again, and so sans any better guidance, I launch in and try a few things that may or may not work. This is a challenge for students when they start working in my research group. Yes, I do tell them the first several molecules to build and calculate, and what data to extract – I’m a computational chemist – but then I try to coax them into coming up with their own ideas of what to try next. For some students, this comes more naturally. For others who resist this approach, they don’t last long in my group – because then research becomes starts to feel like a tedious chore.

 

I’ve been educating myself about machine learning approaches for some of my research projects. Nothing hardcore yet; I’m still mostly playing in the kiddie sandpit. Hence it was fun to read Chapter 7, “Artificial Intelligence and Mountaineering”. Ellenberg introduces gradient descent, a method I’m familiar with, but then he scopes out to discuss how one approaches huge N-dimensional problems – things I will have to tackle in the large data space of chemistry. How does one navigate between underfitting and overfitting? That’s an interesting challenge and a lot of it involves trial and error as you decide how much to layer and how to assign weights to your model neural net. You get the computer to do the number-crunching for you, but you should be always cautious about the output and whether it makes sense. I’ve learned that lesson through trial and error.

 

One way you can do this is to have the algorithms play games against each other, the subject of Chapter 5. Tic-tac-toe, checkers, chess, and Go, are famous in the A.I. and machine learning literature. Tic-tac-toe can be worked out by hand. Checkers can be (almost) exhaustively decision-treed. Chess and Go have too many combinations to be checked at the speed of present processors, although quantum computing may cut the Gordian knot. But these games are all closed systems. I was interested to hear that some folks had written an A.I. for the Lord of the Rings CCG – a much trickier prospect with a random draw deck and different sorts of interactions (the A.I. was written for the cooperative version of the game). Could an A.I. learn to negotiate with players? Apparently, there are some folks working on an A.I. for Diplomacy. That is a very interesting choice for a case study: Limited movement with simple rules, but the tricky part is all about the negotiations among players.

 

Can playing games through trial and error train the machine to play the perfect game? I suppose it depends on how tractable the decision-tree might be and what the complicating factors are, but perhaps this is a less important question. Ellenberg quotes top checkers and chess players and concludes: “Perfection isn’t beauty. We have absolute proof that perfect players will never win and never lose [games that end in Draws based on finite decision trees]. Whatever interest we can have in the game is there only because human beings are imperfect. And maybe that’s not bad. Perfect play isn’t play at all… To the extent that we’re personally present in our game playing, it’s by virtue of our imperfections. We feel something when our own imperfections scrape up against the imperfections of another.”

 

That last line is perhaps the beauty in trial and error.

IChEAmistry

Anyone told you chemistry was complicated? That the concepts were difficult to put together in your mind? And that to learn it you’ll need a master craftsman (or craftswoman) to help you put it together akin to building craftsman-style furniture? What if you didn’t need any of that? What if you could receive a package of parts, open it up, follow the instructions, and build your own knowledge of chemistry? Maybe it’s just like putting together IKEA furniture. Anyone can do it!

 

I confess that as a computational chemist and having poor hands in lab, I’m similarly challenged when it comes to IKEA furniture. I can put it together, but not very well. One of my chairs is still a little shaky. But by and large everything holds together and is functional.

 

Could we turn learning chemistry into something IKEA-like? Can we break down the concepts into bite-sized pieces and build the framework step-by-step? Can you be guided through the process by an instructor? In a sense, that’s how I design my chemistry classes – to help students digest smaller bits of material and get practice working through problems aimed at helping them grasp the concepts. But if I have such a program in place, does one need the human instructor? Could the process be automated with an instruction manual that’s simple enough to follow such that every student can “teach themselves” chemistry?

 

In my dreams, I call this IChEAMistry. I figure K and the “hard” Ch are similarly enunciated. I’ve even hacked together an imitation logo with not quite the right font (IKEA changed Verdana to Noto two years ago) and not quite the right colors from my eyeballing it. Maybe I need a lowercase leading “i”! Needless to say, this program has not gotten off the ground. I haven’t started looking for venture capital funding even though I can handwavingly claim that I will use machine learning to optimize an appropriate algorithm based on student data from my classes as a training set. And since I’ve taught two thousand students or so over the course of my career, maybe I’ve got some decent data.

 

What I’m presenting as a fantasy, someone else will try to sell you as a reality. Not too long ago, we had the robot tutor in the sky. And yes, one of their products was chemistry. While that venture hasn’t quite worked out, I assure you there will be others who will claim to have improved on the problems of the previous approaches. Will this be successful? For a self-motivated student who can learn on their own, it’s possible. We’ve had correspondence courses for a long time, and I opine that it’s possible to replicate that experience with a computer algorithm. Regular assessments run by the algorithm can check if the student has learned the material. Digital badges could be issued. Nothing wrong with any of that. Someone might even accredit the work. We already accept results from standardized multiple-choice computer graded exams such as CLEP or AP for chemistry, provided the student gets a high enough score.

 

Will this work for the majority of students? I don’t know. My guess is that a few of my students could sufficiently hack their way through IChEAmistry. But most of them will flounder. But is it because they’re not used to the system? If an IKEA-like approach was employed in the earlier grades, maybe this could work? I shudder at the thought since I think younger children should have as much human contact and guidance as possible. Maybe the IChEAmistry approach is centered around group work and activity? That introduces its own complications.

 

If launched at the introductory level, will IChEAmistry provide a sufficient foundation for students who want to pursue further courses in that subject? Maybe. As it is, many of my students in P-Chem seem to have trouble remembering what they learned in G-Chem. An appropriately-timed iChEAmistry “booster” just before P-Chem would likely be helpful. Could one construct an iChEAmistry version of P-Chem? Conceivably. Although venture capital funding would be less forthcoming. The killer app of tech dreams is one that will get widespread use. Lots of students take G-Chem. Much fewer take P-Chem.

 

I have a feeling that an education system using IChEAmistry will be impoverished compared to having a human instructor who you can interact with in close quarters. It’s a feeling that may or may not be justified. But I suspect a revolution is coming whether we like it or not that will continually divide the haves from the have-nots. “Elite” education will continue to have expert human tutors, such as the rich and the royal of yestercenturies were accustomed to. For the rest, you hardscrabble whatever you can get to survive. IChEAmistry might be a reality yet. Tech entrepreneurs will surely argue for the best-of-both-worlds hybrid approach where humans and algorithms cooperate to provide an enhanced education for all. But if we follow the money, I think the future looks more dystopian despite the bright cheery colors of my IChEAmistry logo.

Classification

I just finished reading Kate Crawford’s Atlas of AI. It is less about the nuts and bolts of A.I. (although there is some) and more about the ethics and implications of current and future use. Why an atlas? So that one can both zoom out for the broad view, but zoom in at some of the details, and see the intertwining. While the book roams through several different aspects, the question of who benefits and who is being taken advantage of comes up repeatedly. They may be tech workers, Uber drivers, criminals whose mugshots are stored in a database, mining companies, and more. The first paragraph of the book’s concluding chapter, “Power”, is a good summary of its contents:

 

Artificial Intelligence is not an objective, universal, or neutral computational technique that makes determinations without human direction. Its systems are embedded in social, political, cultural, and economic worlds, shaped by humans, institutions, and imperatives that determine what they do and how they do it. They are designed to discriminate, to amplify hierarchies, and to encode narrow classifications. When applied in social contexts… they can reproduce, optimize, and amplify existing structural inequities. This is no accident: AI systems are built to see and intervene in the world in ways that primarily benefit the states, institutions, and corporations they serve. In this sense, AI systems are expressions of power that emerge from wider economic and political forces, created to increase profits and centralize control for those who wield them.

 

These last several weeks, the media has been shining a spotlight on Facebook and its practices. Crawford’s analysis would have been spot-on, as would Zuboff’s, and many others who have sounded the alarm on the unregulated and extractive nature of A.I. and how it impinges on our lives whether we like it or not. We are entangled with machines and there’s no going back. As Roose warns, the question is whether we end up being machine-assisted or machine-managed. The latter is colonizing more area every day in the name of techno-efficiency. I’m not going down this rabbit-hole in today’s post. Instead I want to focus on Crawford’s statement of A.I. encoding narrow classifications. This is what Chapter 4 in Crawford’s book is about, and her main example is ImageNet.

 

How does one classify images to train an A.I.? Well, you need a training set. Lots of pictures and lots of cheap manual labor (via Amazon Mechanical Turk for instance) to attach labels to these pictures. Where do these labels come from? There’s WordNet, which classifies nouns in a particular taxonomy. Hierarchy is pre-built in a narrow way based on the classification system, and the training set further narrows the scope, not to mention implicit human biases in making classification choices.

 

One of my research projects involves training a program to calculate some molecular properties. I built the training set and have been playing around with a standard classification system that subdivides larger molecules into smaller fragments. This isn’t going so well, so I’ve been dreaming up some alternative classification systems that may do a better job “taming” the data. I’m sure my biases are affecting things in some way although likely not to the extent of present machine-learning face-recognition systems. Reading Crawford’s chapter on classification didn’t impact my research so much as make me think about chemistry and how I teach it.

 

As a so-called expert in chemistry, I have particular notions of how chemistry is organized; you might call these my “expert schema”. My goal is to help students move along the continuum from novice to expert by helping them build their schema. To do this, I have to classify things into categories so students can start to spot similarities and differences. The periodic table is an organizing schema of a sort. The division into elements that are metals or non-metals is a useful classifier. The model of atoms as balls connected by springs works well for covalent molecules with their directional bonds. I’ve used the general bond energy curve to help students think more broadly about how particles interact with each other. For that matter, helping students to think about energy requires a number of classifications.

 

Classification is by nature reductionist. This has its pros and cons. Reducing something that would otherwise seem very complicated with all sorts of things going on is very helpful to the novice learner. However, if the system isn’t merely complicated but complex, then the process of simplifying things via classification may throw out the baby with the bathwater. And chemistry is complex – at least I think so. I hope that in the upper-division classes of our curriculum, we guide students back through some of the complexity and help them to enrich their simplistic schema. This is the challenge of teaching chemistry, or any other complex subject material that one can’t just soak up by osmosis unconsciously. It takes a lot of effort on the part of the student to replace their folk-science understanding with the strangeness of nature.

 

But perhaps how scientists look at the natural world is blinkered by the classification schemes we have used so we can comprehend it – or at least we think we do. And this reductionist approach may be part of why science seems both strange and alien, at least when you get into its intricacies. Conceptualization is tricky; I don’t quite understand how it works, and yet somehow I’ve acquired abstract ideas about chemistry that organize my knowledge. Somehow that passes for expertise; at least that’s how students classify me at the moment – the one who knows the stuff. Do I really? I’m not so sure.

The Mesocosm

As human beings, our particular size (in the meter range) strongly influences how we interact with other objects. Some are close to our size, and they are the touchstones by which we try to understand things that are much bigger or much smaller. Planets and stars fall into the ‘much bigger’ category: we might group them into the macrocosm. Amoebae and bacteria fall into the ‘much smaller’ category: they constitute the microcosm. A tennis ball or a chair, objects that we directly interact with – we can see, touch, and use such things – sit somewhere in between and with us are part of the mesocosm. That’s a human perspective, of course!

 

I’m slowly working my way through Philosophy in the Flesh, a tome by George Lakoff and Mark Johnson. They posit that we should recast philosophy in terms of three main points. The first three sentences in the book (appropriate for philosophers I suppose) encapsulate the story: “The mind is inherently embodied. Thought is mostly unconscious. Abstract concepts are largely metaphorical.” Lakoff and Johnson think the discoveries of cognitive neuroscience force us to rethink the basics of philosophy and reject the mind-body separation of Cartesian dualism. They think that the way we think (both conscious and unconscious) are highly influenced by our neural architecture, by biological evolution, and by how we interact with objects relative to our size and timeframe.

 

At the simplest level, we learn by what we see and what we can manipulate. In scientific terms, one might call this observation and experimentation. Babies and toddlers do this a lot. One might posit that as they explore their surroundings, they begin to conceptualize. And in some mysterious way, learning takes place. They begin to understand not just what they can see and touch, but as children grow older they are able to imagine what they cannot see and touch. Lakoff and Johnson might argue that human biology (and by extension, interacting with others in one’s environment) both shapes and limits the way the human thinking does its magic. When we go to school and learn about science, we might even have the opportunity to extend our scope beyond the mesocosm. The authors write:

 

One thing that science has done successfully in many cases has been to extend our basic-level capacities for perception and manipulation via technology. Instruments like telescopes, microscopes, and spectroscopes have extended our basic-level perception, and other technologies have expanded our capacities for manipulation. In addition, computers have enlarged our basic capacity for calculation. Such enhancements of basic bodily capacities extend the basic level for us, the level that is at the heart of embodied realism.

 

Embodied realism. That’s the two-word philosophical backbone of the book. The three instrumental ‘scopes’ mentioned made me think of astronomy, biology, and chemistry, respectively. Spectroscopy is at the heart of chemistry, which lengthscale-wise is in the nanocosm. We can’t see discrete entities as small as single atoms and molecules, but we can infer their structure based on measurements in the electromagnetic spectrum – the ‘spectra’ of the spectroscope. I’ve themed my G-Chem 1 classes around this idea: how we make visible the invisible!

 

Conceptually, though, the way we picture this nanocosm is by imagining blobs made up of balls connected by springs. The blobs are constantly in motion and may interact with one another depending on their properties. But the mind’s-eye picture is mesocosmic – I played pingpong as a child and that’s my picture of small balls. At one point, I even had a bunch of pingpong balls (both white and orange) in a transparent container that I would bring to class as a demo, shaking it around to demonstrate atoms of a gas bouncing off the walls of a container. (Now I just use computer animations.) But we can go further. Lakoff and Johnson write:

 

What fills out embodied realism, permitting us to move far beyond mere observation and manipulation… is the existence of conceptual metaphor, which allows us to conceptualize one domain of experience in terms of another, preserving in the target domain the inferential structure of the source domain. Mathematics allows us to model metaphorical theories and to calculate precisely inferences about literal basic-level categories. Such inferences can then be projected onto scientific subject matters to give explanatory accounts for existing data and to make predictions.

 

This is especially true for the tougher parts of chemistry. I teach P-Chem, much dreaded classes for chemistry and biochemistry majors. The math is heavy-going. Many students struggle through the course. But the mathematics is uncannily powerful, and allows us to access abstract ideas that seem so far out of the realm of what we can see and touch in the mesocosm. That’s part of what makes math challenging once you leave the familiar realms of counting objects to more abstract relations. Yet mathematical models have their limitations like any other model. And what is a model, if not a way to represent something outside the mesocosm to the human mind built evolutionarily to interact within the mesocosm and yet be able to have thoughts outside it.

 

If there’s one thing in particular, outside of balls and springs, that I ask students to conceptualize in my chemistry classes, it is energy diagrams. Higher up means more energy and reduced stability. I gesticulate frequently in class and my arms move up and down to embody this idea. Lower down means lower energy and being in a more stable state. To break a chemical bond, the system moves up in energy. The chemical system must receive energy from the outside to break bonds. Conversely, and non-intuitively, making a chemical bond moves the system down in energy. Energy flows out of the system. Energy is conceptually protean and abstract, but we can count it and keep track of it.

 

I’d like to think that teaching again in-person and using bodily motions as part of my explanations helps student learning. Not so easy to do via online learning, but I suppose I could make videos of myself. I still feel that something is lost in translation through the flat screen, but I’d be hard-pressed to tell you exactly what that is. I’d like to think there’s something particular apt in learning person-to-person physically in the same space and sharing the mesocosm directly!

 

P.S. Interested in the osmocosm? See here.