Theory of Learning and Evolution

I recently read three papers by Vanchurin (and colleagues) that builds a theory of learning and illustrates its generality in reference to machine learning, biological evolution, and the (presumably physico-chemical) origin of life. Math is involved, and I found some parts can be difficult to follow. But the writing is clear and the progression of the argument methodical. Today’s blog will only focus on one of these, the most conceptual of the three: “Toward a theory of evolution as multi-level learning.” (PNAS 2022, DOI: 10.1073/pnas.2120037119). I will be quoting the paper often my paraphrases will be clumsier than their clear prose.

 

What drives evolution in biology? Essentially, “solving optimization problems, which entails conflicts or trade-offs between optimization criteria at different levels or scales, leading to frustrated states…” There are three important pieces here: (1) optimization, (2) an interplay of distinct timescales in a hierarchical system, and (3) non-ergodicity that arises from competing interactions (the “frustrated states”).

 

The paper begins with seven basic principles. The first, and most important, is the existence of a “loss function of time-dependent variables that is minimized during evolution”. To stay alive is to solve an optimization problem. There needs to be some sort of learning that comes from an organism interacting with its environment. And when you’re dealing with the open unknown, the best solutions we know of involve “implementation of a stochastic learning algorithm”. Thus, learning and evolution are “optimization by trial and error”.

 

The next three principles, still very general, cover the following:

·      There’s a “hierarchy of scales”, each of which has its own “dynamical variables that change on different temporal scales”.

·      These timescales are distinct.

·      The faster-changing variables can be statistically defined by the slower-changing ones. (In thermodynamics, we can use macroscopic parameters to encompass the cacophony of the microscopic world.) This is known as Renormalization.

 

The final three principles are more specific to the living systems of Planet Earth, a sample size of one. “Evolving systems have the capacity to recruit additional variables that can be utilized to sustain the system and the ability to exclude variables that could destabilize the system.” Replication plays a key role in this process but requires the sequestering of “information-processing units”. Finally, there is two-way information flow between the slower-moving information units and the faster-changing data-collecting parts that interact with the environment. This, essentially, is learning how to stay alive.

 

What follows is an exposition of ten “basic phenomenological features of life” which the authors link to their theory of learning. I won’t go into these one-by-one, but rather pick out the concepts that I thought noteworthy. Let me preface this by commenting on the existence of a multiscale ‘situation’. It’s physico-chemical. The universe, and by extension Planet Earth, is made up of a mixture of multiple substances, which consists of molecules, which consists of atoms connected by vibrating bonds. Everything is in motion – it’s a dynamical system – but the timescales of motion are vastly different. Chemical bond vibrations are in the pico to femtosecond range. Molecules diffusing and colliding may be in the micro to nanosecond range. Nerves might pulse in a tenth of a second. I measure my time usage in minutes.

 

The inevitability of multiple distinct timescales is that different processes will ‘compete’ leading to frustrated states. But in addition to temporal frustration, there is also spatial frustration. This leads to a balance of sorts – an equilibrium, so to speak, but one that is semi-stable and may shift as the environment changes. Living systems that sequester their slow-changing informational systems that provide some organismal stability must continually receive and adapt to information being relayed from faster-changing ‘detector’ systems that interact directly with the environment. But neither of the two is privileged. It’s hard for us to think about this because we’re used to linear thinking of cause followed by effect. The lines of communication go both ways – a complex system where separation of the parts leads to death. You can’t separate the function of an organism from its genesis.

 

As to trial-and-error learning, the problem is it guarantees “neither finding the globally optimal solution nor retention of the optimal configuration when and if it is found. Rather stochastic optimization tends to rapidly find local optima and keeps the system in their vicinity”. Frustrated competing variables keep things that way. Not a necessarily bad thing since the environment will change. That’s why “biological evolution comprises numerous deleterious changes, comparatively rare beneficial changes and common neutral changes” that explain genetic drift, according to the authors. The diversity of local optima that arise from nonergodicity is why “evolution pushes organisms to explore and occupy all available niches and try all possible strategies”. We should expect a “diversity of solutions”.

 

Why do parasites show up? Diversity means that entities will arise that “scavenge information from the host” and “minimize their direct interface with the environment”. This may lead to symbiosis, but it might not. Competing imperatives are always in play. Why is there cell-programmed-death? There is an overall loss function to be minimized (the first principle!) and in a multicellular system, the tug-of-war between different scales could well result in reducing system failure by having individual cells (that have naturally accumulated problems – thanks, entropy) die off for the greater good.

 

To illustrate all this, the authors set up a (learning) neural network that has variables in different timescales. Some are ‘trainable’, others are not. There’s a logic to how they assign organismal versus environmental variables, and also how they divide the slower-changing variables into ones where changing them can be deleterious, neutral or adaptable. I won’t go into the math. Their conclusion: “slow variables determine the rules of the game, and changing these rules depending on the results of some particular games would be detrimental for the organism”, so it’s better to have “temporally stable rules” rather than an unconstrained optimization. But in the background, some of these rules can change. And as optimization continuously occurs dynamically, an environmental change may lead to a successful adaptation. Or it may lead to system failure.

 

Most important to me were the crucial setup of a system containing within it processes with distinct timescales, the inevitability of frustration, the role of renormalization, and determining what an appropriate loss function should be. The chemical systems I study (for which I build thermodynamic and kinetic maps) are likely too small. I’ve used the maximization of free energy reduction as the function to optimize, but perhaps I need to broaden or subdivide this into several mini-loss-functions. One of the papers that I didn’t discuss today goes into thermodynamics at a more general level and tries to connect it to the origin-of-life. They’re at a very abstract level, and the question is whether I can use their framework and fill in the details. It’s what keeps research interesting!