Brains, Computation And Thermodynamics: A View From The Future?

Progress in science often happens when two or more fields productively meet. Astrophysics got a huge boost when the tools of radio and radar met the age-old science of astronomy. From this fruitful marriage came things like the discovery of the radiation from the big bang. Another example was the union of biology with chemistry and quantum mechanics that gave rise to molecular biology. There is little doubt that some of the most important future discoveries in science in the future will similarly arise from the accidental fusion of multiple disciplines.

One such fusion sits on the horizon, largely underappreciated and unseen by the public. It is the fusion between physics, computer science and biology. More specifically, this fusion will likely see its greatest manifestation in the interplay between information theory, thermodynamics and neuroscience. My prediction is that this fusion will be every bit as important as any potential fusion of general relativity with quantum theory, and at least as important as the development of molecular biology in the mid 20th century. I also believe that this development will likely happen during my own lifetime.

The roots of this predicted marriage go back to 1867. In that year the great Scottish physicist James Clerk Maxwell proposed a thought experiment that was later called ‘Maxwell’s Demon’. Maxwell’s Demon was purportedly a way to defy the second law of thermodynamics that had been proposed a few years earlier. The second law of thermodynamics is one of the fundamental laws governing everything in the universe, from the birth of stars to the birth of babies. It basically states that left to itself, an isolated system will tend to go from a state of order to one of disorder. A good example is how a bottle of perfume wafts throughout a room with time. This order and disorder was quantified by a quantity called entropy.

In technical terms, the order and disorder refers to the number of states a system can exist in; order means fewer states and disorder means more. The second law states that isolated systems will always go from fewer states and lower entropy (order) to more states and higher entropy (disorder). Ludwig Boltzmann quantified this relationship with a simple equation carved on his tombstone in Vienna: S = klnW, where k is a constant called the Boltzmann constant, ln is the natural logarithm (to the base e) and W is the number of states.

Maxwell’s Demon was a mischievous creature which sat on top of a box with a partition in the middle. The box contains molecules of a gas which are ricocheting in every direction. Maxwell himself had found that these molecules’ velocities follow a particular distribution of fast and slow. The demon observes these velocities, and whenever there is a molecule moving faster than usual in the right side of the box, he opens the partition and lets it into the left side, quickly closing the partition. Similarly he lets in slower moving molecules from left to right. After some time, all the slow molecules will be in the right side and the fast ones will in the left. Now, velocity is related to temperature, so this means that one side of the box has heated up and the other has cooled down. To put it another way, the box went from a state of random disorder to order. According to the second law this means that the entropy of the system of the system decreased, which is impossible.

Maxwell’s demon seemingly contravenes the second law of thermodynamics (University of Pittsburgh)

For the next few years scientists tried to get around Maxwell’s Demon’s paradox, but it was in 1922 that the Hungarian physicist Leo Szilard made a dent in it when he was a graduate student hobnobbing with Einstein, Planck and other physicists in Berlin. Szilard realized an obvious truth that many others seem to have missed. The work and decision-making that the demon does to determine the velocities of the molecules itself generates entropy. If one takes this work into account, it turns out that the total entropy of the system has indeed increased. The second law is safe. Szilard later went on to a distinguished career as a nuclear physicist, patenting a refrigerator with Einstein and becoming the first person to think of a chain reaction.

Perhaps unknowingly, however, Szilard had also discovered a connection – a fusion of two fields – that was going to revolutionize both science and technology. When the demon does work to determine the velocities of molecules, the entropy that he creates comes not just from the raising and lowering of the partition but from his thinking processes, and these processes involve information processing. Szilard had discovered a crucial and tantalizing link between entropy and information. Two decades later, mathematician Claude Shannon was working at Bell Labs, trying to improve the communication of signals through wires. This was unsurprisingly an important problem for a telephone and communications company. The problem was that when engineers were trying to send a message over a wire, it would lose its quality because of many factors including noise. One of Shannon’s jobs was to figure out how to make this transmission more efficient.

Shannon found out that there is a quantity that relates to the information transmitted over the wire. In crude terms, this quantity was inversely related to the information as well as to the probability of transmitting that information; the higher the probability of transmitting accurate information over a channel, the lower this quantity was and vice versa. When Shannon showed his result to the famous mathematician John von Neumann, von Neumann with his well-known lightning-fast ability to connect disparate ideas, immediately saw what it was: “You should call your function ‘entropy’”, he said, “firstly because that is what it looks like in thermodynamics, and secondly because nobody really knows what entropy is, so in a debate you will always have the upper hand.” Thus was born the connection between information and entropy. Another fortuitous connection was born – between information, entropy and error or uncertainty. The greater the uncertainty in transmitting a message, the greater the entropy, so entropy also provided a way to quantify error. Shannon’s 1948 paper, “A Mathematical Theory of Communication”, was a seminal publication and has been called the Magna Carta of the information age.

Even before Shannon, another pioneer had published a paper that laid the foundations of the theory of computing. In 1936 Alan Turing published “On Computable Numbers, with an Application to the Entscheidungsproblem”. This paper introduced the concept of Turing machines which also process information. But neither Turing nor von Neumann really made the connection between computation, entropy and information explicit. Making it explicit would take another few decades. But during those decades, another fascinating connection between thermodynamics and information would be discovered.

Stephen Hawking’s tombstone at Westminster Abbey (Cambridge News)

That connection came from Stephen Hawking getting annoyed. Hawking was one of the pioneers of black holes, and along with Roger Penrose he had discovered that at the center of every black hole is a singularity that warps spacetime infinitely. The boundary of the black hole is its event horizon and within that boundary not even light can escape. But black holes posed some fundamental problems for thermodynamics. Every object contains entropy, so when an object disappears into a black hole, where does its entropy go? If the entropy of the black hole does not increase then the second law of thermodynamics would be violated. Hawking had proven that the area of a black hole’s event horizon never decreases, but he had pushed the thermodynamic question under the rug. In 1972 at a physics summer school, Hawking met a graduate student from Princeton named Jacob Bekenstein who proposed that the increasing area of the black hole was basically a proxy for its increasing entropy. This annoyed Hawking and he did not believe it because increased entropy is related to heat (heat is the highest- entropy form of energy) and black holes, being black, could not radiate heat. With two colleagues Hawking set out to prove Bekenstein wrong. In the process, he not only proved him right but also made what is considered his greatest breakthrough: he gave black holes a temperature. Hawking found out that black holes do emit thermal radiation. This radiation can be explained when you take quantum mechanics into account. The Hawking-Bekenstein discovery was a spectacular example of another fusion: between information, thermodynamics, quantum mechanics and general relativity. Hawking deemed it so important that he wanted to put it on his tombstone in Westminster Abbey, and so it has been.

This short digression was to show that more links between information, thermodynamics and other disciplines were being forged in the 1960s and 70s. But nobody saw the connections between computation and thermodynamics until Rolf Landauer and Charles Bennett came along. Bennett and Landauer were both working at IBM. Landauer was an émigré who fled from Nazi Germany before working for the US Navy as an electrician’s mate, getting his PhD at Harvard and joining IBM. IBM was then a pioneer of computing; among other things they had built computers for the Manhattan Project. In 1961, Landauer published a paper titled “Irreversibility and Heat Generation in the Computing Process” that is destined to become a classic of science. In it, Landauer established that the basic act of computation – the change of one bit to another, say a 1 to a 0 – requires a bare minimum amount of entropy. He quantified this amount with another simple equation: S = kln2, with k again being the Boltzmann constant and ln the natural logarithm. This has become known as the Landauer bound; it is the absolute minimum amount of entropy that has to be expended in a single bit operation. Landauer died in 1999 and as far as I know the equation is not carved on his tombstone.

The Landauer bound applies to all kinds of computation in principle and biological processes are also a form of information processing and computation, so it’s tantalizing to ask whether Landauer’s calculation applies to them. Enter Charles Bennett. Bennett is one of the most famous scientists whose name you may not have heard of. He is not only one of the fathers of quantum computing and quantum cryptography but he is also one of the two fathers of the marriage of thermodynamics with computation, Landauer being the other. Working with Landauer in the 1970s and 80s, Bennett applied thermodynamics to both Turing machines and biology. By good fortune he had gotten his PhD in physical chemistry studying the motion of molecules, so his background primed him to apply ideas from computation to biology.

Charles Bennett from IBM has revolutionized our understanding of the thermodynamics of computation (AMSS)

To simplify matters, Bennett considered what he called a Brownian Turing machine. Brownian motion is the random motion of atoms and molecules. A Brownian Turing machine can write and erase characters on a tape using energy extracted from a random environment. This makes the Brownian Turing machine reversible. A reversible process might seem strange, but in fact it’s found in biology all the time. Enzyme reactions occur from the reversible motion of chemicals – at equilibrium there is equal probability that an enzymatic reaction will go forward or backward. What makes these processes irreversible is the addition of starting materials or the elimination of chemical products. Even in computation, only a process which erases bits is truly irreversible because you lose information. Bennett envisaged a biological process like protein translation as a Brownian Turing machine which adds or subtracts a molecule like an amino acid, and he calculated the energy and entropy expenditures involved in running this machine. Visualizing translation as a Turing machine made it easier to do a head-to-head comparison between biological processes and bit operations. Bennett found out that if the process is reversible the Landauer bound does not hold and there is no minimum entropy required. Real life of course is irreversible, so how do real-life processes compare to the Landauer bound?

In 2017, a group of researchers published a fascinating paper in the Philosophical Transactions of the Royal Society in which they explicitly calculated the thermodynamic efficiency of biological processes. Remarkably, they found that the efficiency of protein translation is several orders of magnitude better than the best supercomputers, in some cases as better as a million fold. More remarkably, they found that the efficiency is only one order of magnitude worse than the theoretical minimum Landauer bound. In other words, evolution has done one hell of a job in optimizing the thermodynamic efficiency of biological processes.

But not all biological processes. Circling back to the thinking processes of Maxwell’s little demon, how does this efficiency compare to the efficiency of the human brain? Surprisingly, it turns out that neural processes like the firing of synapses are estimated to be much worse than protein translation and more comparable to the efficiency of supercomputers. At first glance, the human brain thus appears to be worse than other biological processes. However, this seemingly low computational efficiency of the brain must be compared to its complex structure and function. The brain weighs only about a fiftieth of the weight of an average human but it uses up 20% of the body’s energy. It might seem that we are simply not getting the biggest bang for our buck, with an energy-hungry brain providing low computational efficiency. What would explain this inefficiency and this paradox?

My guess is that the brain has been designed to be inefficient through a combination of evolutionary accident and design and that efficiency is the wrong metric for gauging the performance of the brain. Efficiency is the wrong metric because thinking of the brain in digital terms is the wrong metric. The brain arose through a series of modular inventions responding to new environments created by both biology and culture. We now know that thriving in these environments needed a combination of analog and digital functions.; for instance, the nerve impulses controlling blood pressure are digital while the actual change in pressure is continuous and analog. It is likely that digital neuronal firing is built on an analog substrate of wet matter, and that higher-order analog functions could be emergent forms of digital neuronal firing. As early as the 1950s, von Neumann conjectured that we would need to model the brain as both analog and digital in order to understand it. Around the time that Bennett was working out the thermodynamics of computation, two mathematicians named Marian Pour-El and Ian Richards proved a very interesting theorem which showed that in certain cases, there are numbers that are not computable with digital computers but are computable with analog processes; analog computers are thus more powerful in such cases.

If our brains are a combination of digital and analog, it’s very likely that they are this way so that they can span a much bigger range of computation. But this bigger range would come at the expense of inefficiency in the analog computation process. The small price of lower computational efficiency as measured by the Landauer bound would come at the expense of the much greater evolutionary benefits of performing complex calculations that allow us to farm, build cities, know stranger from kin and develop technology. Essentially, the Landauer bound could be evidence for the analog nature of our brains. There is another interesting fact about analog computation, which is its greater error rate; digital computers took off precisely because they had low error rates. How does the brain function so well in spite of this relatively high error rate? Is the brain consolidating this error when we dream? And can we reduce this error rate by improving the brain’s efficiency? Would that make our brains better or worse at grasping the world?

From the origins of thermodynamics and Maxwell’s Demon to the fusion of thermodynamics with information processing, black holes, computation and biology, we have come a long way. The fusion of thermodynamics and computation with neuroscience just seems to be beginning, so for a young person starting out in the field the possibilities are exciting and limitless. A multitude of general questions abound: How does the efficiency of the brain relate to its computational abilities? What might be the evolutionary origins of such abilities? What analogies between the processing of information in our memories and that in computers might we discover through this analysis? And finally, just like Shannon did for information, Hawking and Bekenstein did for black holes and Landauer and Bennett did for computation and biology, can we find out a simple equation describing how the entropy of thought processes relates to simple neural parameters connected to memory, thinking, empathy and emotion? I do not know the answers to these questions, but I am hoping someone who is reading this will, and at the very least they will then be able to immortalize themselves by putting another simple formula describing the secrets of the universe on their tombstone.

Further reading:

1. Charles Bennett – The Thermodynamics of Computation
2. Seth Lloyd – Ultimate Physical Limits to Computation
3. Freeman Dyson – Are brains analog or digital?
4. George Dyson – Analogia: The Emergence of Technology Beyond Programmable Control (August 2020)
5. Richard Feynman – The Feynman Lectures on Computation (Chapter 5)
6. John von Neumann – The General and Logical Theory of Automata

First published on 3 Quarks Daily

The only two equations that you should know

“Chemistry”, declared the Nobel laureate Roger Kornberg in an interview, “is the queen of all sciences. Our best hope of applying physical principles to the world around us is at the level of chemistry. In fact if there is one subject which an educated person should know in the world it is chemistry.” Kornberg won the 2006 Nobel Prize in chemistry for his work on transcription which involved unraveling the more than dozen complicated proteins involved in the copying of DNA into RNA. He would know how important chemistry is in uncovering the details of a ubiquitous life process.

I must therefore inevitably take my cue from Kornberg and ask the following question: What equation would you regard as the most important one in science? For most people the answer to this question would be easy: Einstein’s famous mass-energy formula, E=mc². Some people may cite Newton’s inverse square law of gravitation. And yet it should be noted that both of these equations are virtually irrelevant for the vast majority of practicing physicists, chemists and biologists. They are familiar to the public mainly because they have been widely publicized and are associated with two very famous scientists. There is no doubt that both Einstein and Newton are supremely important for understanding the universe, but they both suffer from the limitations of reductionist science that preclude the direct application of the principles of physics to the everyday workings of life and matter.

Take Einstein’s formula for instance. About the only importance it has for most physical scientists is the fact that it is responsible for the nuclear processes that have forged the elements in stars and supernova. Chemists deal with reactions that involve not nuclear processes but the redistribution of electrons. Except in certain special cases, Einstein therefore does not figure in chemical or biological processes. Newton’s gravitational formula is equally distant for most chemists' everyday concerns. Chemistry hinges on the attraction and repulsion of charges, processes overwhelmingly governed by the electromagnetic force. This force is stronger than the gravitational force by a factor of 10³⁶, an unimaginably large number. Gravity is thus too weak for chemists and biologists to bother with in their work. The same goes for many physicists who deal with atomic and molecular interactions.

Instead here are two equations which have a far greater and more direct relevance to the work done by most physical and biological scientists. The equations lie at the boundary of physics and chemistry, and both of them are derived from a science whose basic truths are so permanently carved in stone that Einstein thought they would never, ever need to be modified. The man who contributed the most to their conception, Josiah Willard Gibbs, was called "the greatest mind in American science" by Einstein. The science that Gibbs, Helmholtz, Clausius, Boltzmann and others created is thermodynamics, and the equations we are talking about involve its most basic quantities. They apply without exception to every important physical and chemical process you can think of, from the capture of solar energy by plants and solar cells to the combustion of fuel inside trucks and human bodies to the union between sperm and egg.

Two thermodynamic quantities govern molecular behavior, and indeed the behavior of all matter in the universe. One is the enthalpy, usually denoted by the symbol H, and roughly representing the quantity of energy and the strength of interactions and bonds between different atoms and molecules. The other is the entropy, usually denoted by the symbol S, and roughly representing the quality of energy and the disorder in any system. Together the enthalpy and entropy make up the free energy G, which roughly denotes the amount of useful work that can be extracted from any living or non-living system. In practical calculations, what we are concerned with are changes in these quantities rather than their absolute values, so each one of them is prefaced by the symbol ∆, indicating change. The celebrated second law of thermodynamics states that the entropy of a spontaneous process always increases, and it is indeed one of the universal facts of life, but that is not what we are concerned with here.

Think about what happens when two molecules – of any kind – interact with each other. The interaction need not even be an actual reaction, it can simply be the binding of two molecules to one another by strong or weak forces. The process is symbolized by an equilibrium constant K_e, which is simply the ratio of the concentrations of the products of the reaction to the starting materials (reactants). The bigger the equilibrium constant, the more the amount of the products. K_e thus tells us how much of a reaction has been completed and how much reactant has been converted to product. Our first great equation relates this equilibrium constant to the free energy of the interaction through the following formula:

∆G⁰ = -RT ln K_e

or, in other words

K_e = e^-∆G0/RT

Here ln is the natural logarithm to base e, R is a fundamental constant called the gas constant, T is the ambient temperature and ∆G₀ is the free energy change under so-called 'standard conditions' (a detail which can be ignored by the reader for the sake of this discussion). This equation tells us two major things and one minor thing. The minor thing is that reactions can be driven in particular directions by temperature increases, and exponentially so. But the major things are what's critical here. Firstly, the equation says that the free energy in a spontaneous process with a favorable positive equilibrium constant is always going to be negative; the more negative it is the better. And that is what you find. The free energy change for many of biology's existential reactions like the coupling of biological molecules with ATP (the “energy currency” of the cell), the process of electron transfer mediated by chlorophyll and the oxidation of glucose to provide energy is indeed negative. Life has also worked out ingenious little tricks to couple reactions with positive (unfavorable) ∆G changes to those with negative ∆G0 values to give an overall favorable free energy profile.

The second feature of the equation is a testament to the wonder that is life, and it never ceases to amaze me. It attests to what scientists and philosophers have called “fine-tuning”, the fact that evolution has somehow succeeded in minimizing the error inherent in life’s processes, in carefully reining in the operations of life to within a narrow window. Look again at that expression. It says that ∆G₀ is related to K_e not linearly but exponentially. That is a dangerous proposition because it means that even a tiny change in ∆G₀ will correspond to a large change in K_e. How tiny? It should be no bigger than 3 kcal/mol.

A brief digression to appreciate how small this value is. Energies in chemistry are usually expressed as kilocalories per mole. A bond between two carbon atoms is about 80 kcal/mol. A bond between two nitrogen atoms is 226 kcal/mol: this is why nitrogen can be converted to ammonia by breaking this bond only at very high temperatures and pressures and in the presence of a catalyst. A hydrogen bond - the "glue" that holds biological molecules like DNA and proteins together - is anywhere between 2 and 10 kcal/mol.

3 kcal/mol is thus a fraction of the typical energy of a bond. It takes just a little jiggling around to overcome this energy barrier. The exponential, highly sensitive dependence of Ke on ∆G₀ means that changing ∆G from close to zero to 3 kcal/mol will translate to changing K_e from 1:99.98 in favor of products to 99.98:1 in favor of reactants (remember that K_e is a ratio). This is a simple mathematical truth. Thus, a tiny change in ∆G₀ can all but completely shift a chemical reaction from favoring products to favoring reactants. Naturally this will be very bad if the goal of a reaction is to create products that are funneled into the next chemical reaction. Little changes in the free energy can therefore radically alter the flux of matter and energy in life’s workings. But this does not happen. Evolution has fine-tuned life so well that it has remained a game played within a 3 kcal/mol energy window for more than 2.5 billion years. It's so easy for this game to quickly spiral out of hand, but it doesn’t. It doesn’t for the trillions of chemical transactions which trillions of cells execute everyday in every single organism on this planet.

And it doesn’t happen for a reason; because cells would have a very hard time modulating their key chemical reactions if the free energies involved in those reactions had been too large. Life would be quickly put into a death trap if every time it had to react, fight, move or procreate it had to suddenly change free energies for each of its processes by tens of kilocalories per mole. There are lots of bonds broken and formed in biochemical events, of course, and as we saw before, these bond energies can easily amount to dozens of kcals/mol. But the tendency of the reactants or products containing those bonds to accumulate is governed by these tiny changes in free energy which nudge a reaction one way or another. In one sense then, life is optimizing small changes (in free energy of reactions) between two large numbers (bond energies). This is always a balancing act on the edge of a cliff, and life has managed to be successful in it for billions of years. It's one of the great miracles of the universe.

The second equation is also a relationship between free energy, enthalpy and entropy. It's simpler than the first, but no less important:

∆G = ∆H - T∆S

The reason this equation is also crucial to the operation of the universe is because it depicts a fine dance between entropy and enthalpy that dictates whether physical processes will happen. Note that entropy is multiplied by the temperature here and the sign is negative. So if it decreases in a process then ∆S becomes negative and the overall product (T∆S) becomes positive. In that case the change in enthalpy needs to be negative enough to compensate, otherwise the free energy will not be negative and the process won't take place. 

For instance, consider the schoolboy experiment of oil and water not mixing. When oil is put into water, the water molecules have to order themselves around oil molecules, leading their entropy to decrease and become negative. The attraction between water and oil on the other hand is weak, so the change in enthalpy does not compensate for the change in entropy, and oil does not mix. This is called the hydrophobic effect. It's a fundamental effect governing a myriad of critical phenomena; drugs interacting with signaling proteins, detergents interacting with grease, food particles attracting or repelling each other inside saucepans and human bodies. On the other hand, salt and water mix easily; in this case, while the entropy is still unfavorable because of the ordering of water molecules around salt molecules, the enthalpy is overwhelmingly favorable (negative) because the positive and negatively charged sodium and chloride ions strongly attract water.

Because temperature is part of the equation it too plays an important role. For instance consider a phenomenon like a chemical reaction in which the change in entropy is favorable but quite small. We can then imagine that this reaction will be greatly accelerated if T is high, making the product of it and the entropy large. This explains why the free energy of chemical reactions can be made much more favorable at high temperatures (there is a subtlety here, however: making the free energy more favorable is not the same as accelerating the reactions, it's simply making the products more stable. The difference is between thermodynamics and kinetics).

Even the origin of life during which the exact nature of molecular interactions was crucial in deciding which ones would survive, replicate and thrive was critically dependent on enthalpy and entropy. When little oily molecules called micelles repelled water molecules because of the unfavorable entropy and enthalpy described above, they sequestered themselves into tiny bags inside which fragile molecules like DNA and RNA could safely isolate themselves from the surrounding water. These DNA and RNA molecules could then experiment with copying themselves at leisure, not having to worry about being hydrolyzed by water. The ones with higher fitness survived, kickstarting the process which, billions of years later, finally led to this biped typing these words on his computer.

That's really all there is to life. We all thus hum along smoothly, beneficiaries of a 3 kilocalorie energy window and of the intricate dance of entropy and enthalpy, going about our lives even as we are held hostage to the quirks of thermodynamic optimization, walking along an exponential energy precipice.

And all because K_e = e^-∆G₀/RT

Macrocycles, flexibility and biological activity: A tortuous pairing

Here's an interesting paper from the Jacobson, Wells and Walsh labs at UCSF and Stanford that seeks to demonstrate how restricting the flexibility of macrocycles may lead to better inhibition of their targets from an entropic perspective. The authors are looking at a non-ribosomal peptide called thiocillin which inhibits the growth of Gram positive bacteria, especially MRSA.

What they wanted to determine was the effect of point mutations in the peptide on the inhibition. They performed saturation mutagenesis between positions 2 and 9 of the peptide and generated 152 mutants whose activities they tested in a minimum inhibitory concentration (MIC) assay. They found that 8 point mutants especially resulted in more potent analogs.

Now there can be several reasons why the potency went up, but one potential reason is entropy. Macrocycles, while often more rigid than their corresponding linear analogs, are still quite flexible. In fact, my own work with the macrocycle dictyostatin in graduate school showed how flexible even a supposedly constrained molecule can be. What this paper finds out is that in cases where the mutant lost activity, there was a corresponding increase in flexibility and entropy as measured by the number and distinctive nature of conformations from a conformational search technique which they have developed. Particularly striking changes in potency occurred when a single residue was modified from having a planar sp2 carbon to a non-planar sp3 carbon: in that case the saturated analog had many more conformations than the unsaturated one.

As someone who has always been partial to the impact of entropy and conformational flexibility on molecular activity, I like this kind of work. But I am not quite convinced yet that it is decreased flexibility that leads to more potent inhibition. For one thing, inhibition is not direct binding, and there are a variety of factors including changes in cell permeability and off target effects that could lead to the observed changes in inhibitory - not binding - affinity. Secondly, there were 152 mutants, and it's not clear to me how many were tested for flexibility: in other words, I am not sure there were enough controls to determine whether the flexibility-inhibition correlation really holds up. For instance, many of the mutants were inactive: were there instances in which some of these were actually less flexible and challenged the hypothesis? Another way to put it is to ask what the right null model for this dataset is. 

Thirdly, decreased or increased inhibition can be a result of both more conformations as well as conformational selection. For instance, two macrocycles can have similar conformations, but in one case a particular conformation more suitable for binding could be more stable (perhaps because of an intramolecular hydrogen bond) and represented to a higher degree in solution, making it easier for a protein target to pick it out. Lastly, it is not clear whether the improved affinity could simply have been a result of better interactions: although that seems unlikely for the sp2 vs sp3 pair above, it is nonetheless a factor that could be operating in other cases.

Entropy is an important consideration in drug design, but it's also trickier than it sounds to both understand its effects and implement its benefits. To their credit the authors acknowledge that rigidity is a necessary but not sufficient condition for increased affinity, and other studies seem to bear it out. Macrocyclization can also be counterintuitive: for instance in my own studies I found out that dictyostatin which is a macrocycle seems more flexible than its corresponding acyclic counterpart discodermolide. In that case it was fairly straightforward syn-pentane interactions which made the acyclic molecule rigid. In other cases it could be the opposite. In any case, this study serves as an interesting starting point for exploring the impact of flexibility on drug affinity, but it also serves to illustrate how thick the jungle of SAR really is.

The only science equation that you should know

As I mentioned in my last post, “Chemistry”, declared Roger Kornberg in an interview, “is the queen of all sciences. Our best hope of applying physical principles to the world around us is at the level of chemistry. In fact if there is one subject which an educated person should know in the world it is chemistry.” Kornberg won the 2006 Nobel Prize in chemistry for his work on transcription which involved unraveling the more than dozen complicated proteins involved in the copying of DNA into RNA. He would know how important chemistry is in uncovering the details of a ubiquitous life process.

I must therefore inevitably take my cue from Kornberg and ask the following question: What equation would you regard as the most important one in science? For most people the answer to this question would be easy: Einstein’s famous mass-energy formula, E=mc². Some people may cite Newton’s inverse square law of gravitation. And yet it should be noted that both of these equations are virtually irrelevant for the vast majority of practicing physicists, chemists and biologists. They are familiar to the public mainly because they have been widely publicized and are associated with two very famous scientists. There is no doubt that both Einstein and Newton are supremely important for understanding the universe, but they both suffer from the limitations of reductionist science that preclude the direct application of the principles of physics to the everyday workings of life and matter.

Take Einstein’s formula for instance. About the only importance it has for most physical scientists is the fact that it is responsible for the nuclear processes that have forged the elements in stars and supernova. Chemists deal with reactions that involve not nuclear processes but the redistribution of electrons. Except in certain cases, Einstein therefore does not figure in chemical or biological processes. Newton’s gravitational formula is equally distant. Chemical reactions involve the attraction and repulsion of charges which are processes governed by the electromagnetic force. This force is stronger than the gravitational force by a factor of 10³⁶, an unimaginable number. Thus gravity is too weak for chemists and biologists to bother with it in their work. The same goes for many physicists who deal with atomic and molecular interactions.

Instead here are two equations which have a far greater and more direct relevance to the work done by most physical and biological scientists. The equations lie at the boundary of physics and chemistry, and both of them are derived from a science whose basic truths are so permanently carved in stone that Einstein thought they would never, ever need to be modified. That science is thermodynamics, and the equations we are talking about involve the most basic variables in thermodynamics. They apply without exception to every important physical and chemical process you can think of, from the capture of solar energy by plants and solar cells to the combustion of fuel inside trucks and human bodies to the union between sperm and egg.

Two thermodynamic quantities govern molecular behavior, and indeed the behavior of all matter in the universe. One is the enthalpy, usually denoted by the symbol H, and roughly representing the quantity of energy and the strength of interactions and bonds between different atoms and molecules. The other is the entropy, usually denoted by the symbol S, and roughly representing the quality of energy and the disorder in any system. Together the enthalpy and entropy make up the free energy G, which roughly denotes the amount of useful work that can be extracted from any living or non-living system. In practical calculations what we are concerned with are changes in these quantities rather than their absolute values, so each one of them is prefaced by the symbol ∆ indicating change. The celebrated second law of thermodynamics states that the entropy of a spontaneous process always increases, and it is indeed one of the universal facts of life, but that is not what we are concerned with here.

Think about what happens when two molecules – of any kind – interact with each other. It need not even be an actual reaction, it can simply be the binding of two molecules to one another by strong or weak forces. The interaction is symbolized by an equilibrium constant K_e, which is simply the ratio of the concentrations of the products of the reaction to the starting material (reactants). The bigger the equilibrium constant, the more the amount of the products. K_e thus tells us how much of a reaction has been completed, how much reactant has been converted to product. Our first great equation relates this equilibrium constant to the free energy of the interaction through the following formula:

∆G⁰ = -RT ln K_e

or, in other words

K_e = e^-∆G0/RT

Here ln is the natural logarithm to base e, R is a fundamental constant called the gas constant, T is the ambient temperature and ∆G₀ is the free energy change under so-called 'standard conditions' (the details of these are not very important for understanding the crux of the matter here). 

This equation tells us two major things and one minor thing. The minor thing is that reactions can be driven in particular directions by temperature increases, and exponentially so (that's not the same as speeding them up though; this goal is the domain of kinetics, not thermodynamics). But the major things are what's critical here. Firstly the equation says that the free energy in a spontaneous process with a favorable positive equilibrium constant is always going to be negative; the more negative it is the better. And that is what you find. The free energy change for many of biology's existential reactions like the coupling of biological molecules with ATP (the “energy currency” of the cell), the process of electron transfer mediated by chlorophyll and the oxidation of glucose to provide energy is indeed negative. Life has also worked out clever little tricks to couple reactions with positive (unfavorable) ∆G changes to those with negative ∆G₀ values to give an overall favorable free energy profile.

The second feature of the equation is a testament to the wonder that is life, and it never ceases to amaze me. It attests to what scientists and philosophers have called “fine-tuning” the fact that evolution has somehow succeeded in minimizing the error inherent in life’s processes, in carefully reining in the operations of life within a narrow window. Look again at that expression. It says that ∆G₀ is related to K_e not linearly but exponentially. That is a dangerous proposition because it means that even a tiny change in ∆G₀ will correspond to a large change in K_e. How tiny? No bigger than 3 kcal/mol.

A brief digression to appreciate how small this value is. Energies in chemistry are usually expressed as kilocalories per mole. A bond between two carbon atoms is about 80 kcal/mol. A bond between two nitrogen atoms is 226 kcal/mol, indicating why nitrogen can be converted to ammonia by breaking this bond only at very high temperatures and pressures and in the presence of a catalyst. A hydrogen bond - the "glue" that holds biological molecules like DNA and proteins together - is anywhere between 2 and 10 kcal/mol.

3 kcal/mol is thus a fraction of the typical energy of a bond. It takes just a little jiggling around to overcome this energy barrier; if you ask a chemist to predict or optimize a reaction within this range she will be extremely uncomfortable. One of the reasons drug designers have such a hard time designing drugs that will bind tightly to proteins is precisely because it's so hard to predict and control the interactions of their drugs with those proteins down to such a small number. The exponential, highly sensitive dependence of Ke on ∆G₀ means that changing ∆G from close to zero to 3 kcal/mol will translate to changing K_e from 1:99.98 in favor of products to 99.98:1 in favor of reactants (remember that K_e is a ratio). It's not even chemistry, actually, it's a simple mathematical truth. Thus, a tiny change in ∆G₀ can all but completely shift a chemical reaction from favoring products to favoring reactants.

Naturally this will be very bad if the goal of a reaction is to create products that are funneled into the next chemical reaction. Little changes in the free energy can therefore radically alter the flux of matter and energy in life’s workings. But this does not happen. Evolution has fine-tuned life so well that it has remained a game played within a 3 kcal/mol energy window for more than 2.5 billion years. It's so easy for this game to quickly spiral out of hand, but it doesn’t. It doesn’t happen for the trillions of chemical transactions which trillions of cells execute everyday in every single organism on this planet.

And it doesn’t happen for a reason; because cells would have a very hard time modulating their key chemical reactions if the free energies involved in those reactions had been too large. Just like we manage to maintain our body temperature between an alarmingly narrow window of comfort, so we also manage to maintain the sprinkling of energy in our essential cellular processes to within 3 kcal/mol. Life would be quickly put into a death trap if every time it had to react, fight, move or procreate it had to suddenly change free energies for each of its processes by tens of kilocalories per mole.

There are lots of bonds broken and formed in biochemical events, of course, and as we saw before, these bond energies can easily amount to dozens of kcals/mol. But the tendency of the reactants or products containing those bonds to accumulate is governed by these tiny changes in free energy which nudge a reaction one way or another. In one sense then, life is optimizing small changes (in free energy of reactions) between two large numbers (bond energies). This is always a balancing act on the edge of a cliff, and life has managed to be successful in it for billions of years.

Thus we all hum along smoothly, beneficiaries of a 3 kcal/mol energy window, going about our lives even as we are held hostage to the quirks of thermodynamic optimization, walking along an exponential energy precipice. And all because

K_e = e^-∆G₀^/RT

^{This is a revised version of an older post.}