Why Technology (and AI) Won’t Save Biology

There seems to be no end to biology's explosive progress. Genomes can now be read, edited, and rewritten with unprecedented scope. Individual neurons can now be studied in both space and time. Mathematical models help us understand the dynamics of virus spread and ecological populations, and vaccines for deadly diseases like HIV and Ebola seem to hold more promise than ever. Many say the twentieth century belonged to physics and the twenty-first belongs to biology, and indeed, this sentiment seems to be borne out by the astonishing advances in the biological sciences.

Six Major Revolutions in Biology

Over the past five hundred years, biology has undergone six major revolutions that transformed our understanding of life. The first was Linnaeus’s classification of organisms into binomial nomenclature. The second was the invention of the microscope by pioneers like Hooke and Leeuwenhoek. The third was the discovery of cells' composition in health and disease by Schwann and Schleiden, thanks to advancements in microscopy. The fourth was Darwin’s formulation of evolution by natural selection. The fifth was Mendel’s discovery of the laws of heredity. And the sixth was the discovery of the structure of DNA by Watson, Crick, and others. An ongoing seventh revolution centers around mapping genomes and understanding their implications for disease and ecology. This has been aided by breakthroughs in statistics and new imaging techniques like MRI and CT scans.

These revolutions were driven not just by new ideas, but also by new tools—a pattern that underscores the intertwined nature of scientific progress. Scientific revolutions are a two-pronged affair: breakthroughs come both from new paradigms of thinking and new technological advances.

The Role of Technology in Biological Discoveries

From the microscope to electron microscopy, x-ray diffraction, and modern imaging techniques, four of the six revolutions in biology hinged on technological advancements. The invention of tools that enabled deeper observation revealed new truths that theory could then catch up with. In genomics, for example, rapid sequencing methods paired with powerful computers and statistical techniques for identifying rare events have unlocked a flood of genetic data. However, this bounty of data brings its own challenges: it hides gems of understanding within mountains of information. Systems biology seeks to make sense of this deluge by piecing together the puzzle at different levels, but even this approach highlights the gaps between what we can observe and what we can truly understand.

The Promise and Peril of Artificial Intelligence in Biology

This is where artificial intelligence (AI) enters the conversation. AI promises to address biology’s complexity by analyzing vast datasets and finding patterns that elude human comprehension. Indeed, AI and machine learning have already shown promise in areas like genomics, protein folding, and drug discovery. For instance, AlphaFold, a machine learning algorithm, achieved a breakthrough in predicting protein structures, which had remained a tough challenge in biology for decades. AI models also excel at recognizing anomalies in medical imaging, predicting the spread of pandemics, and optimizing experimental designs.

However, while AI offers immense potential, it also amplifies existing challenges. The major limitation lies in the fundamental difference between recognizing patterns and understanding causation. AI systems excel at correlating data and identifying intricate relationships, but these correlations do not necessarily reveal underlying biological mechanisms. In fields like cancer genomics, sequencing technologies have revealed hundreds of mutated genes, and AI can help sift through these to find correlations between certain mutations and cancer progression. But beyond identifying these correlations, biologists must still build causal frameworks to explain why and how these mutations impact cancer growth. Without such theoretical underpinnings, AI runs the risk of creating a black box where patterns are identified without leading to genuine understanding. Even in terms of strict utility, these patterns may be inadequate since prediction without understanding can lead to blind spots; for instance witness "activity cliffs" in medicinal chemistry or missed interaction partners in cancer biology, both of which can thwart true prediction.

The very nature of AI models may also reveal their limitations. For instance, LLMs and other leading AI models are trained and based on discrete, binary data, while biology is full of continuous data; something as simple as dose-response curves testifies to this fact. The difference between digital and analog systems comes to mind: computers are digital, but life is a hybrid of analog and digital, combining digital outputs built on analog substrates and vice versa. In a previous post, I argued that the relative thermodynamic inefficiency of the brain might point to its analog workings. If AI were to truly transform drug discovery and biology, its discrete digital models would have to learn to deal with continuous, analog data.

This challenge is not new. Sydney Brenner once pointed out that biology in the 1950s used to be "low input, low throughput, high output," whereas today it's often "low input, high throughput, no output." While AI has the potential to plug gaps in improving output, it can also exacerbate this trend by focusing on high throughput without significant output. Just as we once mistook data accumulation for understanding, we now risk mistaking AI-generated correlations for genuine scientific insight. AI-driven discoveries must therefore be complemented with theoretical and experimental models that go beyond pattern recognition.

Reductionism and Emergence in the Era of AI

One of the key reasons why technology—and now AI—hasn’t “saved” biology lies in the fundamental philosophy of reductionism. Reductionism, the idea of breaking down complex systems into simpler parts, has been the great legacy of twentieth-century science. However, as complexity theorists like Philip Anderson and Stuart Kauffman pointed out, complex systems often display emergent properties that can’t be deduced from their individual components. AI tools, which mostly operate on reductionist principles by breaking down biological data into discrete elements, are thus poorly equipped to handle emergent phenomena.

For example, while AI can analyze neuronal firing patterns and model brain networks, it struggles to explain higher-order cognitive functions and consciousness. Mirror neurons are a case in point: AI can track their activation across brain regions, but understanding their role in human empathy or social behavior remains elusive. Neuroscientists like John Krakauer argue that such limitations reflect the broader challenge of understanding emergent properties within biological systems.

The Future: An Integrated Approach

For AI to truly revolutionize biology, it must not merely amplify reductionist methods but enable a more integrated approach to biological understanding. This involves studying biological systems at multiple levels of organization - both digital and analog - and creating AI models that can reconcile these levels. For example, in neuroscience, we must bridge low-level recordings of single neurons with models of neuronal clusters and behavioral observations of entire organisms. Similarly, in fields like genomics, AI must work alongside biologists to develop causal models that connect molecular changes with physiological and ecological consequences. This is why organizations that center their entire product or discovery pipeline around AI need to be careful and avoid the proverbial pitfall of having everything look like a nail when they have a hammer.

The promise of AI in biology also depends on fostering closer interdisciplinary collaborations. Physicists, computer scientists, biologists, and psychologists need to work together to build models that combine reductionist and holistic perspectives. AI should serve as a tool that integrates data from different levels, facilitating not just more precise measurements, but more comprehensive theories.

Final thoughts

As the late biologist Carl Woese observed, living systems are not simply collections of molecular machines, but resilient patterns in a turbulent flow. In his elegant essay titled “A New Biology for A New Century,” Woese argued for a broader perspective that sees organisms as complex, dynamic organizations. AI must help us move toward this vision by integrating the patterns it finds with deeper theoretical insights, enabling a biology that goes beyond the sum of its parts.

The era of AI in biology holds great promise, but it also poses new risks of mistaking data accumulation for understanding. AI must be seen not as a savior of biological sciences, but as a vital partner in building bridges between different levels of biological understanding. By combining AI’s pattern recognition with theoretical frameworks that address emergence and historical contingency, we can move toward a more integrated, holistic view of life.

As technology advances, we must remember the limitations of reductionism and the dangers of technological solutionism. AI’s role should not be to replace human understanding, but to augment and expand it. Just as we harness the tools of the past revolutions in biology, we must now harness AI to not just process data, but to deepen our understanding of the flow of life itself.

Image link: https://scitechdaily.com/ai-reveals-previously-unknown-biology-we-might-not-know-half-of-whats-in-our-cells/

John Polkinghorne's "Belief in God in an Age of Science"

A book I have been enjoying recently is John Polkinghorne's "Belief in God in an Age of Science." Polkinghorne who died recently was a noted theoretical physicist who was also a theologian. Unlike Polkinghorne I am an atheist, but he makes a good case for why religion, science, poetry, art, literature should all be welcomed as sources for truth about the universe and about human beings. A quote I particularly like from it:

"If we are seeking to serve the God of truth then we should really welcome truth from whatever source it comes. We shouldn’t fear the truth. Some of it will be from science, obviously, but by no means all of it. It will sometimes be perplexing, how this bit of truth relates to that bit of truth; we know that within science itself often enough and we find it outside of science as well. The crucial thing is to be honest.”

I would quibble with the catch-all definition of truth in Polkinghorne's quote (scientific "truth" by its very nature is tentative) but otherwise agree. In my scientific career I have found this as well. Often Tolstoy or the Bhagavad Gita or Bach have taught me deep truths about human beings that I never saw in any physics or chemistry or mathematics textbook. The great thing about human life is its diversity. Science is the most important thing that enriches it, but it's not the only one. That's a good thing. These multiple sources of diversity should keep us busy for as long as there is a human species.

On change

Two weeks ago, outside a coffee shop near Los Angeles, I discovered a beautiful creature, a moth. It was lying still on the pavement and I was afraid someone might trample on it, so I gently picked it up and carried it to a clump of garden plants on the side. Before that I showed it to my 2-year-old daughter who let it walk slowly over her arm. The moth was brown and huge, almost about the size of my hand. It had the feathery antennae typical of a moth and two black eyes on the ends of its wings. It moved slowly and gradually disappeared into the protective shadow of the plants when I put it down.

Later I looked up the species on the Internet and found that it was a male Ceanothus silk moth, very prevalent in the Western United States. I found out that the reason it’s not seen very often is because the males live only for about a week or two after they take flight. During that time they don’t eat; their only purpose is to mate and die. When I read about it I realized that I had held in my hand a thing of indescribable beauty, indescribable precisely because of the briefness of its life. Then I realized that our lives are perhaps not all that long compared to the Ceanothus moth’s. Assuming that an average human lives for about 80 years, the moth’s lifespan is about 2000 times shorter than ours. But our lifespans are much shorter than those of redwood trees. Might not we appear the same way to redwood trees the way Ceanoth moths or ants appear to us, brief specks of life fluttering for an instant and then disappearing? The difference, as far as we know, is that unlike redwood trees we can consciously understand this impermanence. Our lives are no less beautiful because on a relative scale of events they are no less brief. They are brief instants between the lives of redwood trees just like redwood trees’ lives are brief instants in the intervals between the lives of stars.

I have been thinking about change recently, perhaps because it’s the standard thing to do for someone in their forties. But as a chemist I have thought about change a great deal in my career. The gist of a chemist’s work deals with the structure of molecules and their transformations into each other. The molecules can be natural or synthetic. They can be as varied as DNA, nylon, chlorophyll, rocket fuel, cement and aspirin. But what connects all of them is change. At some point in time they did not exist and came about through the union of atoms of carbon, oxygen, hydrogen, phosphorus and other elements. At some point they will cease to be and those atoms will become part of some other molecule or some other life form.

Sometimes popular culture can capture the essence of science and philosophy well. In this case, chemistry as change was captured eloquently by the character of Walter White in the TV show “Breaking Bad”. In his first lecture as a high school chemistry teacher White says,

“Chemistry is the study of matter. But I prefer to think of it as the study of change. Now, just think about this. Electrons change their energy levels. Elements, they change and combine into compounds. Well, that’s…that’s all of life, right? It’s the constant, it’s the cycle, it’s solution, dissolution, just over and over and over. It’s growth, then decay, then transformation. It is fascinating, really.”

Changes in the structure of atoms and molecules are ultimately dictated by the laws of atomic physics and the laws of thermodynamics. The second law of thermodynamics which loosely states that disorder is more likely than order guarantees that change will occur. At its root the second law is an argument from probability: there are simply many more ways for a system to be disordered than to be ordered. The miracle of life and the universe at large is that complex systems like biological systems can briefly defy the second law, assembling order from disorder, letting it persist for a few short decades during which that order can do astonishing things like make music and art and solve mathematical equations enabling it to understand where it came from. The biologist Carl Woese once gave an enduringly beautiful metaphor for life, comparing it to a child playing in a stream.

“If not machines, what are organisms? A metaphor far more to my liking is this. Imagine a child playing in a woodland stream, poking a stick into an eddy in the flowing current, thereby disrupting it. But the eddy quickly reforms. The child disperses it again. Again it reforms, and the fascinating game goes on. There you have it! Organisms are resilient patterns in a turbulent flow—patterns in an energy flow.”

Woese’s metaphor perfectly captures both the permanence and impermanence of life. The structure is interrupted, but over time its essence persists. It changes and yet stays the same.

Although thermodynamics and Darwin’s theory of evolution help us understand how ordered structures can perform these complex actions, ultimately we don’t really understand it at the deepest level. The best illustration of our ignorance is the most complex structure in the universe – the human brain. The brain is composed of exactly the same elements as my table, my cup of coffee and the fern plant growing outside my window. Yet the same elements, when assembled together to create a fern, somehow when assembled in another, very specific way, create a 3-pound, jellylike structure that can seemingly perform miracles like writing ‘Hamlet’, finding the equations of spacetime curvature and composing the Choral Symphony. We have loose terminology like ’emergence’ to describe the unique property of consciousness that arises when human brains are assembled together from inanimate elements, but if we were to be honest as scientists, we must admit that we don’t understand how exactly that happens. The ultimate example of change that makes the essence of us as humans possible is still an enduring mystery. Will we ever solve that mystery? Even some of the smartest scientists on the planet, like the theoretical physicist Edward Witten, think we may not. As Witten puts it,

“I think consciousness will remain a mystery. Yes, that’s what I tend to believe. I tend to think that the workings of the conscious brain will be elucidated to a large extent. Biologists and perhaps physicists will understand much better how the brain works. But why something that we call consciousness goes with those workings, I think that will remain mysterious. I have a much easier time imagining how we understand the Big Bang than I have imagining how we can understand consciousness…”

In other words, what Witten is saying is that even if someday we may understand the how and the what of consciousness, we may never understand the why. One of the biggest examples of change in the history of the universe may well remain hidden behind a veil.

I think about change a lot not just because I am a chemist but because I am a parent. Sometimes it feels like our daughter who is now two and a half years old has changed more in that short time than a caterpillar changes into a butterfly. Her language, reasoning, social and motor skills have undergone an amazing change since she was born. And this is, of course, a change that is observed by every parent: children change an incredible amount during their first few years. Some of that change can be guided by parents, but other change is genetic as well as idiosyncratic and unpredictable. Just like you can coax simple arrangements of atoms into certain compounds but not others, as a parent you have to make peace with the fact that you will be able to mold your child’s temperament, personality and trajectory in life to a certain extent but not beyond that. As the old alchemists figured out, you cannot change mercury into gold or gold into mercury no matter how hard you try. And that’s ultimately for the better because, just like the diversity of elements, we then get a diversity of novel and surprising life trajectories for our children.

Children undergo change but they are are also often the best instruments for causing it. Recently I finished reading Octavia Butler’s remarkable “Parable of the Sower” which is set in a 2024 California that is racked by violence and arson by desperate, homeless people who break into gated communities and burn, murder and rape. The protagonist of the story is a clear-eyed, determined 18-year-old named Lauren Olamina who, after her family is murdered, starts out by herself with the goal of starting a new religion called Earthseed amidst the madness surrounding her. Earthseed sees God as a changeable being and embraces change as the essence of living. Lauren thinks that in a world where people have to deal with unpredictable, seismic, sometimes violent change, a religion that makes the very nature of change a blueprint for God’s work can not just survive but thrive. For an atheist like myself, Earthseed seems as good a religion as any for us to believe in if we want to thrive in an uncertain world. Butler’s story tells us that just like they always have, our children exist to fix the problems our generation has created.

Change permeates the largest scales of the universe as much as it does ourselves, our children and our bodies and brains. One of the most philosophically shattering experiences in the development of science was the realization by Galileo, Brahe, Newton and others that the perfect, crystalline, quiet universe of Aristotle and other ancients was in fact a dynamic, violent universe. In the mid 20th century, astrophysicists worked out that stars go through a life sequence much like we do. When they are born they furiously burn hydrogen into helium and form the lighter elements. As they age they can go in one of several directions. Stars the size of the sun will first blow up into red giants and then quietly settle into the life of a white dwarf. But stars much more massive than the sun can turn into supernovae and black holes, ending their lives in a cosmic show of spectacular explosion or fiery gravitational contraction.

When our sun turns into a red giant, about 6 billion years from now, its outer shell will expand and embrace the orbits of Mercury, Venus and Earth. There is no reason to believe that those planets will survive that encounter. By that time the human race would either be extinct or would have migrated to other star systems; the worst thing that it could do would be to stay put. Even after that we will not escape change. The science of eschatology, the study of the ultimate fate of the universe, has mapped out many changes that will be unstoppable in the far future. At some point the Andromeda galaxy will collide with our Milky Way galaxy. Eventually the stars in the universe will run out of fuel and cease to shine; the universe will become a quieter and darker place. Soon it will only contain black holes and at a further point even black holes will evaporate through the process of Hawking radiation. And way beyond that, the laws of quantum mechanics will ensure that the proton, usually considered a stable particle, will decay. Matter as we know it will dissolve into nothingness. The accelerated expansion of our universe will ensure that most of these processes will inevitably take place. The exact fate of the universe is too uncertain to predict beyond these unimaginable gulfs of time, but there is little doubt that the universe will be profoundly different from what it is now and what it has been before.

The elements from which my body and brain are composed will one day be given back to the universe (I like to think that they will perhaps become part of a redwood tree). That fact does not fill me with a feeling of dread or sadness but instead feels me with peace, joy and gratitude. The ultimate death of the universe described above causes similar feelings to arise. Sometimes I like to sit back, close my eyes and imagine a peaceful, lifeless universe, the galaxies receding past the cosmic horizons, the occasional supernova going off. The carbon, oxygen, nitrogen and other heavier elements in my body came from such supernova explosions a long time ago; the hydrogen came from the Big Bang. Those are astounding facts that science has discovered in the last few decades. Of all the things that could have happened to those elements forged in the furnace of a far off supernova, what were the chances that they would assemble into the exact specific arrangements that would be me? While we understand now how that happens, it could well have gone countless other ways. I feel privileged to exist as part of that brief interval between supernova explosions, to be able to understand, in my own modest way, the workings of our universe. To be a tiny part of the change that makes the universe what it is.

A Science Thanksgiving

It’s Thanksgiving weekend here in the U.S., and there’s an informal tradition on Thanksgiving to give thanks for all kinds of things in our lives. Certainly there’s plenty to be thankful for this year, especially for those of us whose lives and livelihoods haven’t been personally devastated by the coronavirus pandemic. But I thought I would do something different this year. Instead of being thankful for life’s usual blessings, how about being thankful for some specific facts of nature and the universe that are responsible for our very existence and make it wondrous? Being employed and healthy and surrounded by family and friends is excellent, but none of that would be possible without the amazing unity and diversity of life and the universe. So without further ado and in no particular order, I present an entirely personal selection of ten favorites for which I am eternally thankful.

I am thankful for the value of the resonance level energy of the excited state of carbon-12: carbon-12 which is the basis of all organic life on earth is formed in stars through the reaction of beryllium-8 with helium-4. The difference in energies between the starting materials (beryllium + helium) and carbon is only about 4%. If this difference had been even slightly higher, the unstable beryllium-8 would have disappeared long before it had transmuted into carbon-12, making life impossible.

I am thankful for the phenomenon of horizontal gene transfer (HGT): it allowed bacteria during early evolution to jump over evolutionary barriers by sharing genetic material between themselves instead of just with their progeny. The importance of HGT for evolution may be immense since regular HGT early on might have led to the universality of the genetic code. HGT mixed and matched genetic material in the cauldron of life, eventually leading to the evolution of multicellular organisms including human beings.

I am thankful for the pistol shrimp: an amazing creature that can “clap” its pincers and send out a high-pressure bubble with lightning speed to kill its prey. This sonication bubble can produce light when it collapses, and the speed of collapse is such that temperature inside the bubble can briefly approach the surface temperature of the sun. The pistol shrimp shows us that nature hides phenomena that are not dreamt of in our philosophy, leading to an inexhaustible list of natural wonders for us to explore.

I am thankful to the electron: an entire universe within a point particle that performs the subtlest and most profound magic, making possible the chemistry of life; giving rise to the electromagnetic force that holds ordinary matter together; ultimately creating minds that can win prizes for studying electrons.

I am thankful to the cockroach: may humanity have the resilience to survive the long nights of our making the way you have.

I am thankful to the redwoods: majestic observers and guardians of nature who were here before us, who through their long, slow, considered lives have watched us live out our frantic, anxious lives the way we watch ants live out theirs, and whose survival is now consequentially entwined with our own.

I am thankful to the acetyl group, a simple geometric arrangement of two carbon and one oxygen atoms whose diverse, myriad forms fueling life and alleviating pain – acetylcholine, acetyl-coenzyme A, acetaminophen – are tribute to the ingenuity of both human minds and nature.

I am thankful to i, the square root of minus one: who knew that this diabolical creature, initially alien to even the abstract perception of mathematicians, would be as “real” as real numbers and more importantly, underlie the foundation of our most hallowed descriptions of nature such as quantum theory.

I am thankful to the black hole: an endless laboratory of the most bizarre and fantastic wonders; trapping light but letting information escape; providing the ultimate playground for spacetime curvature; working relentlessly over billions of years as a clearinghouse and organizing principle for the universe’s wayward children; proving that the freaks of the cosmos are in fact the soul food of its very existence.

I am thankful for time: that elusive entity which, in the physicist John Wheeler’s words, “keeps everything from happening all at once”; which waits for no one and grinds kings and paupers into the same ethereal dust; whose passage magically changes children every day before our very eyes; whose very fleeting nature makes life precious and gives us the most to be thankful for.

Should a scientist have "faith"?

Scientists like to think that they are objective and unbiased, driven by hard facts and evidence-based inquiry. They are proud of saying that they only go wherever the evidence leads them. So it might come as a surprise to realize that not only are scientists as biased as non-scientists, but that they are often driven as much by belief as are non-scientists. In fact they are driven by more than belief: they are driven by faith. Science. Belief. Faith. Seeing these words in a sentence alone might make most scientists bristle and want to throw something at the wall or at the writer of this piece. Surely you aren’t painting us with the same brush that you might those who profess religious faith, they might say?

But there’s a method to the madness here. First consider what faith is typically defined as – it is belief in the absence of evidence. Now consider what science is in its purest form. It is a leap into the unknown, an extrapolation of what is into what can be. Breakthroughs in science by definition happen “on the edge” of the known. Now what sits on this edge? Not the kind of hard evidence that is so incontrovertible as to dispel any and all questions. On the edge of the known, the data is always wanting, the evidence always lacking, even if not absent. On the edge of the known you have wisps of signal in a sea of noise, tantalizing hints of what may be, with never enough statistical significance to nail down a theory or idea. At the very least, the transition from “no evidence” to “evidence” lies on a continuum. In the absence of good evidence, what does a scientist do? He or she believes. He or she has faith that things will work out. Some call it a sixth sense. Some call it intuition. But “faith” fits the bill equally.

If this reliance on faith seems like heresy, perhaps it’s reassuring to know that such heresies were committed by many of the greatest scientists of all time. All major discoveries, when they are made, at first rely on small pieces of data that are loosely held. A good example comes from the development of theories of atomic structure.

When Johannes Balmer came up with his formula for explaining the spectral lines of hydrogen, he based his equation on only four lines that were measured with accuracy by Anders Ångström. He then took a leap of faith and came up with a simple numerical formula that predicted many other lines emanating from the hydrogen atom and not just four. But the greatest leap of faith based on Balmer’s formula was taken by Niels Bohr. In fact Bohr did not even hesitate to call it anything but a leap of faith. In his case, the leap of faith involved assuming that electrons in atoms only occupy certain discrete energy states, and that figuring out the transitions between these states somehow involved Planck’s constant in an important way. When Bohr could reproduce Balmer’s formula based on this great insight, he knew he was on the right track, and physics would never be the same. One leap of faith built on another.

To a 21st century scientist, Bohr’s and Balmer’s thinking as well as that of many other major scientists well through the 20th century indicates a manifestly odd feature in addition to leaps of faith – an absence of what we call statistical significance or validation. As noted above, Balmer used only four data points to come up with his formula, and Bohr not too many more. Yet both were spectacularly right. Isn’t it odd, from the standpoint of an age that holds statistical validation sacrosanct, to have these great scientists make their leaps of faith based on paltry evidence, “small data” if you will? But that in fact is the whole point about scientific belief, that it originates precisely when there isn’t copious evidence to nail the fact, when you are still on shaky ground and working at the fringe. But this belief also supremely echoes a famous quote by Bohr’s mentor Rutherford – “If your experiment needs statistics, you ought to have done a better experiment.” Resounding words from the greatest experimental physicist of the 20th century whose own experiments were so carefully chosen that he could deduce from them extraordinary truths about the structure of matter based on a few good data points.

The transition between belief and fact in science in fact lies on a continuum. There are very few cases where a scientist goes overnight from a state of “belief” to one of “knowledge”. In reality, as evidence builds up, the scientist becomes more and more confident until there are not enough grounds for believing otherwise. In many cases the scientist may not even be alive to see his or her theory confirmed in all its glory: even the Newtonian model of the solar system took until the middle of the 19th century to be fully validated, more than a hundred years after Newton’s death.

A good example of this gradual transition of a scientific theory from belief to confident espousal is provided by the way Charles Darwin’s theory of evolution by natural selection, well, evolved. It’s worth remembering that Darwin took more than twenty years to build up his theory after coming home from his voyage on the HMS Beagle in 1836. At first he only had hints of an idea based on extensive and yet uncatalogued and disconnected observations of flora and fauna from around the world. Some of the evidence he had documented – the names of Galapagos finches, for instance – was wrong and had to be corrected by his friends and associates. It was only by arduous experimentation and cataloging that Darwin – a famously cautious man – was able to reach the kind of certainty that prompted him to finally publish his magnum opus, Origin of Species, in 1859, and even then only after he was threatened to be scooped by Alfred Russell Wallace. There can be said to be no one fixed eureka moment when Darwin could say that he had transitioned from “believing” in evolution by natural selection to “knowing” that evolution by natural selection was true. And yet, by 1859, this most meticulous scientist was clearly confident enough in his theory that he no longer simply believed in it. But it certainly started out that way. The same uncertain transition between belief and knowledge applies to other discoveries. Einstein often talked about his faith in his general theory of relativity before observations of the solar eclipse of 1919 confirmed its major prediction, the bending of starlight by gravity, remarking that if he was wrong it would mean that the good lord had led him down the wrong garden path. When did Watson and Crick go from believing that DNA is a double helix to knowing that it is? When did Alfred Wegener go from believing in plate tectonics to knowing that it was real? In some sense the question is pointless. Scientific knowledge, both individually and collectively, gets cemented with greater confidence over time until the objections simply cannot stand up to the weight of the accumulated evidence.

Faith, at least in one important sense, is thus an important part of the mindset of a scientist. So why should scientists not nod in assent if someone then tells them that there is no difference, at least in principle, between their faith and religious faith? For two important reasons. Firstly, the “belief” that a scientist has is still based on physical and not supernatural evidence, even if all the evidence may not yet be there. What scientists call faith is still based on data and experiments, not mystic visions and pronouncements from a holy book. More importantly, unlike religious belief, scientific belief can wax and wane with the evidence; it importantly is tentative and always subject to change. Any good scientist who believes X will be ready to let go of their belief in X if strong evidence to the contrary presents itself. That is in fact the main difference between scientists on one hand and clergymen and politicians on the other; as Carl Sagan once asked, when was the last time you heard either of the latter say, “You know, that’s a really good counterargument. Maybe what I am saying is not true after all.”

Faith may also interestingly underlie one of the classic features of great science – serendipity. Unlike what we often believe, serendipity does not always refer to pure unplanned accident but to deliberately planned accident; as Alexander Fleming memorably put it, chance favors the “prepared mind”. A remarkable example of deliberate serendipity comes from an anecdote about his discovery of slow neutrons that Enrico Fermi narrated to Subrahmanyan Chandrasekhar. Slow neutrons unlocked the door to nuclear power and the atomic age. Fermi told Chandrasekhar how he came to make this discovery which he personally considered – among a dozen seminal ones – to be his most important one (From Mehra and Rechenberg, “The Historical Development of Quantum Theory, Vol. 6”):

Chandrasekhar’s invocation of Hadamard’s thesis of unconscious discovery might provide a rational underpinning for what we are calling faith. In this case, Fermi’s great intuitive jump, his seemingly irrational faith that paraffin might slow down neutrons, might have been grounded in the extensive body of knowledge about physics that was housed in his brain, forming connections that he wasn’t even aware of. Not every leap of faith can be explained this way, but some can. In this sense a scientist’s faith, unlike religious faith, is very much rational and based on known facts.

Ultimately there’s a supremely important guiding role that faith plays in science. Scientists ignore believing at their own peril. This is because they have to constantly tread the tightrope of skepticism and wonder. Shut off your belief valve completely and you will never believe anything until there is five-sigma statistical significance for it. Promising avenues of inquiry that are nonetheless on shaky grounds for the moment will be dismissed by you. You may never be the first explorer into rich new scientific territory. But open the belief valve completely and you will have the opposite problem. You may believe anything based on the flimsiest of evidence, opening the door to crackpots and charlatans of all kinds. So where do you draw the line?

In my mind there are a few logical rules of thumb that might help a scientist to mark out territories of non-belief from ones where leaps of faith might be warranted. In my mind, plausibility based on the known laws of science should play a big role. For instance, belief in homeopathy would be mistaken based on the most elementary principles of physics and chemistry, including the laws of mass action and dose response. But what about belief in extraterrestrial intelligence? There the situation is different. Based on our understanding of the laws of quantum theory, stellar evolution and biological evolution, there is no reason to believe that life could not have arisen on another planet somewhere in the universe. In this sense, belief in extraterrestrial intelligence is justified belief, even if we don’t have a single example of life existing anywhere else. We should keep on looking. Faith in science is also more justified when there is a scientific crisis. In a crisis you are on desperate grounds anyway, so postulating ideas that aren’t entirely based on good evidence isn’t going to make matters worse and are more likely to lead into novel territory. Planck’s desperate assumption that energy only comes in discrete packets was partly an act of faith that resolved a crisis in classical physics.

Ultimately, though, drawing a firm line is always hard, especially for topics on the fuzzy boundary. Extra-sensory perception, the deep hot biosphere and a viral cause for mad cow disease are three theories which are implausible although not impossible in principle; there is little in them that flies against the basic laws of science. The scientists who believe in these theories are sticking their necks out and taking a stand. They are heretics who are taking the risk of being called fools; since most bold new ideas in science are usually wrong, they often will be. But they are setting an august precedent.

If science is defined as the quest into the unknown, a foray into the fundamentally new and untested, it is more important than ever especially in this age of conformity, for belief in science to play a more central role in the practice of science. The biggest scientists in history have always been ones who took leaps of faith, whether it was Bohr with his quantum atom, Einstein with his thought experiments or Noether with her deep feeling for the relationship between symmetry and conservation laws, a feeling felt but not seen. For creating minds like these, we need to nurture an environment that not just allows but actively encourages scientists, especially young ones, to tread the boundary between evidence and speculation with aplomb, to exercise their rational faith with abandon. Marie Curie once said, “Now is the time to fear less, so that we may understand more.” To which I may add, “Now is the time to believe more, so that we may understand even more.”

First published on 3 Quarks Daily

Brains, Computation And Thermodynamics: A View From The Future?

Progress in science often happens when two or more fields productively meet. Astrophysics got a huge boost when the tools of radio and radar met the age-old science of astronomy. From this fruitful marriage came things like the discovery of the radiation from the big bang. Another example was the union of biology with chemistry and quantum mechanics that gave rise to molecular biology. There is little doubt that some of the most important future discoveries in science in the future will similarly arise from the accidental fusion of multiple disciplines.

One such fusion sits on the horizon, largely underappreciated and unseen by the public. It is the fusion between physics, computer science and biology. More specifically, this fusion will likely see its greatest manifestation in the interplay between information theory, thermodynamics and neuroscience. My prediction is that this fusion will be every bit as important as any potential fusion of general relativity with quantum theory, and at least as important as the development of molecular biology in the mid 20th century. I also believe that this development will likely happen during my own lifetime.

The roots of this predicted marriage go back to 1867. In that year the great Scottish physicist James Clerk Maxwell proposed a thought experiment that was later called ‘Maxwell’s Demon’. Maxwell’s Demon was purportedly a way to defy the second law of thermodynamics that had been proposed a few years earlier. The second law of thermodynamics is one of the fundamental laws governing everything in the universe, from the birth of stars to the birth of babies. It basically states that left to itself, an isolated system will tend to go from a state of order to one of disorder. A good example is how a bottle of perfume wafts throughout a room with time. This order and disorder was quantified by a quantity called entropy.

In technical terms, the order and disorder refers to the number of states a system can exist in; order means fewer states and disorder means more. The second law states that isolated systems will always go from fewer states and lower entropy (order) to more states and higher entropy (disorder). Ludwig Boltzmann quantified this relationship with a simple equation carved on his tombstone in Vienna: S = klnW, where k is a constant called the Boltzmann constant, ln is the natural logarithm (to the base e) and W is the number of states.

Maxwell’s Demon was a mischievous creature which sat on top of a box with a partition in the middle. The box contains molecules of a gas which are ricocheting in every direction. Maxwell himself had found that these molecules’ velocities follow a particular distribution of fast and slow. The demon observes these velocities, and whenever there is a molecule moving faster than usual in the right side of the box, he opens the partition and lets it into the left side, quickly closing the partition. Similarly he lets in slower moving molecules from left to right. After some time, all the slow molecules will be in the right side and the fast ones will in the left. Now, velocity is related to temperature, so this means that one side of the box has heated up and the other has cooled down. To put it another way, the box went from a state of random disorder to order. According to the second law this means that the entropy of the system of the system decreased, which is impossible.

Maxwell’s demon seemingly contravenes the second law of thermodynamics (University of Pittsburgh)

For the next few years scientists tried to get around Maxwell’s Demon’s paradox, but it was in 1922 that the Hungarian physicist Leo Szilard made a dent in it when he was a graduate student hobnobbing with Einstein, Planck and other physicists in Berlin. Szilard realized an obvious truth that many others seem to have missed. The work and decision-making that the demon does to determine the velocities of the molecules itself generates entropy. If one takes this work into account, it turns out that the total entropy of the system has indeed increased. The second law is safe. Szilard later went on to a distinguished career as a nuclear physicist, patenting a refrigerator with Einstein and becoming the first person to think of a chain reaction.

Perhaps unknowingly, however, Szilard had also discovered a connection – a fusion of two fields – that was going to revolutionize both science and technology. When the demon does work to determine the velocities of molecules, the entropy that he creates comes not just from the raising and lowering of the partition but from his thinking processes, and these processes involve information processing. Szilard had discovered a crucial and tantalizing link between entropy and information. Two decades later, mathematician Claude Shannon was working at Bell Labs, trying to improve the communication of signals through wires. This was unsurprisingly an important problem for a telephone and communications company. The problem was that when engineers were trying to send a message over a wire, it would lose its quality because of many factors including noise. One of Shannon’s jobs was to figure out how to make this transmission more efficient.

Shannon found out that there is a quantity that relates to the information transmitted over the wire. In crude terms, this quantity was inversely related to the information as well as to the probability of transmitting that information; the higher the probability of transmitting accurate information over a channel, the lower this quantity was and vice versa. When Shannon showed his result to the famous mathematician John von Neumann, von Neumann with his well-known lightning-fast ability to connect disparate ideas, immediately saw what it was: “You should call your function ‘entropy’”, he said, “firstly because that is what it looks like in thermodynamics, and secondly because nobody really knows what entropy is, so in a debate you will always have the upper hand.” Thus was born the connection between information and entropy. Another fortuitous connection was born – between information, entropy and error or uncertainty. The greater the uncertainty in transmitting a message, the greater the entropy, so entropy also provided a way to quantify error. Shannon’s 1948 paper, “A Mathematical Theory of Communication”, was a seminal publication and has been called the Magna Carta of the information age.

Even before Shannon, another pioneer had published a paper that laid the foundations of the theory of computing. In 1936 Alan Turing published “On Computable Numbers, with an Application to the Entscheidungsproblem”. This paper introduced the concept of Turing machines which also process information. But neither Turing nor von Neumann really made the connection between computation, entropy and information explicit. Making it explicit would take another few decades. But during those decades, another fascinating connection between thermodynamics and information would be discovered.

Stephen Hawking’s tombstone at Westminster Abbey (Cambridge News)

That connection came from Stephen Hawking getting annoyed. Hawking was one of the pioneers of black holes, and along with Roger Penrose he had discovered that at the center of every black hole is a singularity that warps spacetime infinitely. The boundary of the black hole is its event horizon and within that boundary not even light can escape. But black holes posed some fundamental problems for thermodynamics. Every object contains entropy, so when an object disappears into a black hole, where does its entropy go? If the entropy of the black hole does not increase then the second law of thermodynamics would be violated. Hawking had proven that the area of a black hole’s event horizon never decreases, but he had pushed the thermodynamic question under the rug. In 1972 at a physics summer school, Hawking met a graduate student from Princeton named Jacob Bekenstein who proposed that the increasing area of the black hole was basically a proxy for its increasing entropy. This annoyed Hawking and he did not believe it because increased entropy is related to heat (heat is the highest- entropy form of energy) and black holes, being black, could not radiate heat. With two colleagues Hawking set out to prove Bekenstein wrong. In the process, he not only proved him right but also made what is considered his greatest breakthrough: he gave black holes a temperature. Hawking found out that black holes do emit thermal radiation. This radiation can be explained when you take quantum mechanics into account. The Hawking-Bekenstein discovery was a spectacular example of another fusion: between information, thermodynamics, quantum mechanics and general relativity. Hawking deemed it so important that he wanted to put it on his tombstone in Westminster Abbey, and so it has been.

This short digression was to show that more links between information, thermodynamics and other disciplines were being forged in the 1960s and 70s. But nobody saw the connections between computation and thermodynamics until Rolf Landauer and Charles Bennett came along. Bennett and Landauer were both working at IBM. Landauer was an émigré who fled from Nazi Germany before working for the US Navy as an electrician’s mate, getting his PhD at Harvard and joining IBM. IBM was then a pioneer of computing; among other things they had built computers for the Manhattan Project. In 1961, Landauer published a paper titled “Irreversibility and Heat Generation in the Computing Process” that is destined to become a classic of science. In it, Landauer established that the basic act of computation – the change of one bit to another, say a 1 to a 0 – requires a bare minimum amount of entropy. He quantified this amount with another simple equation: S = kln2, with k again being the Boltzmann constant and ln the natural logarithm. This has become known as the Landauer bound; it is the absolute minimum amount of entropy that has to be expended in a single bit operation. Landauer died in 1999 and as far as I know the equation is not carved on his tombstone.

The Landauer bound applies to all kinds of computation in principle and biological processes are also a form of information processing and computation, so it’s tantalizing to ask whether Landauer’s calculation applies to them. Enter Charles Bennett. Bennett is one of the most famous scientists whose name you may not have heard of. He is not only one of the fathers of quantum computing and quantum cryptography but he is also one of the two fathers of the marriage of thermodynamics with computation, Landauer being the other. Working with Landauer in the 1970s and 80s, Bennett applied thermodynamics to both Turing machines and biology. By good fortune he had gotten his PhD in physical chemistry studying the motion of molecules, so his background primed him to apply ideas from computation to biology.

Charles Bennett from IBM has revolutionized our understanding of the thermodynamics of computation (AMSS)

To simplify matters, Bennett considered what he called a Brownian Turing machine. Brownian motion is the random motion of atoms and molecules. A Brownian Turing machine can write and erase characters on a tape using energy extracted from a random environment. This makes the Brownian Turing machine reversible. A reversible process might seem strange, but in fact it’s found in biology all the time. Enzyme reactions occur from the reversible motion of chemicals – at equilibrium there is equal probability that an enzymatic reaction will go forward or backward. What makes these processes irreversible is the addition of starting materials or the elimination of chemical products. Even in computation, only a process which erases bits is truly irreversible because you lose information. Bennett envisaged a biological process like protein translation as a Brownian Turing machine which adds or subtracts a molecule like an amino acid, and he calculated the energy and entropy expenditures involved in running this machine. Visualizing translation as a Turing machine made it easier to do a head-to-head comparison between biological processes and bit operations. Bennett found out that if the process is reversible the Landauer bound does not hold and there is no minimum entropy required. Real life of course is irreversible, so how do real-life processes compare to the Landauer bound?

In 2017, a group of researchers published a fascinating paper in the Philosophical Transactions of the Royal Society in which they explicitly calculated the thermodynamic efficiency of biological processes. Remarkably, they found that the efficiency of protein translation is several orders of magnitude better than the best supercomputers, in some cases as better as a million fold. More remarkably, they found that the efficiency is only one order of magnitude worse than the theoretical minimum Landauer bound. In other words, evolution has done one hell of a job in optimizing the thermodynamic efficiency of biological processes.

But not all biological processes. Circling back to the thinking processes of Maxwell’s little demon, how does this efficiency compare to the efficiency of the human brain? Surprisingly, it turns out that neural processes like the firing of synapses are estimated to be much worse than protein translation and more comparable to the efficiency of supercomputers. At first glance, the human brain thus appears to be worse than other biological processes. However, this seemingly low computational efficiency of the brain must be compared to its complex structure and function. The brain weighs only about a fiftieth of the weight of an average human but it uses up 20% of the body’s energy. It might seem that we are simply not getting the biggest bang for our buck, with an energy-hungry brain providing low computational efficiency. What would explain this inefficiency and this paradox?

My guess is that the brain has been designed to be inefficient through a combination of evolutionary accident and design and that efficiency is the wrong metric for gauging the performance of the brain. Efficiency is the wrong metric because thinking of the brain in digital terms is the wrong metric. The brain arose through a series of modular inventions responding to new environments created by both biology and culture. We now know that thriving in these environments needed a combination of analog and digital functions.; for instance, the nerve impulses controlling blood pressure are digital while the actual change in pressure is continuous and analog. It is likely that digital neuronal firing is built on an analog substrate of wet matter, and that higher-order analog functions could be emergent forms of digital neuronal firing. As early as the 1950s, von Neumann conjectured that we would need to model the brain as both analog and digital in order to understand it. Around the time that Bennett was working out the thermodynamics of computation, two mathematicians named Marian Pour-El and Ian Richards proved a very interesting theorem which showed that in certain cases, there are numbers that are not computable with digital computers but are computable with analog processes; analog computers are thus more powerful in such cases.

If our brains are a combination of digital and analog, it’s very likely that they are this way so that they can span a much bigger range of computation. But this bigger range would come at the expense of inefficiency in the analog computation process. The small price of lower computational efficiency as measured by the Landauer bound would come at the expense of the much greater evolutionary benefits of performing complex calculations that allow us to farm, build cities, know stranger from kin and develop technology. Essentially, the Landauer bound could be evidence for the analog nature of our brains. There is another interesting fact about analog computation, which is its greater error rate; digital computers took off precisely because they had low error rates. How does the brain function so well in spite of this relatively high error rate? Is the brain consolidating this error when we dream? And can we reduce this error rate by improving the brain’s efficiency? Would that make our brains better or worse at grasping the world?

From the origins of thermodynamics and Maxwell’s Demon to the fusion of thermodynamics with information processing, black holes, computation and biology, we have come a long way. The fusion of thermodynamics and computation with neuroscience just seems to be beginning, so for a young person starting out in the field the possibilities are exciting and limitless. A multitude of general questions abound: How does the efficiency of the brain relate to its computational abilities? What might be the evolutionary origins of such abilities? What analogies between the processing of information in our memories and that in computers might we discover through this analysis? And finally, just like Shannon did for information, Hawking and Bekenstein did for black holes and Landauer and Bennett did for computation and biology, can we find out a simple equation describing how the entropy of thought processes relates to simple neural parameters connected to memory, thinking, empathy and emotion? I do not know the answers to these questions, but I am hoping someone who is reading this will, and at the very least they will then be able to immortalize themselves by putting another simple formula describing the secrets of the universe on their tombstone.

Further reading:

1. Charles Bennett – The Thermodynamics of Computation
2. Seth Lloyd – Ultimate Physical Limits to Computation
3. Freeman Dyson – Are brains analog or digital?
4. George Dyson – Analogia: The Emergence of Technology Beyond Programmable Control (August 2020)
5. Richard Feynman – The Feynman Lectures on Computation (Chapter 5)
6. John von Neumann – The General and Logical Theory of Automata

First published on 3 Quarks Daily