Can we turn biology into engineering?

Vijay Pande of Andreessen-Horowitz/Stanford has a thought-provoking piece in Scientific American in which he lays out a game plan for how we could potentially make biology more like engineering. He takes issue with what Derek (Lowe) once called the “Andy Grove fallacy”, which in a nutshell says that you can make fields like biotechnology and drug discovery as efficient as semiconductor or automobile engineering if you borrow principles from engineering.

There are parts of the piece that I resoundingly agree with; for instance, there’s little doubt that fields like automation and AI are going to have a significant impact on making biological experiments more reproducible, many of which are still more art than science and subject to the whims and sloppiness of their creators and their lab notebooks. Vijay is also optimistic about making biology more modular, so that one can string along parts of molecules, cells and organelles to enable better biological engineering of body parts, drugs and genetic systems. He also believes that bringing more quantitative measurements encoded into key performance indicators (KPI) will make the discipline more efficient and more mindful of its successes and failures. One point which I think is very important is that these kinds of approaches would allow us to gather more negative data, a data collection problem that still hobbles AI and machine learning approaches.

So far I am with him; I don’t believe that biology can’t ever benefit from such approaches, and it’s certainly true that the applications of AI, automation and other engineering-based approaches are only going to increase with time. But the article doesn’t mention some very fundamental differences between biology and engineering which I think demarcate the two substantially from each other and which make knowledge transfer between them highly problematic.

Foremost among these are non-linearity, redundancy and emergence.

Let’s take two examples which the piece talks about that illustrate all three concepts – building bridges and the Apollo project. Comparisons with the latter always make me wince a bit. Vijay is quite right that the right approach to the Apollo program was to break the problems into parts, then further break those parts up into individual steps such as building small models. The scientists and engineers working on the program gradually built up layers of complexity until the model that they tested was the moon landing itself.

Now, the fact is that we already do this in biology. For instance, when we want to understand or treat a disease, we try to break it down to simpler levels – organelles, cells, proteins, genes – and then try to understand and modulate each of these entities. We use animal models like genetically engineered mice and dogs as simpler representatives of complex human biology. But firstly - and as we keep on finding out - these models are pale shadows of true human biology; we use them because we can't do better. And secondly, even these ‘simple’ models are much more complex than we think. The reasons are non-linearity and emergence, both of which can thwart modular approaches. The sum of proteins in a cell is not the same as the cell phenotype itself, just like the sum of neurons in a human brain is not the brain itself. So modulating a protein for instance can cause complex downstream effects that depend on both the strength and nature of the modulating signal. In addition, biological pathways are redundant, so modulating one can cause another one to take over, or for the pathway to switch between complex networks. Many parts downstream, even ones that don’t seem to be directly connected, can interact with each other through complex, non-linear feedback through far-flung networks.

This is very unlike engineering. The equivalent of these unpredictable consequences in building a bridge, for example, would be for a second bridge to sprout out of nowhere when the first one is built, or the rock on the other side of the river suddenly turning from metamorphic to sedimentary, or the sum of weights of two parallel beams on the bridge being more than what simple addition would suggest. Or imagine the Apollo rocket suddenly accelerating to ten times its speed when the booster rockets fall off. Or the shape of the reentry vehicle suddenly changing through some weird feedback mechanisms as it reaches a certain temperature when it’s hurtling through the atmosphere. 

Whatever the complexities of challenging engineering projects like building rockets or bridges, they are still highly predictable compared to the effects of engineering biology. The fact of the matter is that the laws of aerodynamics and gravity were extremely well understood before the Apollo program (literally) took off the ground, so as amazing as the achievement was, it didn't involve discovering new basic scientific laws on the fly, something that we do a lot in biology. Aircraft design is decidedly not drug design. And all this is simply a product of ignorance, ignorance of the laws of biology and evolution – a clunky, suboptimal, haphazard, opportunistic process if there ever was one – relative to the laws of (largely predictable) Newtonian physics that underlie engineering problems.

The concept of modularity in biology therefore becomes very tricky compared to engineering. There is some modularity in biology for sure, but it’s not going to take you all the way. One of the reasons is that unlike modularity in engineering, biological modularity is flexible, both spatially and temporally. This is again a consequence of different levels of emergence. For instance, a long time ago we thought that the brain was modular and that the fundamental modules were neurons. This view has now changed and we think that it’s networks of neurons that are the basic modules. But we don’t even think that these modular networks are fixed through space and time; they likely form, dissolve and change members and locations in the brain according to need, much like political groups fleetingly forming and breaking apart for convenience. The problem is that we don’t know what level of modularity is relevant to addressing a particular problem. For instance, is the right ‘module’ for thinking about Alzheimer’s disease the beta-amyloid protein, or is it the mitochondria and its redox state, or is it the gut-brain axis and the microbiome? In addition, modules in biology are again non-linear, so the effects from combining two modules are not going to simply be twice the effects of one module – they can be twice or half or even zero.

Now, having noted all these problems, I certainly don’t think that biology cannot benefit at all from the principles of engineering. For one thing, I have always thought that biologists should really take the “move fast and break things” philosophy of software engineering to heart; we simply don’t spend enough time trying to break and falsify hypotheses, and this leads to a lot of attrition and time chasing ghosts down rabbit holes. More importantly though, as a big fan of tool-driven scientific revolutions, I do believe that inventing tools like CRISPR and sequencing will allow us to study biological systems at an increasingly fine-grained level. They will allow us to gather more measurements that would allow better AL/machine learning models, and I am all for this.

But all this will work as far as we realize that the real problem is not improving the measurements, it’s knowing what measurements to make in the first place. Otherwise we find ourselves in the classic position of the drunkard trying to find his keys below the lamp, because that’s where the light is. Inventing better lamps, or a metal detector for that matter, is not going to help if we are looking in the wrong place for the keys. Or looking for keys when we should really be looking for a muffin.

Unifiers and diversifiers in physics, chemistry and biology

On my computer screen right now are two molecules. They are both large rings with about thirty atoms each, a motley mix of carbons, hydrogens, oxygens and nitrogens. In addition they have appendages of three or four atoms dangling off their periphery. There is only one, seemingly minor difference: The appendage in one of the rings has two more carbon atoms than that in the other. If you looked at the two molecules in flat 2D - in the representation most familiar to practicing chemists - you will sense little difference between them.

Yet when I look at the two molecules in 3D - if I look at their spatial representations or conformations - the differences between them are revealed in their full glory. The presence of two extra carbons in one of the compounds causes it to scrunch up, to slightly fold upon itself the way a driver edges close to the steering wheel. This slight difference causes many atoms which are otherwise far apart to come together and form hydrogen bonds, weak interactions that are nonetheless essential in holding biological molecules like DNA and proteins together. These hydrogen bonds can in turn modulate the shape of the molecule and allow it to get past cell membranes better than the other one. A difference of only two carbons - negligible on paper- can thus have profound consequences for the three-dimensional life of these molecules. And this difference in 3D can in turn translate to significant differences in their functions, whether those functions involve capturing solar energy or killing cancer cells.

Chemistry is full of hidden differences and similarities like these. Molecules exist on many different levels, and on each level they manifest unique properties. In one sense they are like human beings. On the surface they may appear similar, but probe deeper and each one is unique. And probing even deeper may then again reveal similarities. They are thus both similar and different all at once. But just like human beings molecules are shy; they won't open up unless you are patient and curious, they may literally fall apart if you are too harsh with them, and they may even turn the other cheek and allow you to study them better if you are gentle and beguiling enough. It is often only through detailed analysis that you can grasp their many-splendored qualities. It is this ever-changing landscape of multifaceted molecular personalities, slowly but surely rewarding the inquisitive and dogged mind, that makes chemistry so thrilling and open-ended. It is why I get a kick out of even mundane research.

When I study the hidden life of molecules I see diversity. And when I see diversity I am reminded of how important it is in all of science. Sadly, the history of science in the twentieth century has led both scientists and the general public to value unity over diversity. The main culprit in this regard has been physics whose quest for unity has become a victim of its own success. Beginning with the unification of mechanics with heat and electricity with magnetism in the nineteenth century, physics achieved a series of spectacular feats when it combined space with time, special relativity with quantum mechanics and the weak force with electromagnetism. One of the greatest unsolved problems in physics today is the combination of quantum mechanics with general relativity. These unification feats are both great intellectual achievements as well as noteworthy goals, but they have led many to believe that unification is the only thing that really matters in physics, and perhaps in all of science. They have also led to the belief that fundamental physics is all that is worth studying. The hype generated by the media in fields like cosmology and string theory and the spate of popular books written by scientist-celebrities in these fields have only made matters worse. All this is in spite of the fact that most of the world's physicists don't study fundamental physics in their daily work.

The obsession with unification has led to an ignorance of the diversity of discoveries in physics. In parallel with the age of the unifiers has existed the universe of diversifiers. While the unifiers have been busy proclaiming discoveries from the rooftops, the diversifiers have been quietly building new instruments and cataloging the reach of physics in less fundamental but equally fascinating fields like solid-state physics and biophysics. They have also gathered the important data which allowed the unifiers to ply their trade. Generally speaking, unifiers tend to be part of idea-driven revolutions while diversifiers tend to be part of tool-driven revolutions. The unifiers would never have seen their ideas validated if the diversifiers had not built tools like telescopes, charged coupled devices and, superconducting materials to test the great theories of physics. And yet, just like unification is idolized at the expense of diversification, ideas in physics have also been lionized at the expense of practical tools. We need to praise the tools of physics as much as the diversifiers who build them.

As a chemist I find it easier to appreciate diversity. Examples of molecules like the ones I cited above abound in chemistry. In addition chemistry is too complex to be reduced to a simple set of unifying principles, and most chemical discoveries are still made by scientists looking at special cases rather than those searching for general laws. It's also a great example of a tool-driven revolution, with new instrumental technologies like x-ray diffraction and nuclear magnetic resonance (NMR) completely revolutionizing the science during the twentieth century. There were of course unifiers in chemistry too - the chemists who discovered the general laws of chemical bonding are the most prominent example - but these unifiers have never been elevated to a status seen among physicists. Diversifiers who play in the mud of chemical phenomena and find chemical gems are still more important than ones who might proclaim general theories. There will always be the example of an unusual protein structure, a fleeting molecule whose existence defies our theories or or a new polymer with amazing ductility that will keep chemists occupied. And this will likely be the case for the foreseeable future.

Biology too has seen its share of unifiers and diversifiers. For most of its history biology was the ultimate diversifiers' delight, with intrepid explorers, taxonomists and microbiologists cataloging the wonderful diversity of life around us. When Charles Darwin appeared on the scene he unified this diversity in one stunning fell swoop through his theory of evolution by natural selection. The twentieth century modern synthesis of biology that married statistics, genetics and evolutionary biology was also a great feat of unification. And yet biology continues to be a haven for diversifier. There is always the odd protein, the odd sequence of gene or the odd insect with a particularly startling method of reproduction that catches the eye of biologists. These examples of unusual natural phenomena do not defy the unifying principles, but they do illustrate the sheer diversity in which the unifying principles can manifest themselves, especially on multiple emergent levels. They assure us that no matter how much we may unify biology, there will always be a place for diversifiers in it.

At the dawn of the twenty-first century there is again a need for diversifiers, especially in new fields like neuroscience and paleontology. We need to cast off the spell of fundamental physics and realize that diversifiers play on the same field as unifiers. Unifiers may come up with important ideas, but diversifiers are the ones who test them and who open up new corners of the universe for unifiers to ponder. Whether in chemistry or physics, evolutionary biology or psychology, we should continue to appreciate unity in diversity and diversity in unity. Together the two will advance science into new realms.

Psychiatry and neuroscience: Don't sacrifice proven emergence at the altar of unproven reductionism

John Markowitz who is a clinical psychiatrist at the NIH has a cogent column in the New York Times in which he argues that an excessive focus on neuroscience translational research is stifling useful and proven research in psychiatry. His main point is that the neuroscience research is unproven and long term, and while it may promise attractive dividends, there are many patients who need good psychiatric treatment now, patients who cannot work along the timelines promised by cutting edge neuroscience work.

I think in general he's right. Neuroscience seeks to find out the basic mechanisms governing neural health and disease by way of genes, receptors and small molecule drugs. Psychiatry and especially psychotherapy takes a more empirical and holistic approach, trying various combinations of talk therapy and drugs to treat mental illness. Even psychiatry itself has suffered from the kind of crisis that the author talks about. For instance, it is now increasingly clear that talk therapy (especially CBT) works at least as well as psychiatric drugs like antidepressants.

To me, at least part of the debate seems to be about a topic that I have often explored on this blog: emergence vs reductionism. Generally speaking, the goals of neuroscience are reductionist, seeking to modulate mental processes in health and disease by understanding and engineering interactions between genes, proteins and drugs at the molecular and network levels. The goals of psychiatry are emergent and empirical. Psychiatry does not care about the underlying molecular mechanisms of mental health; instead its goal is to work at a higher and more holistic level, empirically trying out different approaches until a particular combination of methods seems to show efficacy. It is not surprising that drugs like antidepressants which aim to interact with specific protein receptors in the brain are often found wanting because they target only part of a much larger system.

This philosophical difference between neuroscience and psychotherapy also strikes me as being a bit similar to the philosophical difference between chemistry and physics which I have often talked about here. Physics may want to find out how the world works by tracing the interactions between elementary particles like quarks, but chemists have little use for this information, benefiting tremendously instead by understanding semi-empirical concepts like hydrogen bonds and hydrophobic effects. The time discrepancy that the author points out regarding the fruits of neuroscience research and psychiatry also applies to physics and chemistry; if chemists waited long enough to be able to use physics and understand every complex molecular system from first principles, we would still be living in the age of alchemy.

The NYT article ends by appealing to the NIH to not sacrifice proven empirical psychiatry research at the altar of long-term translational research in neuroscience, and this underscores yet another one of the more general problems with translational research that I and others have pointed out. It is why the much celebrated and publicized Brain Initiative troubles me; I fear that it will detract from more mundane but effective psychiatric research. Far flung reductionist research may well promise and eventually bring great insights, but it should not be pursued at the cost of immediately workable emergent research whose very lack of precision makes it so useful.

Physicist Leo Kadanoff on reductionism and models: "Don't model bulldozers with quarks."

I have been wanting to write about Leo Kadanoff who passed away a few weeks ago. Among other things Kadanoff made seminal contributions to statistical physics, specifically the theory of phase transitions, that were undoubtedly Nobel caliber. But he should also be remembered for something else - a cogent and very interesting attack on 'strong reductionism' and a volley in support of emergence, topics about which I have written several times before.

Kadanoff introduced and clarified what can be called the "multiple platform" argument. The multiple platform argument is a response to physicists like Steven Weinberg who believe that higher-order phenomena like chemistry and biology have a strict on-on-one relationship with lower-order physics, most notably quantum mechanics. Strict reductionists like Weinberg tell us that "the explanatory arrows always point downward". But leading emergentist physicists like P W Anderson and Robert Laughlin have taken objection to this interpretation. Some of their counterarguments deal with very simple definitions of emergence; for instance a collection of gold atoms have a property (the color yellow) that does not directly flow from the quantum properties of individual gold atoms.

Kadanoff further revealed the essence of this argument by demonstrating that the Navier-Stokes equations which are the fundamental classical equations of fluid flow cannot be accounted for purely by quantum mechanics. Even today one cannot directly derive these equations from the Schrodinger equation, but what Kadanoff demonstrated is that even a simple 'toy model' in which classical particles move around on a hexagonal grid can give rise to fluid behavior described by the Navier-Stokes equations. There clearly isn't just one 'platform' (quantum mechanics) that can account for fluid flow. The complexity theorist Stuart Kauffman captures this well in his book "Reinventing the Sacred".

Others have demonstrated that a simple 'bucket brigade' toy model in which empty and filled buckets corresponding to binary 1s and 0s (which in turn can be linked to well-defined quantum properties) that are being passed around can account for computation. Thus, as real as the electrons obeying quantum mechanics which flow through the semiconducting chips of a computer are, we do not need to invoke their specific properties in order to account for a computer's behavior. A simple toy model can do equally well.

Kadanoff's explanatory device is in a way an appeal to the great utility of models which capture the essential features of a complicated phenomenon. But at a deeper level it's also a strike against strong reductionism. Note that nobody is saying that a toy model of classical particles is a more accurate and fundamental description of reality than quantum mechanics, but what Kadanoff and others are saying is that the explanatory arrows going from complex phenomena to simpler ones don't strictly flow downward; in fact the details of such a flow cannot even be truly demonstrated.

In some of his other writings Kadanoff makes a very clear appeal based on such toy models for understanding complex systems. Two of his statements provide the very model of pithiness when it comes to using and building models:

"1. Use the right level of description to catch the phenomena of interest. Don't model bulldozers with quarks.

2. Every good model starts from a question. The modeler should aways pick the right level of detail to answer the question."

"This lesson applies with equal strength to theoretical work aimed at understanding complex systems. Modeling complex systems by tractable closure schemes or complicated free-field theories in disguise does not work. These may yield a successful description of the small-scale structure, but this description is likely to be irrelevant for the large-scale features. To get these gross features, one should most often use a more phenomenological and aggregated description, aimed specifically at the higher level. 

Thus, financial markets should not be modeled by simple geometric Brownian motion based models, all of which form the basis for modern treatments of derivative markets. These models were created to be analytically tractable and derive from very crude phenomenological modeling. They cannot reproduce the observed strongly non-Gaussian probability distributions in many markets, which exhibit a feature so generic that it even has a whimsical name, fat tails. Instead, the modeling should be driven by asking what are the simplest non-linearities or non-localities that should be present, trying to separate universal scaling features from market specific features. The inclusion of too many processes and parameters will obscure the desired qualitative understanding."

This paragraph captures as well as anything else why chemistry requires its own language, rules and analytical devices for understanding its details and why biology and psychology require their own similar implements. Not everything can be understood through quantum mechanics, because as you try to get more and more fundamental, true understanding might simply slip away from between your fingers.

RIP, Leo Kadanoff.

Neuroscience and other theory-poor fields: Tools first, simulation later

I have written about the ‘Big Brain Project’ a few times before including a post for the Canadian TV channel TVO last year. The project basically seeks to make sense of that magnificent 3-pound bag of jelly inside our skull at multiple levels, from molecules to neurons to interactions at the whole brain level. The aims of the project are typical of ‘moon shot’ endeavors; ambitious, multidisciplinary, multi-institutional and of course, expensive. Yet right after the project was announced in both the US (partly by President Obama) and in Europe there were whispers of criticism that turned first into a trickle and then into a cascade. The criticism was at multiple levels – administrative, financial and scientific. But even discounting the administrative and financial problems, many scientists saw issues with the project even at the basic scientific level.

The gist of those issues can be boiled down to one phrase: “trying to chew on more than we can bite off”. Basically we are trying to engineer a complex, emergent system whose workings we still don’t understand, even at basic levels of organization. Our data is impoverished and our approaches are too reductionist. One major part of the project especially suffers from this drawback – in-silico simulation of the brain at multiple levels, from neurons to entire mouse and human brains. Now here’s a report from a committee which has examined the pros and cons of the project and reached the conclusion that much of the criticism was indeed valid, and that we are trying to achieve something for which we still don’t have the tools. The report is here. The conclusion of the committee is simple: first work on the tools; then incorporate the findings from those tools into a bigger picture. The report makes this clear in a paragraph that also showcases problems with the public’s skewed perception of the project.

The goal of reconstructing the mouse and human brain in silico and the associated comprehensive bottom-up approach is viewed by one part of the scientific community as being impossible in principle or at least infeasible within the next ten years, while another part sees value not only in making such simulation tools available but also in their development, in organizing data, tools and experts (see, e.g., http://www.bbc.com/future/story/ 20130207-will-we-ever-simulate-the-brain). A similar level of disagreement exists with respect to the assertion that simulating the brain will allow new cures to be found for brain diseases with much less effort than in experimental investigations alone.

The public relations and communication strategy of the HBP and the continuing and intense public debate also led to the misperception by many neuroscientists that the HBP aims to cover the field of neuroscience comprehensively and that it constitutes the major neuroscience research effort in the European Research Area (ERA).

This whole discussion reminds me of the idea of tool-driven scientific revolutions publicized by Peter Galison, Freeman Dyson and others, of which chemistry is an exemplary instance. The Galisonian picture of scientific revolutions does not discount the role of ideas in causing seismic shifts in science, but it places tools on an equal footing. Discussions of grand ideas and goals (like simulating a brain) often give short shrift to the mundane but critical everyday tools that need to be developed in order to enable those ideas in the first place. They are great for sound bytes for the public but brittle in their foundations. Although scientific ideas are often considered the progenitors of a lot of everyday scientific activity by the public, in reality the progression can equally often be the opposite: first come the tools, then the ideas. Sometimes tools can follow ideas, as was the case with a lot of predictions of the general theory of relativity. At other times ideas follow the tools and the experiments, as was the case with the Lamb Shift and quantum electrodynamics. 

Generally speaking it’s more common for ideas to follow tools when a field is theory-poor, like quantum field theory was in the 1930s, while it’s more common for tools to follow ideas when a field is theory-rich. From this viewpoint neuroscience is currently theory-poor, so it seems much more likely to me that ideas will follow the tools in the field. To be sure the importance of tools has long been recognized in neurology; where would we be without MRI and patch-clamp techniques for instance? And yet these tools have only started to scratch the surface of what we are trying to understand. We need much better tools before we get our hands on a theory of the brain, let alone one of the mind.

I believe the same progression also applies to my own field of molecular modeling in some sense. Part of the problem with modeling proteins and molecules is that we still don’t have a good idea of the myriad factors that drive molecular recognition. We have of course had an inkling of these factors (such as water and protein dynamics) for a while now but we haven’t really had a good theoretical framework to understand the interactions. We can wave this objection away by saying that sure we have a theoretical framework, that of quantum mechanics and statistical mechanics, but that would be little more than a homage to strong reductionism. The problem is we still don’t have a handle on the quantitative contribution of various factors to protein-small molecule binding. Until we have this conceptual understanding the simulation of such interactions is bound to suffer. And most importantly, until we have such understanding what we really need is not simulation but improved instrumental and analytical techniques that enable us to measure even simple things like molecular concentrations and the kinetics of binding. Once we get an idea of these parameters using good tools, we can start incorporating the parameters in modeling frameworks.

Now the brain project is indeed working on tools too, but reports like the current one ask whether we need to predominantly focus on those tools and perhaps divert some of the money and attention from the simulation aspects of the project to the tool-driven aspects. The message from the current status report is ultimately simple: we need to first stand before we can run.

Image link

Modular complexity and the problem of reverse engineering the brain

Bell's number calculates the number of connections between
various components of a system and scales exponentially
with those components (Image: Science Magazine).

I have been reading an excellent collection of essays on the brain titled "The Future of the Brain" which contains ruminations on current and future brain research from leading neuroscientists and other researchers like Gary Marcus, George Church and the Moser husband and wife pair who won last year's Nobel prize. Quite a few of the authors are from the Allen Institute for Brain Science in Seattle. In starting this institute, Microsoft co-founder Paul Allen has placed his bets on mapping the brain…or at least the mouse visual cortex for starters. His institute is engaged in charting the sum total of neurons and other working parts of the visual cortex and then mapping their connections. Allen is not alone in doing this; there’s projects like the Connectome at MIT which are trying to do the same thing (and the project’s leader Sebastian Seung has written a readable book about it).

Now we have heard prognostications about mapping and reverse engineering brains from more eccentric sources before, but fortunately Allen is one of those who does not believe that the singularity is around the corner. He also seems to have entrusted his vision to sane minds. His institute’s chief science officer is Christof Koch, former professor at Caltech, longtime collaborator of the late Francis Crick and self-proclaimed “romantic reductionist” who started at the institute earlier this year. Koch has written one of the articles in the essay collection. His article and the book in general reminded me of a very interesting perspective that he penned in Science last year which points out the staggering challenge of understanding the connections between all the components of the brain; the “neural interactome” if you will. The article is worth reading if you want to get an idea of how even simple numerical arguments illuminate the sheer magnitude of mapping out the neurons, cells, proteins and connections that make up the wonder that’s the human brain.

Koch starts by pointing out that calculating the interactions between all the components in the brain is not the same as computing the interactions between, say, all atoms of an ideal gas since unlike a gas, the interactions are between different kinds of entities and are therefore not identical. Instead, he proposes, we have to use something called Bell’s number B_n which reminds me of the partitions that I learnt about when I was sleepwalking through set theory in college. Briefly for n objects, B_n refers to the number of combinations (doubles, triples, quadruples etc.) that can be formed. Thus, when n=3 B_n is 5. Not surprisingly, B_n scales exponentially with n and Koch points out that B₁₀ is already 115,975. If we think of a typical presynaptic terminal with its 1000 proteins or so, B_n starts giving us serious heartburn. For something like the visual cortex where n= 2 million B_n would be inconceivable, and it's futile to even start thinking about what the number would be for the entire brain. Koch then uses a simple calculation based on Moore’s Law in trying to estimate the time needed for “sequencing” these interactions. For n = 2 million the time needed would be of the order of 10 million years. And as the graph on top demonstrates, for more than 10⁵ components or so the amount of time spirals out of hand at warp speed.

This considers only the 2 million neurons in the visual cortex; it doesn’t even consider the proteins and cells which might interact with the neurons on an individual basis. In addition, at this point we are not even really aware of how neuronal types there are in the brain: neurons are not all identical like indistinguishable electrons. What makes the picture even more complicated that these types may be malleable so that sometimes a single neuron can be of one type while at other types it can team up with other neurons to form a unit that is of a different type. This multilayered, fluid hierarchy rapidly reveals the outlines of what Paul Allen has called the “complexity brake”: he described this in the same article that was cogently critical of Ray Kurzweil's singularity. And the neural complexity brake that Koch is talking about seems poised to make an asteroid-sized impact on our dreams.

So are we doomed in trying to understand the brain, consciousness and the whole works? Not necessarily, argues Koch. He gives the example of electronic circuits where individual components are grouped separately into modules. If you bunch a number of interacting entities together and form a separate module, then the complexity of the problem reduces since you now have to only calculate interactions between modules. The key question then is, is the brain modular, and how many modules does it present? Commonsense would have us think it is modular, but it is far from clear how we can exactly define the modules. We would also need a sense of the minimal number of modules to calculate interactions between them. This work is going to need a long time (hopefully not as long as that for B_{2 million}) and I don’t think we are going to have an exhaustive list any time soon, especially since these are going to be composed of different kinds of components and not just one kind. But it's quite clear that whataver the nature of these modules, delineating their particulars would go a long way in making the problem more manageable.

Any attempt to define these modules are going to run into problems of emergent complexity that I have occasionally written about. Two neurons plus one protein might be different from two neurons plus two proteins in unanticipated ways. Also if we are thinking about forward and reverse neural pathways, I would hazard a guess that one neuron plus one neuron in one direction may even be different from the same interaction in the reverse direction. Then there’s the more obvious problem of dynamics. The brain is not a static entity and its interactions would reasonably be expected to change over time. This might interpose a formidable new barrier in brain mapping, since it may mean that whatever modules are defined may not even be the same during every time slice. A fluid landscape of complex modules whose very identity changes every single moment could well be a neuroscientist’s nightmare. In addition, the amount of data that captures such neural dynamics would be staggering since even a millimeter sized volume of rat visual tissue requires a few terabytes of data to store all its intricacies. However, the data storage problem pales in comparison to the data interpretation problem.

Nevertheless this goal of mapping modules seems far more attainable in principle than calculating every individual interaction, and that’s probably the reason Koch left Caltech to join the Allen Institute in spite of the pessimistic calculation above. The value of modular approaches goes beyond neuroscience though; similar thinking may provide insights into other areas of biology, such as the interaction of genes with proteins and of proteins with drugs. As an amusing analogy, this kind of analysis reminds me of trying to understand the interactions between different components in a stew; we have to appreciate how the salt interacts with the pepper and how the pepper interacts with the broth and how the three of them combined interact with the chicken. Could the salt and broth be considered a single module?

If we can ever get a sense of the modular structure of the brain, we may have at least a fighting chance to map out the whole neural interactome. I am not holding my breath too hard, but my ears will be wide open since this is definitely going to be one of the most exciting areas of science around.

Adapted from a previous post on Scientific American Blogs.

Edward Witten, chemistry and the problems with falsification

Science writer and journalist John Horgan who wrote the notorious and thought-provoking book "The End of Science" in the 90s has an interesting interview with theoretical physics giant Edward Witten. Witten, who won the Fields Medal back in the 80s, is widely regarded because of his huge status and influence as one of the main reasons a number of physicists switched to doing string theory in the 90s.

The whole interview is worth reading but there was one chemistry-related response that Witten gave which especially caught my eye:

Horgan: Do you agree with Sean Carroll that falsifiability is overrated as a criterion for distinguishing science from pseudo-science? 

Witten: Scientists aim to get as reliable and precise an understanding of nature as we can.  The gold standard is a precise prediction that can be tested in a precise way in a laboratory experiment.  Experiments that disprove theories are an important part of the scientific process. 

With that said, it is a little too narrow to claim that science consists of trying to falsify theories because a lot of science consists of trying to discover things. (Chemists who attempt a new synthesis could say they are trying to falsify the hypothesis that this new synthesis won’t work.  But that isn’t what they usually say.  People who search for life on Mars could say they are trying to falsify the hypothesis that there is no life on Mars. Again, people don’t usually talk that way.)

I have to give credit to Witten for pointing out the problems with falsifiability in the context of chemistry; in my experience there's not even too many chemists who talk about this rather central aspect of the science. The most prominent one who does is Roald Hoffmann who in his recent collection of essays (which I reviewed for Nature Chemistry) does analyze the issues with using falsifiability as a significant criterion in the philosophy of chemistry. Hoffmann also points to other well-known constructs from the philosophy of science, such as hypothesis testing, as being insufficient guides in understanding the methodology of chemistry.

As both Hoffmann and Witten point out, most chemists when they are trying to make molecules are not really trying to falsify anything, except in the trivial and general sense of trying to "falsify" the basic principles of chemistry. But this fact also goes to the heart of chemistry as a science and art that is more akin to architecture and which really tries to build things rather than simply breaking them down. That is also the reason why, in my opinion, the whole reductionist paradigm has some clear limitations when applied to chemistry; most chemists are really trying to see how emergent molecular properties arise from putting atoms together in different arrangements. Breaking things down in the form of elemental or spectroscopic analysis is of course a reductionist activity that's an essential part of chemistry, but even that process really is in service to understanding how atoms can combine together to give rise to novel structure and function.

It's good to have even physicists like Witten point this out. But it's also something that more chemists and chemistry popularizers should be aware of, especially when they describe their philosophy of doing science to the wider world. Explicating the unique philosophy of chemistry not only sheds light on why chemistry is different in its own way but also demonstrates why we must not hold exalted notions from the philosophy of science like falsification sacrosanct.

The Higgs boson and the future of science

My latest post on the Scientific American blog network ties together several threads about reductionism, emergence and the nature of scientific problems which I have explored on this blog.

Philip Anderson: Anderson first described the so-called Higgs mechanism and also fired the first modern salvo against strong reductionism (Image: Celeblist)

The discovery of the Higgs boson (or the "Higgs-like particle" if you prefer) is without a doubt one of the signal scientific achievements of our time. It illustrates what sheer thought - aided by data of course - can reveal about the workings of the universe and it continues a trend that lists Descartes, Hume, Galileo and Newton among its illustrious forebears. From sliding objects down an incline to smashing atoms at almost the speed of light in a 27 kilometer tunnel, we have come a long way. Dissecting our origins and the universe around us scarcely gets any better than this.

Yet even as the exciting discovery was being announced, I could not help but think about what the Higgs does not do for us. It does not speed up the time needed to discover a new cancer drug. It does not help us understand consciousness. It does not tell us how life began or whether it exists elsewhere in the universe. It does not explain romantic love, how to design the best solar cell, why people have certain political preferences and how exactly to predict the effects of climate change. In fact we can safely predict that the discovery of the Higgs boson, as consciousness-elevating as it is, does not impact the daily work of 99% of all pure and applied scientists in the world.

I do not say all this to downplay the discovery of the particle which is an unparalleled triumph of human thought, hard work and experimental ingenuity. I also do not say this to make the obvious point that a discovery in one field of science does not automatically solve problems in other fields. Rather, I say this to probe the deeper reality beyond that point, to highlight the multifaceted nature of science and the sheer diversity of problems and phenomena that it presents to us at every level of inquiry. And I say this with a suspicion that the Higgs boson may be the most fitting tribute to the limitations of what has been the most potent philosophical instrument of scientific discovery - reductionism.

In one sense the discovery of this fundamental component of matter can be seen as the culmination of reductionist thinking, accounting as it does for the very existence of mass. Reductionism is the great legacy of the twentieth century, a philosophy whose seeds were sown when Greek philosophers started mulling the nature of matter. The method is in fact quite intuitive; ever since they stepped down from the trees, human beings have tried to solve problems by breaking them down into simpler parts. In the twentieth century the fruits of reductionism have been nothing short of awe-inspiring. Reductionism is what told us that molecules are made of atoms, that the universe is expanding, that DNA is a double helix and that you can build lasers and computers. The reductionist ethic has given us quantum mechanics, relativity, quantum chemistry and molecular biology. Over the centuries it has been used by its countless practitioners as a fine scalpel which has laid bare the secrets of nature. In fact many of the questions answered using the reductionist method were construed as being amenable to this method even before their answers were provided; for instance, how do atoms combine to form molecules? What is the basic nature of the gene? What are atoms themselves made up of?

Yet as we enter the second decade of the twenty-first century, it is clear that reductionism as a principal weapon in our arsenal of discovery tools is no longer sufficient. Consider some of the most important questions facing modern science, almost all of which deal with complex, multifactorial systems. How did life on earth begin? How does biological matter evolve consciousness? What are dark matter and dark energy? How do societies cooperate to solve their most pressing problems? What are the properties of the global climate system? It is interesting to note at least one common feature among many of these problems; they result from the buildup rather than the breakdown of their operational entities. Their signature is collective emergence, the creation of attributes which are greater than the sum of their constituent parts. Whatever consciousness is for instance, it is definitely a result of neurons acting together in ways that are not obvious from their individual structures. Similarly, the origin of life can be traced back to molecular entities undergoing self-assembly and then replication and metabolism, a process that supersedes the chemical behavior of the isolated components. The puzzle of dark matter and dark energy also have as their salient feature the behavior of matter at large length and time scales. Studying cooperation in societies essentially involves studying group dynamics and evolutionary conflict. The key processes that operate in the existence of all these problems seem to almost intuitively involve the opposite of reduction; they all result from the agglomeration of molecules, matter, cells, bodies and human beings across a hierarchy of unique levels. In addition, and this is key, they involve the manifestation of unique principles emerging at every level that cannot be merely reduced to those at the underlying level.

The traditional picture of science asserts that X can be reduced to Y. Reality is more complicated (Image: P. W. Anderson, Science, 1972)

A classic example of emergence: The exact shape of a termite mound is not reducible to the actions of individual termites (Image: Wikipedia Commons)

This kind of emergence has long since been seen as key to the continued unraveling of scientific mysteries. While emergence had been implicitly appreciated by scientists for a long time, its modern salvo was undoubtedly a 1972 paper in Science by the Nobel Prize winning physicist Philip Anderson titled "More is Different", a title that has turned into a kind of clarion call for emergence enthusiasts. In his paper Anderson (who incidentally first came up with the so-called Higgs mechanism) argued that emergence was nothing exotic; for instance, a lump of salt has properties very different from those of its highly reactive components sodium and chlorine. A lump of gold evidences properties like color that don't exist at the level of individual atoms. Anderson also appealed to the process of broken symmetry, invoked in all kinds of fundamental events - including the existence of the Higgs boson - as being instrumental for emergence. Since then, emergent phenomena have been invoked in hundreds of diverse cases, ranging from the construction of termite hills to the flight of birds. The development of chaos theory beginning in the 60s further illustrated how very simple systems could give rise to very complicated and counterintuitive patterns and behavior that are not obvious from the identities of the individual components.

Many scientists and philosophers have contributed to considered critiques of reductionism and an appreciation of emergence since Anderson wrote his paper. These thinkers make the point that not only does reductionism fail in practice (because of the sheer complexity of the systems it purports to explain), but it also fails in principle on a deeper level. In his book "The Fabric of Reality" for instance, the Oxford physicist David Deutsch has made the compelling point that reductionism can never explain purpose; to drive home this point he asks us if it can account for the existence of a particular atom of copper on the tip of the nose of a statue of Winston Churchill in London. Deutsch's answer is a clear no, since the fate of that atom was based on contingent, emergent phenomena, including war, leadership and adulation. Nothing about the structure of copper atoms allows us to directly predict that a particular atom will someday end up on the tip of that nose. Chance plays an outsized role in these developments and reductionism offers us little solace to understand such historical accidents.

Complexity theorist Stuart Kauffman who has written about the role of contingency as a powerful argument against strong reductionism (Image: Wikipedia Commons)

An even more forceful proponent of this contingency-based critique of reductionism is the complexity theorist Stuart Kauffman (supposedly an inspiration for the Jeff Goldblum character in "Jurassic Park") who has laid out his thoughts in two books. Just like Anderson, Kauffman does not deny the great value of reductionism in illuminating our world, but he also points out the factors that greatly limit its application. One of his favorite examples is the role of contingency in evolution and the object of his attention is the mammalian heart. Kauffman makes the case that no amount of reductionist analysis could explain tell you that the main function of the heart is to pump blood. Even in the unlikely case that you could predict the structure of hearts and the bodies that house them starting from the Higgs boson, such a deductive process could never tell you that of all the possible functions of the heart, the most important one is to pump blood. This is because the blood-pumping action of the heart is as much a result of historical contingency and the countless chance events that led to the evolution of the biosphere as it is of its bottom-up construction from atoms, molecules, cells and tissues. As another example, consider the alpha amino acids which make up all proteins on earth. These amino acids come in two potential varieties, left-handed and right-handed. With very few exceptions, all the functional amino acids that we know of are left handed, but there's no reason to think that right handed amino acids wouldn't have served life equally well. The question then is, why left-handed amino acids? Again, reductionism is silent on this question mainly because the original use of left-handed amino acids during the origin of life was to the best of our knowledge a matter of contingency. Now some form of reductionism may still explain the subsequent propagation of left-handed amino acids and their dominance in biological processes by resorting to molecular level arguments regarding chemical bonding and energetics, but this description will still leave the origins issue unresolved. Even something as fundamental as the structure and function of DNA - which by all accounts was a triumph of reductionism - is much better explained by principles of chemistry like electrostatic attraction and hydrogen bonding.

Life as we know it is based on left-handed amino acids. But there is no reason why right-handed amino acids could not sustain life (Image: Islamickorner)

Reductionism then falls woefully short when trying to explain two things; origins and purpose. And one can see that if it has problems even when dealing with left-handed amino acids and human hearts, it would be in much more dire straits when attempting to account for say kin selection or geopolitical conflict. The fact is that each of these phenomena are better explained by fundamental principles operating at their own levels. Chemistry has its covalent bonds and steric effects, geology has its weathering and tectonic shifts, neurology has its memory potentiation and plasticity and sociology has its conflict theory. And as far as we can tell, these sciences will continue to progress without needing the help of Higgs bosons and neutrinos. This also seems to make it unlikely that the discovery of a single elegant equation linking the four fundamental forces (the purported "theory of everything"), while undoubtedly representing one of the greatest intellectual achievements of humanity, will give sociologists and economists little pause for thought, even as they continue to study the stock market and democracies using their own special toolkit of bedrock principles.

This rather gloomy view of reductionism may sound like science is at a dead end or at the very least has started collapsing under the weight of its own success. But such a view would be as misplaced as announcements about the "end of science" which have surfaced every couple of years for the last two hundred years. Every time the end of science has been announced, science itself proved that claims of its demise were vastly exaggerated. Firstly, reductionism will always be alive and kicking since the general approach of studying anything by breaking it down into its constituents will continue to be enormously fruitful. But more importantly, it's not so much the end of reductionism as the beginning of a more general paradigm that combines reductionism with new ways of thinking. The limitations of reductionism should be seen as a cause not for despair but for celebration since it means that we are now entering new, uncharted territory. There are still an untold number of deep mysteries that science has to solve, ranging from dark energy, consciousness and the origin of life to more supposedly pedestrian concerns like superconductivity, cancer drug discovery and the behavior of glasses. Many of these questions require interdisciplinary approaches which result in the crafting of fundamental principles that are unique to the problem statement. Such a meld will inherently involve reductionism only as one component.

Now there are some who may not consider these problems as "fundamental" enough but that is because they would be peering through the lens of traditional twentieth century science. One of the sad casualties of the reductionist undertaking is a small group of people who think that cosmology and particle physics constitute the only things truly worth doing and the epitome of fundamental science; the rest is all detail that can be filled in by second-rate minds. This is in spite of the inconvenient fact that perhaps 80% of physicists are not concerned at all with fundamental questions. But you would be deluding yourself if you are thinking that turbulence in fluids is a second-rate problem (still unsolved) for second-rate minds, especially if you remember that Heisenberg thought that God would will be able to provide an explanation for quantum mechanics but not for turbulence. The fact is that "pedestrian" concerns like superconductivity have engaged some of the best minds of the last fifty years without fully succumbing to them, and at their own levels they are as hard as the discovery of the Higgs boson or the accelerating universe. Exploring these worthy conundrums is every bit as exciting, deep and satisfying as any other endeavor in science. Those who are wondering what's next should not worry; a sparkling journey lies ahead.

To guide us on this journey all we have to remember are the words of one of the twentieth century's great reductionists and one of Peter Higgs's heroes. Paul Dirac closed his famous text on quantum theory with stirrings that will hopefully be as great a portent for the emergent twenty-first century as they were for the reductionist twentieth: "Some new principles are here needed".

References:
1. P. W. Anderson, More is Different, Science, 1972, 177, 393
2. David Deutsch, "The Fabric of Reality", 2004
3. Stuart Kauffman, "Reinventing the Sacred", 2009; "At Home in the Universe", 1996
Other reading:
1. Terrence Deacon, "Incomplete Nature", 2011
2. John Horgan, "The End of Science", 1997
3. Robert Laughlin, "A Different Universe", 2006