The fundamental philosophical dilemma of chemistry

The classic potential energy curve of chemistry
hides a fundamental truth: bonds mean short distances,
but short distances don't mean bonds

Every field has its set of great philosophical dilemmas. For physics it may be the origin of the fundamental constants of nature, for biology it might be the generation of complexity by random processes. Just like physics and biology chemistry operates on both grand and local scales, but the scope of its fundamental philosophical dilemmas sometimes manifests itself in the simplest of observations.

For me the greatest philosophical dilemma in chemistry is the following: It is the near impossibility of doing controlled experiments on the molecular level. Other fields also suffer from this problem, but I am constantly struck by how directly one encounters it in chemistry.

Let me provide some background here. Much of chemistry is about understanding the fundamental forces that operate within and between molecules. These forces come in different flavors: strong covalent bonds (dictated by the sharing of electrons), hydrogen bonds (dictated by weak electrostatic interactions), strong charge-charge interactions (dictated by attraction between unlike charges), hydrophobic effects (dictated by the interaction between 'water-loving' and 'water-hating' parts of molecules) etc. The net interaction or repulsion between two molecules results from the sum total of these forces, some of which may be attractive and others might be repulsive. Harness these forces and you can control the structure, function and properties of molecules ranging from those used for solar capture to those used as breakthrough anticancer drugs.

Here’s how the fundamental dilemma manifests itself in the control of all these interactions: it is next to impossible to perform controlled experiments that would allow one to methodically vary one of the interactions and see its effect on the overall behavior of the molecule. In a nutshell, the interactions are all correlated, sometimes intimately so, and it can be impossible to change one without changing the other.

The fundamental dilemma is evident in many simple applications of chemistry. For instance, my day job involves looking at the crystal structures of proteins involved in disease and then designing small organic molecules which bind to and block such proteins. For binding to their target protein, these small molecules exploit many different interactions including hydrogen bonds, charge-charge interactions and hydrophobic effects to bring about a net lowering of their interaction energy with the protein. The lower this interaction or "free energy" the better the interaction. Unfortunately, while one can visualize the geometry of the various interactions by simply looking at the crystal structure, it is very difficult to say anything about their energies, for to do so would entail varying an interaction individually and looking at its effects on the net energy. Crystal structures thus can be very misleading when it comes to making a statement about how tightly a small molecule binds to a protein.

Let’s say I am interested in knowing how important a particular hydrogen bond in the small molecule is. What I could do would be to replace the atoms comprising the hydrogen bond with non hydrogen-bonding atoms and then look at the change in the affinity of the resulting molecule for the protein, either computationally or experimentally. Unfortunately this change also impacts other properties of the molecules; its molecular weight, its hydrophobicity, its steric or spatial interactions with other molecules. Thus, changing a hydrogen bonding interaction also changes other interactions, so how can we then be sure that any change in the binding affinity came only from the loss of the hydrogen bond? The matter gets worse when we realize that we can’t even do this experimentally; in my colleague Peter Kenny’s words, an individual interaction between molecules such as a hydrogen bond is not really an experimental observable. What you see in an experiment is only the sum total, not the dissection into individual parts.

There have of course been studies on ‘model systems’ where the number of working parts is far less than those in protein-bound small molecules, and from these model systems we have gotten a good sense of the energies of typical hydrogen bonds, but how reliably can we extend the results of these systems to the particular complex system that we are studying? Some of that extrapolation has to be a matter of faith. Also, model systems usually provide a ranges of energies rather than a single value and we know that even a tiny change in the energy of binding can correspond to a substantial loss of effective blocking of a protein, so the margin of error entrusted to us is slim indeed.

It is therefore very hard, if not impossible, to pin down a change in binding affinity resulting from a single kind of interaction with any certainty, because changing a single interaction potentially changes all interactions; it is impossible to perform the truly controlled experiment, a concept which has been at the heart of the scientific method. Sometimes these changes in other interactions can be tiny and we may get lucky, but the tragedy is that we can’t even calculate with the kind of accuracy we would like, what these tiny increments or reductions might be. The total perturbation of a molecule’s various interactions remains a known unknown.

The roots of the problem run even deeper. At the most elemental level, all interactions between molecules are simply a function of one of the four fundamental forces known in nature - the electromagnetic force. Of the four basic forces, gravity is too weak to play a role, while the strong and weak nuclear forces don't usually apply to molecular interactions since such interactions only involve the sharing of electrons. It is the electromagnetic force that is thus ascendant in mediating every single molecular interaction in the universe. When we divide this force up into hydrogen bonds, electrostatic interactions, hydrophobic interactions etc. what we are doing is imposing an artificial division on an indivisible fundamental force, purely for our convenience. It's a bit like the parable of the blind men and the elephant - there is only one electromagnetic force, just like there is only one elephant, but each of us describing that force divides it up into multiple flavors. No wonder then that we are led astray when we think we are doing a controlled experiment, since whenever we think we are varying one flavor or another we are actually varying the same basic parameter and not its independent components. That is because there are no independent components in the true sense of the term.

This inability to perform the truly controlled experiment is thus what I call the great philosophical dilemma of chemistry. The dilemma not only makes the practical estimation of individual interactions very hard but it leads to something even more damning: the ability to even call an interaction an 'interaction' or 'bond' in the first place. This point was recently driven home to me through an essay penned by one of the grand old men of chemistry and crystallography – Jack Dunitz. Dunitz’s point in the essay is that we are often misled by ‘short’ distances between atoms observed in crystal structures. We ascribe these distances to ‘attractive interactions’ and even ‘bonds’ when there is little evidence that these distances are actually attractive.

Let’s backtrack a bit to fundamentals. The idea of ascribing a short distance to an attractive interaction comes from the classic van der Waals potential energy curve (figure above) that is familiar to anyone who has taken a college chemistry class. The minimum of this curve corresponds to both the shortest distance (called the van der Waals distance) between two molecules and the lowest energy, typically taken to signify a bond. However this leads to a false equivalence that seems to flow both ways: van der Waals distances correspond to bonds and bonds correspond to van der Waals distances.

In reality the connection only flows one way. Bonds do correspond to short distances but short distances do not necessarily correspond to bonds. So then why do we observe short distances in molecules in the first place? Again, Dunitz said it very succinctly in a previous review: simply because ‘Atoms have to go somewhere’. The fact is that a crystal structure is the net result of a complex symphony of attractive and repulsive interactions, a game of energetic musical chairs if you will. At the end, when the dust has settled everyone has to find a chair, even if it means that two people might end up uncomfortably seated on the same chair. Thus, when you see a short distance between two atoms in a crystal, it does not mean at all that the interaction between them is attractive. It could simply mean that other interactions between other atoms are attractive and that those two atoms have simply then settled where they find a place, even if the interaction between them may be repulsive. 

The message here is clear: it is folly to describe an interaction as ‘attractive’ simply because the distance is short. This applies especially to weaker interactions like those between aromatic (benzene) rings. I am always wary when I see a benzene ring from a small molecule nicely sandwiched between another benzene ring in a protein and hear the short distance between the two described as a ‘stacking interaction’. Does that mean there is actually an attractive stacking interaction between the two? Perhaps, but maybe it means simply that there was no other place for the benzene ring to be. How could I test my hypothesis? Well, I know that varying the substituents or groups of atoms attached to benzene rings is known to vary their energies of interaction with other benzene rings. So I ask the chemist to make some substituted versions of that benzene ring. But hold on! Based on the previous discussion, I just remembered that varying the substituents is not going to just change the stacking energy; it’s also going to change other qualities of the ring that mess up the other interactions in the system. It’s that problem with performing controlled experiments all over again - welcome to the fundamental dilemma of chemistry.

The fundamental dilemma is why it is so hard to understand individual interactions in chemical systems, let alone exploit them for scientific or commercial gain. We see it in a myriad of chemical experiments, from investigating the effects of structural changes on the rates of simple chemical reactions to investigating the effects of structural changes on the metabolism of a drug. We can’t change one component without changing every other. There may be cases where these other changes might be minuscule, but in reality the belief that they may be minuscule in a particular case will always remain a matter of faith than of fact.

The fundamental dilemma then is why drug design, materials designs and every other kind of molecular design in chemistry is so tricky. It is why so much of complicated chemistry is still trial and error, why observations on one system cannot be easily extrapolated to another, and why even supercomputers are not yet able to nail down the precise balance of forces that dictate the structure and function of specific molecules. In a nutshell, the fundamental dilemma is why chemists are always ignorant and why chemistry will therefore always be endlessly fascinating.

Chemical and Engineering News celebrates 90 years: How chemistry has come a long way

Chemistry is - in the true sense - the central science, reaching inside every aspect of our lives (Image: Marquette University)

Chemical and Engineering News (C&EN) is celebrating 90 years of its existence this year, and I can only imagine how perplexed and awestruck its editors from 1923 would have been had they witnessed the state of pure and applied chemistry in 2013. I still remember devouring the articles published in the magazine during its 75th anniversary, and this anniversary also offers some tasty perspectives on a diverse smattering of topics; catalysis, structural biology and computational chemistry to name a few. 

There's an article in the magazine documenting how the single-most important concept in chemistry - that of the chemical bond - has undergone a transformation; from fuzzy, to rigorously defined, to fuzzy again (although in a very different sense).

Nobel Laureate Roald Hoffmann had something characteristically insightful to say about The Bond:

"My advice is this: Push the concept to its limits. Be aware of the different experimental and theoretical measures out there. Accept that at the limits a bond will be a bond by some criteria, maybe not others. Respect chemical tradition, relax, and instead of wringing your hands about how terrible it is that this concept cannot be unambiguously defined, have fun with the fuzzy richness of the idea.”

In a bigger sense the change in chemistry during these 90 years has been no less than astounding. In 1923 the chemical industry already made up the foundations of a great deal of daily life, but there was little understanding of how to use the concepts and products of chemical science in a rational manner. Since 1923 our knowledge of both the most important aspect of pure chemistry (the chemical bond) and of applied chemistry (synthesis) has grown beyond the wildest dreams of chemistry's founders.

If we had to pinpoint two developments in chemistry during these 90 years that would truly be described as "paradigm shifts", they would be the theoretical understanding of bonding and the revolution in instrumental analysis. As I and others have argued before, chemistry unlike physics is more "Galisonian" than "Kuhnian", relying as much on new instrumental techniques as on conceptual leaps for its signal achievements.

The two most important experimental advances in chemistry - x-ray diffraction and nuclear magnetic resonance - both came from physics, but it was chemists who honed these concepts into a routine laboratory tool for the structure determination of a staggeringly diverse array of substances, from table salt to theribosome. The impact of these two developments on chemistry, biology, medicine and materials science cannot be underestimated; they cut down the painstaking task of molecular structure determination from months to hours, they allowed us to find out the nature of novel drugs, plastics and textiles and they are now used by every graduate student every single day to probe the structure of matter and synthesize new forms of it. Other developments like infrared spectroscopy, electron diffraction, atomic force microscopy and single molecule spectroscopy are taking chemistry in novel directions.

The most important theoretical development in chemistry also derived from physics, but its progress against demonstrates chemists' central role in acting as mediators between concept and application. It also serves to make a key point about reductionism and the drawbacks of trying to reduce chemistry to physics. The chemical bond is an abstract concept going back to "affinities" between atoms (which when illustrated were replete with hooks and eyes). But it was in 1923 that the great American chemist G. N. Lewis propounded the idea in terms of atoms sharing electrons. This was a revolutionary brainwave and illuminated the way for Linus Pauling, John Slater, Robert Mulliken, John Pople and others to use the newly developed machinery of quantum mechanics to fashion the qualitative principle into an accurate, quantitative tool which  - with the development of modern computing - now allows chemists to routinely calculate and predict important properties for any number of chemical substances.

Yet the ramifications of the chemical bond tempt and beguile physicists and constantly escape from their grasp when they try to define them too accurately. The above quote by Roald Hoffmann puts the problem in perspective; quintessentially chemical ideas like aromaticity, the hydrophobic effect, steric effects and polarity "fray at the edges" (in Hoffmann's words) when you try to push them to their limits and try to define them in terms of subatomic physics. Chemistry is a great example of an emergent discipline. It is derived from physics and yet independent of it, relying on fundamental definitions at its own level when progressing.

The chemical bond and other theoretical aspects of chemistry have enabled the rise of the one activity pursued by chemists of which society is an unsurpassed beneficiary - the science, art and commerce of synthesis. Every single molecule that bathes, clothes, feeds, warms, transports and heals us has been either derived from nature using chemical techniques or has been synthetically made in a chemical laboratory. The social impact of these substances is hard to underestimate; even a sampling of a few such as the contraceptive pill, antibiotics or nylon attests to the awesome power of chemistry to completely transform our lives.

In 1923 synthesis was a haphazard process and there was virtually no understanding of how we could do it rationally. All of this changed in the 1950s and 60s when a group of pioneering scientists led by the legendary organic chemist Robert Burns Woodward revolutionized the process and honed synthesis into a precisely rational science which took advantage of the course of chemical reactions, the alignment of orbitals, the development of new chemical reagents and the three-dimensional shape of molecules. Many Nobel Prizes were handed out for these groundbreaking discoveries, but none surpassed the sheer impact that synthesis will continue to have on our way of life.

As is inevitably the case for our embrace of science and technology, with progress also come problems, and chemists have had to deal with their share of issues like environmental pollution, drug side effects and the public perception of chemistry. Suffice it to say that most chemists are well aware of these and are working hard to address them. They recognize that with knowledge comes responsibility, and the responsibility they bear to mitigate the ills of the wrongful application of their science transcends their narrow professional interests and encompasses their duties as citizens.

In the new century chemistry continues to build upon its past and chemists continue to push its boundaries. Another change which the editors of C&EN would not have foreseen in 1923 is the complete integration of chemistry into other disciplines like biology, medicine and engineering and its coming into its own as the true "central science". Today chemistry deeply reaches into every single aspect of our lives. The cardinal problems facing civilization - clean and abundant food and water, healthcare, national security, overpopulation, poverty, climate change and energy - cannot be solved without a knowledge of chemistry. Simply put, a world without chemistry would be a world which we cannot imagine, and we should all welcome and integrate the growth of chemical science into our material and moral worldview.

First published on the Scientific American Blog Network.

What is your favorite deep, elegant or beautiful explanation (in chemistry)?

Over at Edge, they have a survey asking leading scientists, thinkers and writers about what they think is their favorite "elegant, deep or beautiful explanation". This is meant to be a very general question, not even limited to science and includes responses from people as diverse as the economist Richard Thaler, complexity theorist Stuart Kauffman and Stewart Brand (founder of the Whole Earth Catalog). The answers include ideas, explanations, experiments and entities as general and diverse as the scientific method, genes, Pascal's wager, bounded rationality, relativity and the limits of intuition.

The explanations run across the gamut of the sciences and the humanities including physics, biology, economics, neuroscience, politics and business. But conspicuously absent is chemistry, except for a few peripheral references like Charles Simonyi's listing of Besicovitch's theory of atomic forces. And this is in spite of our friend Derek Lowe of "In the Pipeline" being included in this august list. I was gratified to see a chemist being asked for his opinion, and was somewhat disappointed that Derek's favorite explanation was not chemical (his favorite is the rather deceptively simple notion of "freefall"). I of course don't blame Derek for his choice since there is no law dictating that a chemist's favorite explanation should be from chemistry just because he or she is a chemist. My own favorite beautiful explanation is probably Cantor's notion of multiple infinities.

But I did regret the striking omission of chemistry from the list. Sometime back I had a whole post on elegance in chemistry. And I certainly don't want people to think that deep and elegant explanations are limited to physics and biology, because they are not. Chemistry may not boast of profound philosophical explanatory frameworks like the Big Bang or evolution by natural selection. But it makes up for this fact by creating paradigms that directly touch the lives of millions of human beings in ways much more palpable than the Big Bang and evolution. So I thought I would add my own modest thoughts on my favorite deep idea in chemistry.

There's actually a few things at the top of my list; you certainly don't have to think hard to come up with several foundational chemical ideas. But if you really asked for my absolute favorite deep and elegant explanation, it is the shared-electron chemical bond. That's it. Right there is the simple concept that is at the heart of the material world, a concept that if you think about it has had a staggering impact on our quality of life, our relationships with other nations, our notion of prosperity itself. Chemical bonds as manifested in the foundations of modern civilization have certainly contributed as much to life, liberty and the pursuit of happiness as any scientific idea.

The idea itself as formulated by the great Gilbert Newton Lewis and comprehensible to any high-school student is simplicity incarnated; atoms combine into molecules and form a bond when electrons are shared. Everything that comes after the stating of this fact, important as it is, is details. All the quantum chemical wizardry, the thinking-in-orbitals, the great Gaussian simplification, it's after this basic groundwork has been laid. Heitler and London, Pauling, Slater, Mulliken, Pople, all of them made critical contributions to chemical bonding, but they all stood on Lewis's shoulders and built up from his landscape of the shared electron chemical bond.

Given the absolutely foundational role that the chemical bond plays in the thinking of chemists, it may be both ironic and a tad disturbing that chemists still cannot completely agree on the precise definition of every molecular bond out there. But that's not because the basic framework underlying bonding is uncertain. Part of the reason is simply because there is no such thing as "the" chemical bond. The bonding zoo sports a bewildering variety of animals, from the upstanding "normal" chemical bonds in, say the hydrogen or methane molecules, to the (literally) ready-to-snap pressure cooker entities in strained organic compounds, from the wily, shape-shifting bonds between metals and organic compounds to the ephemeral but biologically vital hydrogen bonds. Although the basic theory of the chemical bond is securely in place, it's going to take some time to craft a net wide and yet rigorous enough to snare the unruly and colorful creatures dotting the chemical landscape.

Now physicists may try to appropriate the chemical bond as their own, but they are out of luck. No explanation based purely on physics can truly impart a feel for the sheer diversity of bonds quoted above and their context-specific personalities. Just one bond serves to create a nightmare for purely reductionist approaches to defining chemical bonding- the hydrogen bond. Last year chemists convened at a meeting with the express purpose of tweaking their description of this all-important biological mediator, the glue that holds life together. Several questions were bandied about, but none more important than the very definition of a hydrogen bond. The problem was simple; hydrogen bonds can be weak or strong, sometimes so weak as to strain the definition of a bond, sometimes strong enough to suspiciously qualify as a covalent bond. How much of hydrogen bonding is electrostatic and how much is covalent? Is "bond" even the right term, or would "bridge" be more accurate? How do you define hydrogen bonds to metals? A consensus was finally reached on a new definition, but not even Linus Pauling could say that the definition would hold for all of eternity. Defining a hydrogen bond would give every physicist out there a run for his money. I find the concept of the chemical bond so enticing and elegant partly because even a single kind of bond like the hydrogen bond can hide a richly textured world of possibilities lurking behind its surface.

So there it is, why the concept of the chemical bond is my favorite idea, certainly in chemistry. It is deep because it underlies the making of the material universe, explaining the stuff that everything from crab shells to the Crab Nebula is made of. It is elegant because of the virtually unlimited amount of explanatory power that it hides in a simple statement of definition. And it is beautiful because of the sheer diversity of materials and structures that are created from a simple law of attraction. A lot of the thinkers in the Edge survey quoted evolution as their favorite deep idea. It certainly is beautiful. But Darwin could well have slightly paraphrased his words to apply to Lewis's shared-electron chemical bond:

"There is grandeur in this view of the material world, with its several powers, having been originally breathed into a single bond; and that, whilst this planet has gone cycling on according to the fixed law of gravity, from so simple a bond endless forms most beautiful and most wonderful have been, and are being, evolved."

Ode to a classic: The Nature of the Chemical Bond

No other chemistry book in the twentieth century influenced the general thinking of chemists more than Linus Pauling's "The Nature of the Chemical Bond". Yet take an opinion poll among undergraduates or graduate students and hardly anyone would have read a single page from this classic. This unfortunate facts only reflects a broader problem with undergraduate education; the relentless urge to teach problem solving at the expense of an appreciation of the essential philosophy of the subject. And very few chemistry books ever published communicate the deep structure of chemical thinking as well as Pauling's memorable volume.

In the latest issue of Nature, Philip Ball visits this lost gem. The book is remarkably and fortunately still in print but it's not really making an impact. My own introduction to "The Nature" was fortuitous. I had of course known about Pauling and his legendary status but never had the chance to actually peruse the volume. Sometime during my senior high school year a friend showed me the book which he had borrowed from his uncle and I was hooked. At first put off because of the extraordinary volume of detail in it, I soon realized that the elegant explanation of this voluminous material through a few simple principles was the crowning achievement of Pauling's thinking. It was all (or at least most) of chemistry through a few good concepts of chemical bonding. Although I did not read every single thing in the book, I read most key parts multiple times and still keep referring to it constantly.

As I noted in an earlier post, the immense impact of the book is driven home by the fact that in the first ten years after publication, it was cited no less than sixteen thousand times. Most of the principles that Pauling developed like resonance, hybridization, electronegativity and hydrogen bonding are now such fundamental parts of chemistry that everyone takes them for granted. With Pauling, chemistry was transformed from a descriptive science to one based on rational notions of bond breaking and formation based on the laws of physics. And yet as Ball describes in his article, the end effect was unmistakably chemical:

"The significance of The Nature of The Chemical Bond was not so much that it pioneered the quantum-mechanical view of bonding, but that it made this a chemical theory: a description that chemists could understand and use, rather than a mathematical account of wave functions. It recognized that, if a model of physical phenomena is to be useful, it needs to accommodate itself to the intuitions and heuristics that enable scientists to talk coherently about the problem. Emerging from the forefront of physics, this was nevertheless a chemists' book."

Physicists' dream is to find five equations that describe the entire universe. Pauling came the closest to doing this for chemistry; no wonder that a poll by Time magazine about the greatest scientists of all time included Pauling as only one among two twentieth century scientists, along with Einstein. His concepts underlie every branch of the science and crucially extend into interdisciplinary branches like biology; it was his insights into chemical bonding that made him one of the founding fathers of molecular biology by way of important ideas on the structures of proteins, enzymes and antibodies. "The Nature" contains scores of examples drawn from physical, organic, inorganic and biological chemistry. The sheer sweep of Pauling's contribution to bonding is astonishing. It would not be an exaggeration to say that his book did for chemistry something like what Darwin's "The Origin" did for biology; it brought all of chemistry under a unifying rubric. And like "The Origin", "The Nature" is one of the very few founding texts of science whose language is simple enough to be understood by beginning students (of course Darwin did one better since his book can be understood easily even by laymen).

Every student of chemistry must be exposed to this foundation of chemistry, yet "The Nature" has been forgotten in colleges and graduate schools. Of course it cannot replace a modern chemistry course, of course the language is somewhat dated, of course chemistry has made exciting advances since Pauling that are not included in the book and of course the valence bond theory described by Pauling has been replaced by molecular orbitals in many important cases. But the book should be required reading for understanding the core philosophy of the subject and how a few simple concepts can explain the astounding variety of the material world around us. It's also a superb vehicle for demonstrating the limitations of the reductionism of physics and the empirical character of chemistry. The goal of a chemistry education is not simply to solve chemical problems, but it is to view the world through a chemical lens. And that means to view the world through the language that we inherited from Pauling.

If you want to think like a chemist, you cannot do better than "The Nature of the Chemical Bond". Buy it, it's actually not that expensive compared to most college and graduate school textbooks.