What is chemical intuition?

Image source: Buchi.com

Recently I read a comment by a leading chemist in which he said that in chemistry, intuition is much more important than in physics. This is a curious comment since intuition is one of those things which is hard to define but which most people who play the game appreciate when they see it. It is undoubtedly important in any scientific discipline and certainly so in physics; Einstein and Feynman for instance were regarded as the outstanding intuitionists of their age, men whose grasp of physical reality largely unaided by mathematical analysis was unmatched. Yet it seems to me that "chemical intuition" is a phrase which you hear much more than "physical intuition". When it comes to intuition, chemists seem to be more in the league of financial traders, geopolitical experts and psychologists than physicists.

 

Why is this the case? The simple reason is that in chemistry, unlike physics, armchair mathematical manipulation and theorizing can take you only so far. Most chemical systems are too complex for the kind of first-principles approaches that yield predictions in physics to an uncanny degree of accuracy; the same is true of biology. While armchair speculation and order-of-magnitude calculations can certainly be very valuable, no chemist can design a zeolite, predict the ultimate product of a complex polymer synthesis or list the biological properties that a potential drug can have by simply working through the math. As the great organic chemist R B Woodward once said of his decision to pursue chemistry rather than math, in chemistry, ideas have to answer to reality. Chemistry much more than physics is an experimental science built on a foundation of rigorous and empirical models, and as the statistician George Box once memorably quipped, all models are wrong, but some are useful. It is chemical intuition that can separate the good models from the bad ones.

How then, to acquire chemical intuition? All chemists crave intuition, few have it. It's hard to define it, but I think a good definition would be that of a quality that lets one skip a lot of the details and get to the essential result, often one that is counter intuitive. That definition reminds me of a recent book by the philosopher Daniel Dennett in which he describes an intellectual device called an "intuition pump". An "intuition pump" is essentially a shortcut - anything from a linguistic trick to a thought experiment - that allows one to skirt the usual process of rigorous and methodical analysis and get to the point. A lot of chemical thinking involves the fine art of manipulating intuition pumps. It is the art of asking the simple, decisive question that gets to the heart of the matter. As in a novel mathematical proof, a moment of chemical intuition commands an element of surprise. And as with a truly ingenious mathematical derivation, it should ideally lead us to smack our foreheads and ask why we could not think of something so simple before.

Ultimately when it comes to harnessing intuition, there can be no substitute for experience. Yet the masters of the art in the last fifty years have imparted valuable lessons on how to acquire it. Here are three that I have noticed, and I would think they would apply as much to other disciplines as to chemistry.

1. Don't ignore the obvious: One of the most striking features of chemistry as a science is that very palpable properties like color, smell, taste and elemental state are directly connected to molecular structure. For instance, there is an unforgettably direct connection between the smell of a simple molecule called cis-3-hexenol and that of freshly cut grass. Once you smell both separately it is virtually impossible to forget the connection. Chemists who are known for their intuition never lose sight of these simple molecular properties, and they use them as disarming filters that can cut through the complex calculations and the multimillion dollar chemical analysis.

Colors, smells and explosions are what often attract budding chemists to their trade at an early age, and these qualities are also precisely the ones which can be an important elements of chemical intuition. I remember an anecdote about the Caltech chemist Harry Gray (an expert among other things on colored chemical compounds) who once deflated the predictions of some sophisticated quantum mechanics calculation by simply asking what the color of the proposed compound was; apparently there was no way the calculations could have been right if the compound had a particular color. As you immerse yourself in laborious compound characterization, computational modeling and statistical significance, don't forget what you can taste, touch, smell and see. As Pink Floyd said, this is all that your world will ever be.

2. Get a feel for energetics: The essence of chemistry can be boiled down to a fight: a fight unto death of countless factors that rally either for or against the amount of useful energy - technically called the free energy - that a system can provide. In one sense all of chemistry is one big multivariable optimization problem. When you are designing molecules as anticancer agents, for hydrogen storage or solar energy conversion or as enzyme mimics, ultimately what decides whether they will work or not is energetics, how well they can stabilize and be stabilized and ultimately lower the free energy of the system. Intimate familiarity with numbers can help in these cases. Get a feel for the rough contributions made by hydrogen bonds, electrostatics, steric effects and solvent influences, essentially all the important interactions between molecules that dictate the fate of chemical systems. Often the key to improving the properties of molecules is to figure out what single interaction or combination of interactions is responsible for a particular property; you can then tweak that property by turning the knobs on the relevant factors.

Order of magnitude calculations and rough guesses are especially important for chemists working at the interface of chemistry and biology; remember, life is a game played within a 3 kcal/mol window and any insight that allows you to nail down numbers within this window can only help. Linus Pauling was lying in bed with a cold when he managed to build accurate models of protein structure, largely based on his unmatched feel for such numbers that allowed him to make educated guesses about bond lengths and angle. And every chemist can learn from the incomparable intuition of Enrico Fermi who tossed pieces of paper in the air when the first atomic bomb went off, and used the distance at which they fell to calculate a crude estimate of the yield.

A striking case of insights acquired through thinking about energetics is illustrated by a story that the Nobel Prize winning chemist Roald Hoffmann narrates in an issue of the magazine "American Scientist". Hoffmann was theoretically investigating the conversion of graphene to graphane, which is the saturated counterpart of graphene (one in which all double bonds have been converted to single ones), under high pressure. After having done some high-level calculations, his student came into his office and communicated a very counter-intuitive result; apparently graphane was more stable than the equivalent number of benzenes. This was highly counterintuitive since every chemistry student learns that so-called aromatic compounds with alternating double bonds are more stable that their single-bond analogs because of the ubiquitous phenomenon of resonance. Hoffmann could not believe the result and his first reaction was to suspect that something must be wrong with the calculation.

Then, as he himself recalls, he leaned back in his chair, closed his eyes and brought half a century's store of chemical intuition to bear on the problem. Ultimately after all the book-keeping had been done, it turned out that the result was a simple consequence of energetics; the energy gained in the formation of strong carbon-carbon bonds more than offset that incurred due to the loss of aromaticity. The fact that it took a Nobel Laureate some time to work out the result is not in any way a criticism but a resounding validation of thinking in terms of simple energetics. Chemistry is full of surprises- even for Roald Hoffmann- and that's what makes it endlessly exciting.

3. Stay in touch with the basics, and learn from other fields: This is a lesson that is often iterated but seldom practiced. An old professor of mine used to recommend flipping open an elementary chemistry textbook every day to a random page and reading a few pages from it. Sometimes our research becomes so specialized and we become so enamored of our little corner of the chemical world that we forget the big picture. Part of the lessons cited above simply involves not missing the forest for the trees and always thinking of basic principles of structure and reactivity in the bigger sense.

This also involves keeping in touch with other fields of chemistry since an organic chemist never knows when a basic fact from his college inorganic chemistry textbook will come in handy. Most great chemists who were masters of chemical intuition could seamlessly transition their thoughts between different subfields of their science. This lesson is especially important in today's age when specialization has become so intense that it can sometimes lead to condescension toward fields other than your own. A corollary of learning from other fields is collaboration; what you don't have you can at least partially borrow. As Oppenheimer used to say about afternoon tea when he was director of the Institute for Advanced Study, "Tea is where we explain to each other what we don't understand". Chemists and scientists in general need to have tea more often.

Ultimately if we want to develop chemical intuition, it is worth remembering that all our favorite molecules, whether solar energy catalysts, cancer drugs or fertilizers, are all part of the same chemical universe, obeying the same rules even if in diverse contexts. Ultimately, no matter what kind of molecule we are interrogating, "Wir sind alle chemikers", every single one of us.

This is a revised version of an older post. 

A bond by any other name...: How the simple definition of a hydrogen bond gives us a glimpse into the heart of chemistry

Basic hydrogen bonding between two water molecules,
with the central hydrogen shared between two oxygens

A few years ago, a committee organized by the International Union of Pure and Applied Chemistry (IUPAC) - the international body of chemists that defines chemical terms and guides the lexicon of the field - met to debate and finalize the precise definition of a hydrogen bond.

Defining a hydrogen bond in the year 2011? That was odd to say the least. Hydrogen bonds have been known for at least seventy years now. It was in the 1940s that Linus Pauling defined them as foundational elements of protein structure; the glue that holds molecules including life-giving biological molecules like proteins and DNA together. Water molecules form hydrogen bonds with each other, and this feature accounts for water's unique properties. Whether it's sculpting the shape and form of DNA, governing the properties of materials or orchestrating the delicate dance of biochemistry performed by enzymes, these interactions are essential. Simply put, without hydrogen bonds, life would be impossible. No wonder that they have been extensively studied in hundreds of thousands of molecular structures since Pauling highlighted them in the 1940s. Today no chemistry textbook would be considered legitimate without a section on hydrogen bonding. The concept of a hydrogen bond has become as familiar and important to a chemist as the concept of an electromagnetic wave or pressure is to a physicist. 

What the devil, then, were chemists doing defining them in 2011?

It turns out that behind the story of defining hydrogen bonds lies a paradigm that takes us into the very guts of the nature and philosophy of chemistry. It leads us to ask what a chemical bond is in the first place; drill down deep enough into it, and it starts appearing like one of those existential questions a Zen monk would ask about life or the soul.

Remarkably enough, in spite of the debate and redefinitions, the basic features of a hydrogen bond are not controversial. A hydrogen bond is a bond that exists between a hydrogen and two other non-hydrogen atoms. Those two other atoms are most commonly oxygen and nitrogen, but that's where the consensus seems to end and the endless debate begins.

The basic reason for the debate has to do with the very definition of a bond as laid out by pioneering chemists like Gilbert Newton Lewis, Irving Langmuir and Linus Pauling in the 1920s. Loosely speaking a bond is a force of attraction between two atoms. It was Lewis who made the groundbreaking suggestion that a chemical bond results from the donation, acceptance or sharing of electrons. Pauling codified this definition into a rigorous principle using the laws of quantum mechanics. He showed that using quantum mechanics, you could prove that the sharing of electrons between two atoms leads to a net lowering of energy between them: this sounds logical since if there were no lowering of energy, there would be no incentive for two atoms to come close to each other.

So far so good. Everyone accepts this basic fact about chemical bonds. Treatises have been written about the various bonds between different atoms seen in a bewildering array of metallic, organic and inorganic molecules; Pauling's seminal book, "The Nature of the Chemical Bond", described such interactions in loving detail. In this book he showed that bonds can be covalent or ionic; the former involve a symmetric sharing of electrons while the latter involve an asymmetric give and take. There are also many intermediate cases. Furthermore, whether a bond is ionic or covalent depends on the electronegativity of the atoms involved. Electronegativity is the innate property of an atom to hold on closely to its electrons. Fluorine for instance is the most electronegative element of all, holding on to its electrons in a fiery embrace. Metals on the other hand are happy to give away their electrons. Not surprisingly, bonds between metals and fluorine are readily formed.

It is in the context of electronegativity that it's easy to understand hydrogen bonds. A hydrogen atom, just like a metal, is happy to give away its lone electron and form a positive ion. Oxygen and nitrogen atoms are happy to accept this electron. Thus, a hydrogen bond forms when a hydrogen atom is shared between two oxygen atoms (or one oxygen and one nitrogen, or two nitrogens). Because of the difference in electronegativity, the hydrogen acquires a slightly positive partial charge, and the other atoms acquire a slightly negative partial charge (represented in the picture above). Think of two oxygen atoms with a hydrogen between them acting as a bridge: that's a hydrogen bond between them. Because a hydrogen atom is small and has only a single electron it can achieve the feat of forming this bridge; what is fascinating is that this is partly the result of a unique quantum mechanical phenomenon called the tunnel effect.

All this is rather uncontroversial. The debate starts when we ask what criteria we use to actually identify such hydrogen bonds. Chemists usually identify bonds by looking at experimental structures of molecules. The most common method of acquiring these structures is by x-ray diffraction. Bouncing x-rays off a crystallized molecule allows us to accomplish the molecular equivalent of taking a photograph. The molecules under question can be anything from common salt to complicated proteins. These structures form a cornerstone of chemistry, and they are not just of academic importance but have also been immensely useful in the design of new drugs and materials.

When chemists acquire the crystal structure of a molecule, they will usually do something very simple: they will measure the distance between various atoms. The hallmark of a chemical bond is a short distance between two atoms; think of it as two people embracing each other and reducing the distance between themselves. How short, exactly? Short enough to be less than the sum of Van der Waals radii of the two atoms. Imagine the Van der Waals radius of an atom as a kind of safe perimeter which an atom uses to hold other atoms at bay. Each atom has a unique Van der Waals radius defining its safe perimeter which is measured in the unit angstroms (1 Å = 10^-10 meters). Hydrogen for instance has a radius of 1.2 Å, nitrogen has a radius of 1.5 Å. If a chemist measures the distance between a hydrogen and nitrogen in a crystal structure as being less than 2.7 Å (1.2 + 1.5), she would usually declare that there's a bond between the two atoms. If the three atoms in a hydrogen bond satisfy this geometric criteria, then one could legitimately claim a bond between them.

But this simple procedure turns out to be far trickier to accept than we can imagine. Here's the problem: a crystallized molecule is a complex assembly of hundreds or even thousands of atoms, all of which are jostling for space. The eventual formation of the crystal is not a zero-sum game; some atoms have to give way so that others can be happy. As the pioneering chemist Jack Dunitz has succinctly noted, "Atoms have to go somewhere". Thus, some atoms are simply forced together against their will, just like people in a cramped train compartment who are forced together against their will. Now, just like those cramped people, the distance between two such cramped atoms will be unnaturally short, often shorter than the the sum of their Van der Waals radii. The key term here is "unnaturally": the distance is short not because the atoms actually want to be close, but because they are forced to be so. Can one then say that there is a bond between them? And how short does this distance need to be in order to be called a bond?

There's another conundrum that emerged as thousands of crystal structures of molecules started to appear during the last few decades. While the "classical" hydrogen atoms bridging oxygens and nitrogens were well known, short distances also started appearing between hydrogen and atoms like carbon and chlorine. The purported hydrogen bond zoo started to get populated with even more exotic creatures. For instance chemists started noticing close contacts between hydrogens and the flat face of benzene rings: these hydrogens are attracted to the ghostly electron clouds lying on top of the rings. Sometimes you did not even need a benzene ring; two doubly bonded carbon atoms would appear as magnets for hydrogens and pull them close. And these hydrogens in turn again would not just be bonded to oxygens or nitrogens as their second partners; they could be bonded to carbons, or even to metallic elements. Experimental observation forced the concept of hydrogen bonds to be extended not just to other bonafide atoms but to more abstract entities like clouds of electrons.

An illustration of two kinds of hydrogen bonds:
conventional ones with oxygen atoms in blue
and those with a benzene ring in red

Two overriding questions thus emerged from these decades-long exercise of observation and analysis: Can one claim a bond between two atoms in a molecule - specifically a hydrogen bond in case of hydrogen, oxygen or nitrogen - simply based on the distance between them? And what atoms exactly constitute a hydrogen bond?

The IUPAC committee in 2011 seems to have exorcised both demons with a single stroke. They defined a hydrogen bond as any kind of bond between a hydrogen and two other atoms, provided that the other two atoms are more electronegative than hydrogen itself. That always included oxygen and nitrogen, but now the definition was expanded to include carbon and halogens (chlorine, fluorine, bromine and iodine) as well as ghostly entities like electron clouds on top of benzene rings. As far as the distance criterion was concerned, IUPAC made a wise decision in allowing it to be fairly variable and not simply limited to a hard cutoff within the sum of Van der Waals radii.

The IUPAC decision illuminates three very important aspects of chemistry. Firstly, it tells us that many chemical concepts are fuzzy and only approximately defined. For instance, although we have defined the distance between the hydrogen and other two atoms in a hydrogen bond as being less than the sum of their Van der Waals radii, in reality these distances are best derived as a statistical distribution. Some hydrogen bonds are very short, some are very long, and most are in the middle: even this classification has been a hotbed of contention, and you will find arguments about the nature and energies of "short" and "weak" hydrogen bonds in the scientific literature that approach the vigor of gladiatorial contests. The ones that are short usually involve nitrogen and oxygen, while the long ones involve the elements included in the expansive IUPAC definition. This fuzzy reality extends to other concepts in chemistry such as aromaticity, polarizability and acid and base behavior. Unlike concepts in physics like wavelength and electrical conductivity, concepts in chemistry can be much more malleable. However this malleability is not a bug but a feature, since it allows you to capture a vast amount of chemical diversity in a central idea.

The second key aspect of chemistry that the IUPAC statement reveals is the fact that scientific concepts don't have to always be rigorously defined in order to be useful. The definition of a hydrogen bond goes to the heart of chemistry as a science based on models. A hydrogen bond is a bond, but it's really a model based on our ideas of the myriad ways in which molecules interact. The key feature of a model is utility, not reality. In my own work for instance, I can recognize and exploit hydrogen bonds between proteins and drugs and improve their directionality and strength, all without worrying one bit about their precise, rigorous definition. Similarly polymer chemists can create new forms of hydrogen-bonded plastics with novel functions without caring much whether their hydrogen bonds conform to some well-defined ideal. Whatever the debate about hydrogen bonds may be, my view of them is similar to Potter Stewart's: I know them when I see them. Or, if I wanted to offer a cheesy but still quotable line from one of "The Matrix" movies, a hydrogen bond is merely a phrase; what matters is the connection it implies.

Ultimately, this fuzzy yet useful view of hydrogen bonding makes an important statement about the philosophy of chemistry. It tells us that chemistry has its own philosophy based on models and utility that is independent of its roots in physics. As the chemist Roald Hoffmann memorably put it, chemical concepts like aromaticity and electronegativity start to "wilt at their edges" when examined too closely, and Hoffmann could have been talking about the hydrogen bond there. The physics-based view of a hydrogen bond would involve writing down the Schrodinger equation for these bonds and rigorously solving it to obtain quantities like the energy and geometry of the bond. But that exercise is too reductionist; it does not actually help me understand the nature and variety of these bonds. It won't tell me why water is liquid and why it flows from mountains, why it's a universal solvent and why it's an existential life-giver. Chemistry proclaims its own emergent language and philosophy, and while the material entities of which molecules are composed are grounded in physics, the molecules themselves belong squarely in the domain of chemistry.

The fact that chemists are still debating hydrogen bonds means that chemistry remains an exciting and vibrant discipline, one in which even basic concepts are still being sculpted and fine-tuned. It tells us that for the foreseeable future chemistry will continue to be a science without limits, a country with open borders. The hydrogen bond should give chemists an opportunity to celebrate the very soul of their work.

Nobelist Roger Kornberg: "If there is any one subject an educated person should know, that is chemistry"

Roger Kornberg received the 2006 Nobel Prize in Chemistry for figuring out the atomic-level details of the process of transcription, that foundational aspect of life that frees up the message in DNA and allows it to work its magic through messenger RNA. 

Following the announcement of the prize, Kornberg had a remarkable exchange with an interviewer in Stockholm when the interviewer asked him whether the work he was doing could rightly be classified as 'structural biology' or something else. In the process he offered a supreme paean to chemistry as the central science.

“The fact remains as I said that the boundaries have blurred, but what is more is that the work that we do in the structural biology department is in some places done in departments of chemistry and in other places done in departments of biology.  

Beyond that I might add that in my personal view chemistry is really the queen of the sciences; chemistry is the common ground for all scientific investigations. So our best hope of applying physical principles for the world around us is at the level of chemistry. Our best hope of understanding the biological organism and ultimately the form and function of the human body is at the level of chemistry. I have said before and I would repeat that if there is any one subject that an educated person should know in the world that is chemistry.”

As if this glowing praise was not enough, Kornberg further elaborated on his view of the necessity of chemistry when the interviewer asked him about what kind of preparation he expects potential students and postdocs in his lab to have.

"I have a very straightforward answer to your question. The one thing I look for is chemistry. People come to my lab from a background in biology, they come from a background in physics, and the question I always ask is about their training in chemistry; if they have studied chemistry then what chemistry did they study and how well did they do it. 

If someone comes along who has studied physical and organic chemistry and have been good at it, then I know they will fit in well, that they will succeed in what we do…absent that I’m doubtful…if they’re a biologist they must have done chemistry, if they’re a physicist then they must at least show an aptitude for chemistry."

Kornberg's superlative leanings toward chemistry may seem excessive but he makes an excellent point. His statement that our best hope of applying physical principles for the world around us is at the level of chemistry succinctly captures the emergent nature of chemical science that frees it from its underpinnings in physics. DNA is composed of quarks, but you cannot understand its principal properties like base pairing and hydrogen bonding at the level of quarks. You can only understand them - both philosophically and practically - only at the molecular level, only at the level of molecular orbitals and electronegativity.

The central science of chemistry reigns over biology, over applied physics, over astronomy, over engineering, but it often works its subtle but powerful influence through the shadows. Like the spacetime of general relativity it simply is everywhere.