Chemistry is not harder than other sciences...just different.

A well known physicist turned venture capitalist asked on Twitter the other day why people seem to have a harder time understanding chemistry rather than physics or biology. Chemistry is by no means harder to understand than physics or biology, but it occupies a tricky middle ground between rigor and intuition, between deduction and creation, between creativity and understanding. Understanding it can bring great dividends: Robert Oppenheimer once said that “If you want to get someone interested in science teach them a course on elementary chemistry…unlike physics it gets very quickly to the heart of things.”

Chemistry’s path was partly driven by an impulse to understand the physical world, much like the path of physics and astronomy, but somewhat differently from physics and astronomy, to consciously improve the material conditions of life. What passed for medicine, art, architecture, agriculture and commerce in the ancient world was suffused with chemistry. Whether it was indigo dye for royal textiles, mercury or arsenic for medicine, lime for protecting crops or plaster for holding together stones of medieval stone buildings, the world looked to chemistry, whether consciously or not, to feed, transport, clothe and sustain itself. But this foundational practical role that chemistry played also obscured its philosophy.

The philosophy of chemistry developed in the 18th and 19th centuries through the work of Dalton, Lavoisier, Liebig, Kekule, Mendeleev and other thinkers. Much like biologists had spent their time collecting specimens and systematizing their science before someone like Darwin could make a great theoretical leap, chemists had systematized the vast body of observations that natural philosophers had documented and assimilated over the years. But key questions still remained: Why did water freeze at 0 degrees celsius and expand as it cooled? Why were gallium and mercury liquids? Why was lithium relatively stable while its cousin sodium a fiery, unstable beast? Even Mendeleev’s famed periodic table, after answering the how and what, did not answer the why.

It was only with the advent of atomic physics and quantum theory in the 20th century that these questions started to be answered. Niels Bohr’s atomic model led to the idea of the atom as an entity with a central dense nucleus surrounded by fuzzy probabilistic shells of electrons. Concomitant developments by 19th century chemists that had led to the precise measurements of atomic weights and the elucidation of rules that predicted how elements combine with each other intersected with the basic Bohr atom and the science of spectroscopy to illuminate how different elements were built up with different numbers of electrons and protons (the neutron whose discovery explained isotopes came only in 1932).

It was only after Walter Heitler, Fritz London, Gilbert Newton Lewis and especially Linus Pauling explained how the chemical bond was formed that chemistry truly exploded as a self-contained discipline. By showing how different atoms shared electrons in different ways so that they were held together by a variety of forces – weak dispersion forces and strong electrostatic forces for instance – modern chemistry finally started answering those questions about liquid ice and mercury that had been asked for centuries.

But how was the philosophy of chemistry faring compared to the philosophy of science during this period? Not very well. Firstly, philosophers were more naturally drawn first to physics and then to biology as deductive disciplines for laying out their conception of how science was done. Quantum mechanics especially, with its paradoxes and mysteries, became a fertile ground for philosophers to erect their edifice. Biology with evolution and heredity seemed to go to the heart of human existence and also attracted philosophical theorizing. Somehow chemistry slipped through the fingers of the prominent philosophers, partly because it seemed too practical like engineering (although engineering has its own philosophy) and partly because they simply didn’t get it.

Why? Because chemistry largely defies the traditional philosophy of science as laid down not only in physics and biology but in science in general in the centuries since the competing visions of Baconian and Cartesian science molded the way both scientists and philosophers view the natural world. Francis Bacon said, “All depends on keeping the eye fixed on the facts of nature.” Descartes said, “I think, therefore I am.” Science developed along both these lines and it led to the familiar set of ideas about hypothesis testing, observation, experiment and theorizing, and later in the 20th century, to conjectures and refutations, falsification and paradigm shifts. Most people were comfortable dealing with sciences that seem to at least broadly fit these notions from the philosophy of science.

Chemistry does not neatly fit into these categories every time because it’s more akin to the creative arts of architecture and painting. The Nobel Prize winning chemist, writer and poet Roald Hoffmann asks what hypothesis we are exactly generating or falsifying when we are synthesizing a molecule like quinine or indigo, or for that matter what hypothesis we are exactly trying to generate or falsify when we are composing a poem like “J. Alfred Prufrock”. Synthesis of novel substances is really at the heart of chemistry and it has had an incalculable impact of our way of life. There is great science as well as great art in synthesizing a complex molecule through the precise, creative assembly of simple atomic components; there is great beauty as well, of the kind found in constructing the finest cathedrals. 

There is really nothing that a chemist is trying to falsify when she makes a new compound, except to prove that it can actually be made. In addition, chemistry much more than physics is a tool-driven science, and instrumental revolutions like x-ray crystallography and NMR spectroscopy are counter to the more traditional idea-driven revolutions framework by Thomas Kuhn that is popular among science philosophers. Chemistry is thus a slippery eel, easily escaping the grasp of the flowing waters of philosophy. It’s this inability of traditional boxes of philosophy to hold chemistry that often makes it hard for people to appreciate it.

The second aspect of chemistry makes it easier for biologists to appreciate it than physicists. Hoffmann provocatively hits on this aspect when he says, “When I talk about chemistry I have three audiences in mind; fellow academics in the humanities and arts, the man on the street and physicists. Among these three I find it hardest to explain chemistry to physicists, because they think they understand, but they don’t”. The problem here is that chemistry did depend on physics, especially atomic physics and quantum mechanics, to provide some of its key foundations. There is little doubt that explaining the Bohr atom allowed theoretical chemists to then explain the chemical bond. But this success also lulled physicists – and I would say a good number of laymen – into an illusory sense of total explanatory power.

This illusion was reflected in the words of Paul Dirac, as great a theoretical physicist as one can find, when, after setting into place the full laws of quantum mechanics in the late 1920s, he said that “The underlying physical laws necessary for the mathematical theory of a large part of physics and the whole of chemistry are thus completely known, and the difficulty is only that the exact application of these laws leads to equations much too complicated to be soluble. It therefore becomes desirable that approximate practical methods of applying quantum mechanics should be developed, which can lead to an explanation of the main features of complex atomic systems without too much computation.”

Dirac was both presciently, profoundly right in saying this as well as profoundly wrong. Profoundly right because it is indeed true that many simplifying approximations and massive computations have to be brought to bear when quantum mechanics is applied to real chemical systems. Profoundly wrong because while this fact is true in theory, it’s almost irrelevant for real chemical systems. Even if you could hypothetically solve the Schrödinger equation for every single molecule of DNA in the body, that solution would still not tell you why DNA is a double helix, why it replicates semi-conservatively, why it mutates, how these mutations are passed down from parents to children or how the information it encodes is passed from DNA to RNA to protein.

All these are examples of emergent phenomena, unique to chemistry that cannot be completely reduced to physics. One can write down the Schrödinger equation for DNA, but the exact functions of DNA are the consequences of its unique structure combined with evolutionary contingency that selected the replication and transmission of hereditary characteristics as one among many functions of DNA. Contingency and emergence confer a special status on DNA the chemical as opposed to DNA the collection of atoms described by quantum theory. The same theme permeates other parts of chemistry. A good example is the hydrogen bond, a bonding interaction that’s strong enough to hold the molecules of life together but weak enough to allow them to shape-shift between structures performing a variety of functions essential to life. The hydrogen bond is a minimalist feature of chemical and biological systems that’s composed of just three atoms, oxygen or nitrogen and hydrogen being exchanged between them like a tennis ball. One can write a Schrödinger equation for a hydrogen bond and it’s useful in deriving fairly accurate energies for it, but the solution by itself doesn’t inform us how useful hydrogen bonds are, how they differ on different length and time scales and how their distribution of energies leads us to a more refined understanding of biological systems.

There are concepts in chemistry like hydrogen bonding, electronegativity, aromaticity and polarizability that get “frayed at their edges”, in Hoffmann’s words, when one tries to scrutinize them too finely using the scalpel of physics; in that sense they are like the mythical electron that physicists talk about, best-behaved when not observed and left alone. It’s not that physics is useless for understanding these ideas, it’s that they are best understood at the level of chemistry itself as semi-qualitative concepts.

It’s this emergent nature of chemical concepts which still keep one foot rooted in physics, this imprecise and yet immensely useful blend of rigor and qualitative understanding, this inability of traditional philosophy of science to keep chemistry encased within its boxes, that makes chemistry a unique science. It’s not hard to understand. It’s just complicated.

First published on 3 Quarks Daily.