Book review: Quantum mechanics and quantum mechanics. David Kaiser's "Quantum Legacies: Dispatches from an Uncertain World"

David Kaiser is a remarkable man. He has two PhDs from Harvard, one in physics and one in the history of science, and is a professor in the Science, Technology and Society department at MIT. He has written excellent books on the history of particle physics and the quirky personalities inhabiting this world. On top of it all he is a genuinely nice guy - he once wrote me a long email out of the blue, complimenting me on a review of his book "How the Hippies Saved Physics". And while his primary focus is the history and philosophy of physics, Kaiser still seems to find time for doing research in quantum entanglement.

What makes Kaiser unique is the attention he gives to what we can call the sociological aspects of physics, things like the physics job market, portrayals of physicists in the humanities, parallel threads of science and history, and perhaps most uniquely, the publications of physics - both the bread-and-butter textbooks that students use and the popular physics books written for laymen. It's this careful analysis of physics's sociological aspects that makes "Quantum Legacies" a delightful read, tread as it does on some of the under-explored aspects of physics. There are chapters on quantum indeterminacy and entanglement and the lives of Schrödinger, Einstein and Dirac, a nice chapter on computing and von Neumann's computer and interesting essays on the Large Hadron Collider and the tragic Superconducting Supercollider which was shelved in 1993 and the Higgs boson. All these are worth reading. But the real gem in the book as far as I am concerned is a collection of three chapters on physics publishing; this is the kind of material that you won't find in other books on the history and philosophy of physics.

The first chapter is about a book that fascinated me to no end while I was growing up - Fritjof Capra's "The Tao of Physics" which explored parallels between quantum physics and Eastern mysticism. This book along with the downright dubious "aliens-visited-earth" literature by the Danish writer Erich von Daniken dotted my bedroom for a while until I grew up and discovered in particular that Daniken was peddling nonsense. But Capra isn't that easy to dismiss, especially as Kaiser tells us, his book hit the market at a perfect time in 1975 when physicists had become disillusioned by the Vietnam War, the public had become disillusioned by physicists, and both groups of people had become smitten with the countercultural movement, Woodstock and Ravi Shankar. There could be no better time for a book exploring the ins and outs of both the bizarre world of quantum mechanics and the mystical world of Buddhism and the "Dance of Shiva" to become popular. Kaiser describes how Capra's book set the tone for many similar ones, and while most of the parallels described in it are fanciful, it did get the public interested in both quantum physics and Eastern philosophy - no small feat. Capra's own personal story, one in which he comes to the United States from Vienna, has a hard time making ends meet and goes back and then decides to first write a textbook and then a more unique popular book based on his experiences in California and advice from famed physicist Victor Weisskopf, is also quite interesting.

The second interesting chapter is about a textbook, albeit a highly idiosyncratic one, that is a household name to students of general relativity - a 1200 page doorstop of a tome by Kip Thorne, Charles Misner and John Wheeler, all legendary physicists. "MTW" as the textbook became known was a kind of landmark event in physics publishing. The textbook was the first major book to introduce advanced undergraduate and graduate students to fascinating concepts like time dilation, spacetime curvature and black holes. The joke about its size was that not only was the book *about* gravity but that it also *generated* gravity. But everything about the book was highly unconventional and quirky, including the typeface, the non-linear narrative and most importantly, serious and advanced mathematical calculations interspersed with boxes containing cartoons, physicist biographies and outrageous speculations about wormholes and time travel. Most people didn't know what to make of it, and perhaps the best review came from the Indian-American astrophysicist Subrahmanyan Chandrasekhar who said, "The book espouses almost a missionary zeal in preaching its message. I (probably for historical reasons) am allergic to missionaries." Nonetheless, "MTW" occupies a pride of place in the history of physics textbooks, and a comparable one on sagging student shelves where it's probably more seen than read.

The last chapter and perhaps the one I found most interesting is about the content of traditional quantum mechanics textbook, which is really a history of the quantum mechanics textbook in general. The first quantum mechanics textbooks in the United States came out in the 1940s and 50s. Many of them came out of the first modern school of theoretical physics in the country founded by J. Robert Oppenheimer at the University of California, Berkeley. Two of Oppenheimer's students, David Bohm and Leonard Schiff, set the opposing tones for two different kinds of textbooks (I remember working through a bit of Schiff's book as an undergraduate). After the war Schiff taught at Stanford, Bohm at Princeton.

Bohm was old school and believed in teaching quantum mechanics as a subject fraught with fascinating paradoxes and philosophical speculations. His approach was very close in spirit to the raging debates of the original scientist-philosophers who had founded the revolutionary paradigm - Niels Bohr, Albert Einstein, Erwin Schrödinger and Werner Heisenberg in particular. Bohm of course had a very eventful life in which he was accused on being a Communist and hounded out of the country, after which he settled in England and became known for carrying out and publishing a set of philosophical dialogues with Indian philosopher J. D. Krishnamurthy. His textbook is still in print and is worth reading, but it's worth noting that the Schrödinger equation is not even introduced until several chapters into the volume.

Schiff's book was different and was a practical textbook that taught students how to solve problems, mirroring a philosophy called "shut up and calculate" that was then taking root in American higher physics education. The Schrödinger equation was introduced on page 6. What Kaiser fascinatingly demonstrates, often through analysis of the original lecture notes from Bohm and Schiff's classes, is that this attitude reflected both a mushrooming of physics students as well as a higher demand for physicists engendered by the Cold War and the military-industrial complex. Not surprisingly, when you had to turn out large numbers of competent physicists with jobs waiting for them in the nation's laboratories and universities, you had little time or patience to teach them the philosophical intricacies of the field. Shut up, calculate, and get out there and beat the Soviets became the mantra of the American physics establishment.

Fascinatingly, Kaiser finds out that the philosophical trends and the practical ones in physics textbook publishing wax and wane with the times; when the job market was good and enrollment was high, the practical school prevailed and textbooks accordingly reflected its preferences, and when the pickings were slim, the job market was tight and enrollment drastically dropped, philosophical questions started making a comeback on tests and in textbooks. Especially after 1970 when the job market tanked, the Vietnam War disillusioned many aspiring physicists and the countercultural movement took off, philosophical speculations took off as well and combined with Fritjof Capra's "The Tao of Physics". Perhaps the ultimate rejection of philosophy among physicists might be said to have come during the second job slump in the early 90s, when many physicists left the world of particles and fields for the world of parties and heels on Wall Street.

Physics publishing, the physics market, the lives of physicists and physics theories have a strange and unpredictable entanglement of their own, one which even Einstein and Bohr might not have anticipated. Kaiser's book explores these well and brings a unique perspective to some of the most interesting aspects of a science that has governed men's lives, their education and their wallets.

Spooky factions at a distance

For me, a highlight of an otherwise ill-spent youth was reading mathematician John Casti’s fantastic book “Paradigms Lost“. The book came out in the late 1980s and was gifted to my father who was a professor of economics by an adoring student. Its sheer range and humor had me gripped from the first page. Its format is very unique – Casti presents six “big questions” of science in the form of a courtroom trial, advocating arguments for the prosecution and the defense. He then steps in as jury to come down on one side or another. The big questions Casti examines are multidisciplinary and range from the origin of life to the nature/nurture controversy to extraterrestrial intelligence to, finally, the meaning of reality as seen through the lens of the foundations of quantum theory. Surprisingly, Casti himself comes down on the side of the so-called many worlds interpretation (MWI) of quantum theory, and ever since I read “Paradigms Lost” I have been fascinated by this analysis.

So it was with pleasure and interest that I came across Sean Carroll’s book that also comes down on the side of the many worlds interpretation. The MWI goes back to the very invention of quantum theory by pioneering physicists like Niels Bohr, Werner Heisenberg and Erwin Schrödinger. As exemplified by Heisenberg’s famous uncertainty principle, quantum theory signaled a striking break with reality by demonstrating that one can only talk about the world only probabilistically. Contrary to common belief, this does not mean that there is no precision in the predictions of quantum mechanics – it’s in fact the most accurate scientific framework known to science, with theory and experiment agreeing to several decimal places – but rather that there is a natural limit and fuzziness in how accurately we can describe reality. As Bohr put it, “physics does not describe reality; it describes reality as subjected to our measuring instruments and observations.” This is actually a reasonable view – what we see through a microscope and telescope obviously depends on the features of that particular microscope or telescope – but quantum theory went further, showing that the uncertainty in the behavior of the subatomic world is an inherent feature of the natural world, one that doesn’t simply come about because of uncertainty in experimental observations or instrument error.

At the heart of the probabilistic framework of quantum theory is the wave function. The wave function is a mathematical function that describes the state of the system, and its square gives a measure of the probability of what state the system is in. The controversy starts right away with this most fundamental entity. Some people think that the wave function is “epistemic”, in the sense that it’s not a real object and is simply related to our knowledge – or our ignorance – of the system. Others including Carroll think it’s “ontological”, in the sense of being a real entity that describes features of the system. The fly in the ointment concerns the act of actually measuring this wave function and therefore the state of a quantum system, and this so-called “measurement problem” is as old as the theory itself and kept even the pioneers of quantum theory awake.

The problem is that once a quantum system interacts with an “observer”, say a scintillation screen or a particle accelerator, its wave function “collapses” because the system is no longer described probabilistically and we know for certain what it’s like. But this raises two problems: Firstly, how do you exactly describe the interaction of a microscopic system with a macroscopic object like a particle accelerator? When exactly does the wave function “collapse”, by what mechanism and in what time interval? And who can collapse the wave function? Does it need to be human observers for instance, or can an ant or a computer do it? What can we in fact say about the consciousness of the entity that brings about its collapse?

The second problem is that contrary to popular belief, quantum theory is not just a theory of the microscopic world – it’s a theory of everything except gravity (for now). This led Erwin Schrödinger to postulate his famous cat paradox which demonstrated the problems inherent in the interpretation of the theory. Before measurement, Schrödinger said, a system is deemed to exist in a superposition of states while after measurement it exists only in one; does this mean that macroscopic objects like cats also exist in a superposition of entangled states, in case of his experiment in a mixture of half dead-half alive states? The possibility bothered Schrödinger and his friend Einstein to no end. Einstein in particular refused to believe that quantum theory was the final word, and there must be “hidden variables” that would allow us to get rid of the probabilities if only we knew what they were; he called the seemingly instantaneous entanglement of quantum states “spooky action at a distance”. Physicist John Bell put that particular objection to rest in the 1960s, proving that at least local quantum theories could not be based on hidden variables.

Niels Bohr and his group of followers from Copenhagen were more successful in their publicity campaign. They simply declared the question of what is “real” before measurement irrelevant and essentially pushed the details of the measurement problem under the rug by saying that the act of observation makes something real. The cracks were evident even then – the physicist Robert Serber once pointedly pointed out problems with putting the observer on a pedestal by asking if we might regard the Big Bang unreal because there were no observers back then. But Bohr and his colleagues were widespread and rather zealous, and most attempts by physicists like Einstein and David Bohm met with either derision or indifference.

Enter Hugh Everett who was a student of John Wheeler at Princeton. Everett essentially applied Occam’s Razor to the problem of collapse and asked a provocative question: What are the implications if we simply assume that the wave function does not collapse? While this avoids asking about the aforementioned complications with measurement, it creates problems of its own since we know for a fact that we can observe only one reality (dead vs alive cat, an electron track here rather than there) while the wave function previously described a mixture of realities. This is where Everett made a bold and revolutionary proposal, one that was as courageous as Einstein’s proposal of the constancy of the speed of light: he surmised that when there is a measurement, the other realities encoded in the wavefunction split off from our own. They simply don’t collapse and are every bit as real as our own. Just like Einstein showed in his theory of relativity that there are no privileged observers, Everett conjectured that there are no privileged observer-created realities. This is the so-called many-worlds interpretation of quantum mechanics.

Everett proposed this audacious claim in his PhD thesis in 1957 and showed it to Wheeler. Wheeler was an enormously influential physicist, and while he was famous for outlandish ideas that influenced generations of physicists like Richard Feynman and Kip Thorne, he was also a devotee of Bohr’s Copenhagen school – he and Bohr had published a seminal paper explaining nuclear fission way back in 1939, and Wheeler regarded Bohr’s Delphic pronouncements akin to those of Confucius – that posited observer-generated reality. He was sympathetic to Everett but could not support him in the face of Bohr’s objections. Everett soon left theoretical physics and spent the rest of his career doing nuclear weapons research, a chain-smoking, secretive, absentee father who dropped dead of an unhealthy lifestyle in 1982. After a brief resurrection by Everett himself at a conference organized by Wheeler, many-worlds didn’t see much popular dissemination until writers like Casti and the physicist David Deutsch wrote about it.

As Carroll indicates, the MWI has a lot of things going for it. It avoids the prickly, convoluted details of what exactly constitutes a measurement and the exact mechanism behind it; it does away with especially thorny details of what kind of consciousness can collapse a wavefunction. It’s elegant and satisfies Occam’s Razor because it simply postulates two entities – a wave function and a Schrödinger equation through which the wave function evolves through time, and nothing else. One can calculate the likelihood of each of the “many worlds” by postulating a simple rule proposed by Max Born that assigns a weight to every probability. And it also avoids an inconvenient split between the quantum and the classical world, treating both systems quantum mechanically. According to the MWI, when an observer interacts with an electron, for instance, the observer’s wave function becomes entangled with the electron’s and continues to evolve. The reason why we still see only one Schrödinger’s cat (dead or alive) is because each one is triggered by distinct random events like the passage of photons, leading to separate outcomes. Carroll thus sees many-worlds as basically a logical extension of the standard machinery of quantum theory. In fact he doesn’t even see the many worlds as “emerging” (although he does see them as emergent); he sees them as always present and intrinsically encoded in the wave function’s evolution through the Schrödinger equation.

A scientific theory is of course only as good as its experimental predictions and verification – as a quote ascribed to Ludwig Boltzmann puts it, matters of elegance should be left to the tailor and the cobbler. Does MWI postulate elements of reality that are different from those postulated by other interpretations? The framework is on shakier ground here since there are no clear observable predictions except those predicted by standard quantum theory that would truly privilege it over others. Currently it seems that the best we can say is that many worlds is consistent with many standard features of quantum mechanics. But so are many other interpretations. To be accepted as a preferred interpretation, a theory should not just be consistent with experiment, but uniquely so. For instance, consider one of the very foundations of quantum theory – wave-particle duality. Wave-particle duality is as counterintuitive and otherworldly as any other concept, but it’s only by postulating this idea that we can ever make sense of disparate experiments verifying quantum mechanics, experiments like the double-slit experiment and the photoelectric effect. If we get rid of wave-particle duality from our lexicon of quantum concepts, there is no way we can ever interpret the results of thousands of experiments from the subatomic world such as particle collisions in accelerators. There is thus a necessary, one-to-one correspondence between wave-particle duality and reality. If we get rid of many-worlds, however, it does not make any difference to any of the results of quantum theory, only to what we believe about them. Thus, at least as of now, many-worlds remains a philosophically pleasing framework than a preferred scientific one.

Many-worlds also raises some thorny questions about the multiple worlds that it postulates. Is it really reasonable to believe that there are literally an infinite copies of everything – not just an electron but the measuring instrument that observes it and the human being who records the result – splitting off every moment? Are there copies of me both writing this post and not writing it splitting off as I type these words? Is the universe really full of these multiple worlds, or does it make more sense to think of infinite universes? One reasonable answer to this question is to say that quantum theory is a textbook example of how language clashes with mathematics. This was well-recognized by the early pioneers like Bohr: Bohr was fond of an example where a child goes into a store and asks for some mixed sweets. The shopkeeper gives him two sweets and asks him to mix them himself. We might say that an electron is in “two places at the same time”, but any attempt to actually visualize this dooms us, because the only notion of objects existing in two places is one that is familiar to us from the classical world, and the analogy breaks down when we try to replace chairs or people with electrons. Visualizing an electron spinning on its axis the way the earth spins on its is also flawed.

Similarly, visualizing multiple copies of yourself actually splitting off every nanosecond sounds outlandish, but it’s only because that’s the only way for us to make sense of wave functions entangling and then splitting. Ultimately there’s only the math, and any attempts to cast it in the form of everyday language is a fundamentally misguided venture. Perhaps when it comes to talking about these things, we will have to resort to Wittgenstein’s famous quote – whereof we cannot speak, thereof we must be silent (or thereof we must simply speak in the form of pictures, as Wittgenstein did in his famous ‘Tractatus’). The other thing one can say about many-worlds is that while it does apply Occam’s Razor to elegantly postulating only the wave function and the Schrödinger equation, it raises questions about the splitting off process and the details of the multiple worlds that are similar to those about the details of measurement raised by the measurement problem. In that sense it only kicks the can of complex worms down the road, and in that case believing what particular can to open is a matter of taste. As an old saying goes, nature does not always shave with Occam’s Razor.

In the last part of the book, Carroll talks about some fascinating developments in quantum gravity, mainly the notion that gravity can emerge through microscopic degrees of freedom that are locally entangled with each other. One reason why this discussion is fascinating is because it connects many disparate ideas from physics into a potentially unifying picture – quantum entanglement, gravity, black holes and their thermodynamics. These developments don’t have much to do with many-worlds per se, but Carroll thinks they may limit the number of “worlds” that many worlds can postulate. But it’s frankly difficult to see how one can find definitive experimental evidence for any interpretation of quantum theory anytime soon, and in that sense Richard Feynman’s famous words, “I think it is safe to say that nobody understands quantum mechanics” may perpetually ring true.

Very reasonably, many-worlds is Carroll’s preferred take on quantum theory, but he’s not a zealot about it. He fully recognizes its limitations and discuss competing interpretations. But while Carroll deftly dissects many-worlds, I think that the real value of this book is to exhort physicists to take what are called the foundations of quantum mechanics more seriously. It is an attempt to make peace between different quantum factions and bring philosophers into the fold. There’s a huge number of “interpretations” of quantum theory, some more valid than others, being separated by each other as much by philosophical differences as by physical ones. There was a time when the spectacular results of quantum theory combined with the thorny philosophical problems it raised led to a tendency among physicists to “shut up and calculate” and not worry about philosophical matters. But philosophy and physics have been entwined since the ancient Greeks, and in one sense, one ends where the other begins. Carroll’s book is a hearty reminder for physicists and philosophers to eat at the same table, otherwise they may well remain spooky factions at a distance when it comes to interpreting quantum theory.

The birth of a new theory: Richard Feynman and his adversaries

Leading physicists discuss their field's most pressing problems
at Shelter Island in April 1947; including, among others, Julian
Schwinger, Richard Feynman and J. Robert Oppenheimer

This post was written on occasion of Richard Feynman's 100th birthday on May 11th and was first published on the website 3 Quarks Daily. It's the second in a pair of articles about two landmark meetings in postwar American physics.

A new theory seldom comes into the world like a fully formed, beautiful infant, ready to be coddled and embraced by its parents, grandparents and relatives. Rather, most new theories make their mark kicking and screaming while their fathers and grandfathers try to disown, ignore or sometimes even hurt them before accepting them as equivalent to their own creations. Ranging from Darwin’s theory of evolution by natural selection to Wegener’s theory of continental drift, new ideas in science have faced scientific, political and religious resistance. There are few better examples of this jagged, haphazard, bruised birth of a new theory as the scientific renaissance that burst forth in a mountain resort during the spring of 1948.

April 2, 1948. Twenty-eight of the country’s top physicists met at the Pocono Manor Hotel near the Delaware Water Gap in Pennsylvania. Kept apart from their first love of fundamental research in physics by the war, they were eager to regroup and rethink the problems which had plagued the heights of their profession before they were called away for war duty to Los Alamos, Cambridge and Chicago.

The listing of participants provides a rare snapshot of one of those hallowed transitions in the history of science, a passing of the torch. Both the old and the new guards were there. The old guard was represented, among others, by Niels Bohr, Paul Dirac and Eugene Wigner – the men who had formulated and then shaped the material world in its quantum mechanical image during the 1920s and 30s. The new guard was represented by Richard Feynman, John Wheeler and Julian Schwinger – the swashbuckling young theorists who wanted to take quantum theory to new heights, even if it meant challenging the old wisdom. J. Robert Oppenheimer who led the conference represented a prophet of the middle ground; a guide joining old hands with new. In retrospect, a clash of worldviews seems almost inevitable.

The problem that was at the forefront of everyone’s attention was the plague of infinities. The infinities had started showing up just a few years after the quantum revolution had burst upon the world. Within a short span of five years or so, between 1925 and 1930, a handful of theorists in their twenties and thirties including Dirac, Werner Heisenberg, Erwin Schrödinger, Max Born and Wolfgang Pauli had completely reworked the foundations of our physical picture of the world. Niels Bohr who along with Albert Einstein and Max Born had kicked off the revolution a decade before was their avuncular godfather; Einstein himself was a reluctant pioneer. The father of quantum mechanics, Max Planck, had showed that energy came only in discrete packets; now these new frontiersmen extended the concept to every physical entity in the universe. The work done by the quantum pioneers revealed a world steeped in probabilities rather than certainties, a world where you could not know the values of even simple parameters like a particle’s position and momentum to infinite precision, a world where particles and waves blurred themselves into each other in a mirage of probability amplitudes and wavefunctions. It was this fundamental ambiguity about what you could know about subatomic entities that led to Einstein’s famous remark about God playing dice.

And yet the theory was accuracy exemplified. Whatever its mathematical and philosophical ambiguities, it kept on providing astonishingly accurate answers to both old and new problems in disparate branches of physics. Whether it was a matter of calculating the frequency of radiation emitted by electrons transitioning in an atom or the resistance of metals to electrical current, quantum theory gave you the right answers, marching in perfect lockstep with numbers from experiment. It seemed to work for virtually every problem you threw it at. Except one.

That puzzle was the interaction between light and matter. It turned up in the simplest of situations, such as calculating the energy of an electron in an atom, and was recognized by the father of quantum field theory, Paul Dirac. Quantum field theory is the most comprehensive description of the world of subatomic particles, and in its simplest sense involves subjecting both particles and the electromagnetic fields which surround them to the rules of quantization. But even a cursory glimpse at the issue made the intractability clear: the energy of a charge in an electromagnetic field – called the ‘self energy’ – is given by the ratio of the strength of the field at various points and the distances between the charge and these points. One calculates the total self-energy by summing up the values at every point. The difficulty is obvious when you think about it: at distances close to zero, you divide by an increasingly smaller number, blowing up the value precipitously. Exactly at the location of the charge, where the distance is zero, the energy becomes infinite; the writers Robert Crease and Charles Mann have described it as a plane being blown to smithereens in its own wake. Clearly this is an absurd result since every energy value which you measure in a real world laboratory is finite.

Starting in about 1930, this problem of infinite self-energy was tackled by many of the most brilliant theoreticians of their time without resolution: Oppenheimer, Heisenberg, the acerbic Wolfgang Pauli and his mild-mannered assistant Victor Weisskopf all described the problem and tried to resolve it in various ways. If anything the picture got even worse; instead of just one infinity, other infinities started rearing their ugly heads like the heads of the mythical Hydra. One of these infinities was pointed out by Dirac. It turns out that during its transition in an atom, an electron can briefly spit out a photon and reabsorb it; this seemingly ex nihilo act of creation is allowed by quantum mechanics as long as it’s done in an exceedingly small amount of time. The problem is that the energy between the electron and this so-called ‘virtual’ photon can be apportioned again in an infinite numbers of ways. If you sum up all these ways you again get the dreaded explosion of infinity.

There seemed to be no end to attempts to exorcising these infinities. Then war intervened, the community of American physicists was drawn up for work on radar and the bomb, and the community of European physicists, many of whom had already fled from Hitler and Mussolini and were scattered across at least two continents, joined them. There the matter of the infinities rested until 1947, when an extraordinary conference of physicists was organized at a small inn off the coast of Long Island near New York City. The Shelter Island conference later went down in history as the conference that kicked off the postwar rejuvenation of particle physics, but in April 1947 it still represented the first stirrings of a revolution. The conference was again chaired by Oppenheimer and included a mix of the old and young guards.

The attention of the participants at Shelter Island was focused on one number of singular importance. Sometimes it takes hard experiment to cut the theoretical Gordian knot. While the theorists had struggled with infinities even before the war, they were galvanized by the experiments of Willis Lamb and his colleague Robert Retherford. Lamb was one of those rare breeds of scientist who are comfortable with both theory and experiment. Combining highly skilled techniques in microwave spectroscopy developed during wartime work on radar with a good understanding of the problems with infinities plaguing quantum field theory, Lamb and Retherford discovered a slight difference in energy between two states of the hydrogen atom at a place where the original Dirac theory predicted no difference. In science revolutions are sometimes engineered by the slightest and most mundane-looking discrepancies in the behavior of matter – Arthur Eddington’s measurement of a tiny shift in the position of the stars predicted by Einstein’s general theory of relativity comes to mind – and the Lamb Shift is as good an example as any of this pivot point in scientific history.

The Lamb Shift is also a telling example of what happens when multiple ideas are in the air, vying with each other for publicity and survival. In the conference Weisskopf had already presented a calculation that could potentially explain the shift, and so had one of the members of the old guard, Hendrik Kramers, who had been Niels Bohr’s assistant. But neither of these efforts got rid of the infinities. It took Hans Bethe with his absolutely mastery of synthesizing different ideas to take the Lamb Shift to its logical conclusion. Nobody surpassed Bethe in his knowledge of multiple branches of physics and his ability to calculate real world answers using the right combination of mathematical techniques and approximations. During a train journey back from Shelter Island, Bethe had the stroke of insight to attempt a calculation of the Lamb Shift using a non-relativistic approximation that ignored effects due to Einstein’s special theory of relativity. In addition, he introduced a physically sensible cutoff for the infinities to get a finite answer. Everyone knew that a correct quantum field would have to include special relativity, so it took some courage on Bethe’s part to attempt a non-relativistic calculation. Strikingly, the result was very close to experiment; 1040 MHz vs 1000 MHz. It still wasn’t the exact answer, but Bethe’s calculation was a shot in the arm, a signal that the theorists’ thinking was on the right track. It was also a fine illustration of how sometimes even a strictly non-realistic, approximate model can guide you in the right direction.

Bethe’s work breathed new life into the work of many others, including Weisskopf and Lamb, both of whom kicked themselves for not thinking about it first. But the biggest impact was on two young members of the group who had already distinguished themselves by their brilliant work during the war – Julian Schwinger and Richard Feynman.

Feynman and Schwinger were two of the earliest products of the American school of theoretical physics. Until the 1930s or so, most American theorists had to go to Europe to learn quantum mechanics at the feet of the masters: Niels Bohr in Copenhagen, Max Born in Göttingen and Arnold Sommerfeld in Munich. In the 30s the center of research started moving to the United States, partly engendered by the exodus of Jewish refugee physicists and partly because of the creation of prominent schools of physics by American physicists themselves. Two of the most prominent schools were Robert Oppenheimer’s at Berkeley and John Archibald Wheeler’s at Princeton. Schwinger came from Oppenheimer’s school; Feynman came from Wheeler’s.

Both Schwinger and Feynman were from New York, but otherwise were very different characters. Feynman was a practical joker who cracked safes, disdained pretension, played the bongos and spoke in colloquial New York City slang. While he had been recognized as a brilliant physicist, he still did not enjoy the star power that Schwinger – a child prodigy who had written his first paper on quantum electrodynamics when he was sixteen – did. Unlike Feynman, the leonine Schwinger wore expensive suits, drove a Cadillac and was the very picture of the distinguished academic. Physicists of the stature of Bethe and Fermi had already paid homage to Schwinger and everyone thought him to be the future. At Shelter Island they had listened to him with reverence; as Oppenheimer put it, “When other physicists do a calculation they want to tell you how they do it; when Schwinger does a calculation he wants to tell you that only he can do it.”

After Shelter Island, the physicists went off to their universities and laboratories, attempting a full calculation of quantum electrodynamics that was relativistic. Schwinger managed to calculate a precise value for the magnetic moment of the electron – a parameter for which comparison between theory and experiment would come to represent the most accurate agreement in all of physics – for the first time in November 1947. Most importantly, the calculation gave a finite answer. One of the elder statesmen of physics, a tough-minded New Yorker named Isidor Rabi, rushed off a note to Bethe: “Schwinger’s calculation is as accurate as yours. God is Great!”.

Feynman was on a completely different track. Working with John Wheeler, he had come up with a novel approach called the path integral approach that included particles traveling backward and forward in time. His bookkeeping technique used a principle familiar from classical mechanics, the principle of least action, that minimized the energy a particle takes in order to travel from A to B. For quantum theory, in Feynman’s hands, one had to consider every single trajectory that the particle would take in order to calculate the most probable one. This so-called 'sum over histories' approach was completely different from anyone else's, although its first trappings had been anticipated by the always prescient Dirac in a paper which Feynman had eagerly read in the Princeton library as a graduate student. Feynman represented his calculations in the form of squiggly and straight lines symbolizing virtual and real particles traveling backward and forward in time. When the Pocono Conference rolled around, he was ready to dazzle his listeners.

Unfortunately Feynman was up against two major obstacles. One was the traditional and hidebound old physics establishment. The other was Julian Schwinger. Schwinger had just given a marathon six-hour talk in which he brought all the formal machinery of mathematical physics to bear on calculating finite answers for electron-photon interactions. His talk was described by some as a virtuoso violin performance, more technique than comprehension. By one account, only Hans Bethe and Enrico Fermi – men who were particularly known for their stamina and powers of concentration – stayed awake and alert enough to follow the entire presentation.

Then Feynman took the podium. Knowing that his listeners would have trouble following the novel derivation of his results, he instead proceeded to simply show them worked out examples. His strategy was understandable, but he was attempting something akin to simply showing worked out examples in a mathematics textbook without showing the underlying theory. For the mandarins of theory who had spent their entire careers trying to take apart and understand all the gory details of how nature worked, this impressionistic-looking display was most unsatisfactory. Immediately they interrupted.

Edward Teller, the Hungarian-born physicist who hadn’t yet achieved the infamous moniker of ‘father of the hydrogen bomb’, thought that Feynman was violating the exclusion principle, a central tenet of physics and chemistry discovered by Wolfgang Pauli that precludes having two electrons with the same energy and spins in the same state. Dirac asked Feynman about a mathematical matrix that carried particle probabilities forward in time. He was wondering about a recondite mathematical property of a matrix called the unitary property that had nonetheless been key in understanding all particle interactions in quantum mechanics. Finally, the elder statesman of physics, the father of them all, Niels Bohr interrupted. Bohr had been impressed at Los Alamos by Feynman’s willingness to brazenly question all authority, including Bohr’s. He now took umbrage at the unfamiliar thicket of squiggles representing particle trajectories. Already in the 1930s, Bohr said in his soft but firm voice, we knew that the classical notion of a trajectory does not make sense in quantum mechanics. Now Feynman seemed to be violating this basic tenet of quantum theory. Bohr strode up to the stage and, standing next to Feynman, speaking in his notorious mumble, delivered a humiliating lecture that seemed to convey Feynman’s lack of understanding of even elementary ideas.

Feynman realized that it was hopeless; Teller was obsessed with a basic fact of quantum mechanics, Dirac was hung up over mathematical formalism, Bohr was still stuck in the 1930s. Clearly his approach was too unconventional and too novel for the old guard. The only way they would listen would be if he laid it all out in an academic paper. History was witnessing the passing of the torch between generations, but for the time being it would have to allow the old generation to win the battle, even if they lost the war. Feynman was undoubtedly on the right track. His new theory had given the right answers for all outstanding problems posed by the new physics. And after his talk, in the next few days, he compared his results with Schwinger’s. These two rivals nonetheless had a healthy respect for each other’s unique approaches, and they realized that were both traversing different trajectories on the mountain of truth.

Within a year Feynman had written up a seminal paper spurred by the disappointment and urgency he witnessed at Pocono. “Space-Time Approach to Quantum Mechanics” would become one of the most important physics papers of the twentieth century. In time, Feynman diagrams would come to dot the pages of the leading physics journals like an art form, much like the native art found on the caves at Lascaux represented its creator’s innermost desires and motivations. And like God bringing, in Schwinger’s words, “computation to the masses”, Feynman would have his own prophet: his colleague Freeman Dyson would unify Schwinger and Feynman’s versions of the promised land and deliver a set of powerful tools that would allow physicists to apply the duo’s techniques to problems in fields ranging from particle physics to astrophysics. And finally, like a voice from the deep, a lonesome letter would come floating to America from the troubled East, where a physicist named Sin-Itiro Tomonaga would have astonishingly worked out Schwinger’s formulation of QED in the isolation and destruction of wartime Japan. In time, QED would provide the most astonishingly accurate between theory and experiment in the history of physics.

But it had all started at Shelter Island and Pocono, where history changed hands and took a new direction, where a thirty year old physicist presented a novel vision; in Dyson’s words, “this wonderful vision of the world as a woven texture of world lines in space and time, with everything moving freely, a unifying principle that would either explain everything or explain nothing.”

Heisenberg on Helgoland

The sun was setting on a cloudless sky, the gulls screeching in the distance. The air was bracing and clear. Land rose from the blue ocean, a vague apparition on the horizon.

He breathed the elixir of pure evening air in and heaved a sigh of relief. This would help the godforsaken hay fever which had plagued him like a demon for the last four days. It had necessitated a trip away from the mainland to this tiny outcrop of flaming red rock out in the North Sea. Here he could be free not just of the hay fever but of his mentor, Niels Bohr.

For the last several months, Bohr had followed him like a shadow, an affliction that seemed almost as bad as the hay fever. It had all started about a year earlier, but really, it started when he was a child. His father, an erudite scholar but unsparing disciplinarian, made his brother and him compete mercilessly with each other. Even now he was not on the best terms with his brother, but the cutthroat competition produced at least one happy outcome: a passion for mathematics and physics that continued to provide him with intense pleasure.

He remembered those war torn years when Germany seemed to be on the brink of collapse, when one revolution after another threatened to tear apart the fabric of society. Physics was the one refuge. It sustained him then, and it promised to sustain him now.

If only he could understand what Bohr wanted. Bohr was not his first mentor. That place of pride belonged to Arnold Sommerfeld in Munich. Sommerfeld, the man with the impeccably waxed mustache who his friend Pauli called a Hussar officer. Sommerfeld, who would immerse his students not only in the latest physics but in his own home, where discussions went on late into the night. Discussions in which physics, politics and philosophy co-existed. His own father was often distant; Sommerfeld was the father figure in his life. It was also in Sommerfeld’s classes that he met his first real friend – Wolfgang Pauli. Pauli was still having trouble attending classes in the morning when there were all those clubs and parties to frequent at night. He always enjoyed long discussions with Pauli, the ones during which his friend often complimented him by telling him he was not completely stupid. It was Pauli who had steered him away from relativity and toward the most exciting new field in physics – quantum theory.

Quantum theory was the brainchild of several people, but Bohr was its godfather, the man who everyone looked up to. It was Bohr who had first applied the notion of discontinuity to the interior of the atom. It was Bohr who had explained the behavior of the simplest of atoms, hydrogen. But much more than that, it was Bohr who had an almost demonic obsession both with the truths of quantum theory and the dissemination of its central tenets to young physicists like him.

Darkness was approaching as he descended the rock and started walking back to his inn. He smiled as he remembered his first meeting with Bohr. After the war, Germany was the world’s most hated nation. Nobody wanted to deal with her. The Versailles treaty had imposed draconian measures on her already devastated economy. How could they do this? Bohr was one of those very few who had extended a statesmanlike hand toward his country. War is war, Bohr had said, but science is science. Its purity cannot be violated by the failings of humanity. The University of Göttingen had invited Bohr to inaugurate a new scientific relationship between Germany and the rest of the world. The day was as clear in his memory as the air around him. The smell of roses wafting through the windows, the audience standing or sitting on the windowsills, the medieval churches chiming in the distance.

Bohr was explaining one of the finer points related to spectroscopy which his quantum theory explained. But there was clearly a mathematical error. Had anyone else seen it? The error was an elementary one, and it did not seem worthy of Bohr. Later as he found out, Bohr was a competent but not particularly noteworthy mathematician. Physical and philosophical intuition was his forte. The mathematics he left to lesser souls, to young men who he called scientific assistants. At Göttingen he pointed out the mistake from the back and offered some other comments. He was all of twenty. Bohr graciously admitted the mistake. After the talk, when he was leaving, Bohr caught up with him. Walk with me, said Bohr. Walk, and talk. It was what Bohr did best.

They climbed up the hill near the university, then discussed the problems of atomic physics in a nearby cafe. He felt he could pledge his soul to Bohr. After Munich he had been tempted to go to Copenhagen right away, but Sommerfeld had cautioned him otherwise. Bohr was an excellent physicist, Sommerfeld had said, but at this stage in his career he would be much better served by a more rigorous and mathematical immersion in atomic physics. The best man to mentor him in this regard was Max Born in Göttingen. Born was hesitant, sometimes too sensitive to perceived slights, often in undue awe of his own students, but there was no one else who combined physical insights with mathematical rigor the way he did. Born could acquaint him much better with the formal techniques; he could always spend time with the philosophical Niels. His friend Pauli had already served as Born’s assistant and had vouched for Born’s first-rate mentorship. However he had cautioned him about Born’s insistence on early morning meetings, an expectation that had been so hard for him to meet that Born had had to send a maid to wake him up.

The moonlight illuminated the path in front of him, but there were few other lights on the tiny island. This was what he liked best about it though. Very few people, very few lights, almost nobody to talk with, but plenty of opportunities for walking and swimming in the cool water. And the air, the air. Crystal clear and seemingly designed for clearing both his nasal passages and the cobwebs in his mind. His hay fever seemed almost gone already. He could read Goethe and think about physics as much as he wanted. When he arrived at the inn he greeted the innkeeper, who when he arrived four days ago, had seemed horrified at his swollen face. She had asked him if he had been in a brawl. Sadly, political brawls and beatings were not uncommon in Germany. After a light meal of sausages and potato dumplings, he retired to his room.

In Munich, for his doctoral dissertation, he had chosen an uncontroversial topic in fluid dynamics. The final exam had been a fiasco though, and he wrinkled his brow as he thought about it. One of the examiners, Wilhelm Wien, had asked him a question from elementary physics about the resolving power of a microscope. He had forgotten the formula and had gotten hopelessly entangled in trying to work it out. He was trying to solve problems at the forefront of quantum theory; why was he being asked to answer questions that were better suited to a second-rate undergraduate? Wien would not let up, however, and Sommerfeld finally had to step in, assuring the examiners that his student was certainly promising enough to be awarded his doctorate. He had still barely escaped with a passing grade. It still rankled.

He had packed his bags and gone straight to Göttingen from Munich. It was partly to start off on quantum theory right away, but also to escape the depressing pessimism that gripped German society. The past year had seen unprecedented inflation cripple his beloved country. At its height an American dollar had been worth a trillion marks. People were carrying entire carts full of money to trade for a load of bread or for some potatoes. They were using it as insulating wallpaper in their homes. Is this what his country really deserved? As he pondered the situation he felt a spring of resentment welling up inside him. If nothing else, he would show them that Germany was still not lacking in scientific talent.

After spending some time with Born and becoming familiar with the fundamental mathematical tools of atomic physics, he had finally made it to Copenhagen. The past few months there had been among the happiest of his life. Bohr had created an atmosphere whose spirit of camaraderie exceeded even Sommerfeld’s seminars. The days would be filled with deep scientific and philosophical discussions, long walks in the Faelledparken behind the institute and games of ping-pong. Evenings were spent in entertaining Bohr and his kind wife Margarethe with Beethoven and Schubert on the piano, which after physics had been his main passion. Even more than Sommerfeld Bohr had become a father figure to him. His avuncular nature, his obsession with quantum theory and his physical agility; all of these were impressive. He would take stairs two at a time, and it seemed nobody could beat him at ping-pong.

But he had also encountered aspects of Bohr’s personality that had not been apparent before. Bohr was very gentle in personal relations, but when it came to divining scientific truth he could be ferocious, unremittingly persistent, a fanatic without scruples. He had been arguing the validity of some rather well known facts of atomic physics, but Bohr’s relentless questioning of even the basic existence of the properties of electrons and photons - questioning that continued well into the night even after he had expressed his fatigue - had almost reduced him to tears. As if Bohr’s inquisition-style interrogation had not been enough, another hitherto unobserved particle had entered Bohr’s orbit since he last met him. His name was Hendrik Kramers. Kramers was Dutch, voluble, mathematically sophisticated, could speak four languages and could play both the piano and the cello. He had been struggling with Danish and English for some time and it was difficult not to be jealous of Kramers. A kind of sibling rivalry had developed between them, both vying for the attention of the father figure.

While he had been putting the finishing touches on his mundane dissertation on fluid dynamics, Bohr, Kramers, and a young American postdoctoral fellow named John Slater had created a compelling picture of electrons in the atom as a set of pendulum-like objects. The technical term for this was harmonic oscillators. The oscillators would vibrate with certain frequencies that would correspond to transitions of electrons between different states in the atoms. Bohr and Kramers were using these oscillators as convenient representations to picture what goes on inside an atom, but they were still concerned with the well-known basic properties of atoms like their positions and velocities. He had been asked to see what he could do with Bohr and Kramers’s model.

This was where the problems had started. He liked the idea of using oscillators to represent electrons. The oscillators expressed themselves in the form of a well-known mathematical device called a Fourier series. His time with Born had made him quite familiar with Fourier series. But when he had inserted formulas for the series into the basic equations of motion, single numbers had grotesquely multiplied into entire lists of numbers. Every time he got rid of certain numbers others would mushroom, like the heads of a Hydra. He had played algebraic games, filled tables upon tables with numerical legerdemain, had gotten not an inch closer to expressing any physical quantity. And then, suddenly, like a gale from the North Sea, he had been swept off his feet by the worst bout of hay fever he remembered. It kept him awake at night. It made him feel groggy during the day. It made the morass of numbers appear even bigger than what it was.

He had finally had enough. Time for resetting the mental gears, he had told himself. The little rocky outcrop with its very low pollen count had been a favored destination for sufferers. That’s where he would go, away from the stifling hay fever and the intellectual hothouse, to the ocean, mountains and clear air which he loved best. He had known this part of the country during expeditions with his youthful Pfadfinder classmates. There they had sung songs about the fatherland and had had fervent patriotic discussions about the spiritual and political revival of Germany. He felt at home there.

The light on the ceiling was flickering as he started thinking about oscillators, about frequencies, about electrons. How does one ever know what goes inside an atom? And that’s when it struck him. It seemed like a bolt out of the blue then, but later on he realized that it was part of a continuum of mental states, a flash of insight that only seemed discontinuous like the transitions of electrons. Once again, how does one ever know what goes on inside an atom? Nobody has seen an atom or electron; they are unobservable. And yet we know they are real because we observe their tangible effects. Unobservable entities have been part of science for a very long time. Nobody knew what went on inside the sun. But scientists – German scientists among them – had figured it out based on the frequencies of spectral lines that indicated the presence of certain elements. Spectroscopy had also been paramount in the development of atomic theory. Bohr himself had demonstrated the success of the theory by using it to explain spectral lines of hydrogen.

He took a step back, looked at the whole picture from a fresh viewpoint, saw the forest for the trees. What we see are spectral lines and nothing else but spectral lines. We do not see the electron’s position; we do not see its momentum. Position and momentum may have been the primary variables in classical physics, but that was because we could measure them. In case of atoms and electrons, all we see are the frequencies of the spectral lines. What we do not see we do not know. Then why pretend to use it? Why pretend to calculate it? The frequencies are the observables. Why not use them as the primary variables, with the positions and momenta as secondary quantities? He had always been a first-rate mathematician, but now he thought about the physics. It was a fundamental shift of a frame of reference, so memorably introduced by Einstein before. The problem was that representing the position and momenta as Fourier series and frequencies still led to a list of numbers rather than a single number obtained by multiplication. But here is where his physical intuition proved pivotal. One could know which numbers from the list to keep and which ones to discard based on whether they represented transitions between real energy states in atoms. That information was available and implicit in the frequency of the spectral lines. Nature could steady that tentative march of numbers.

It was finally time to use his strange calculus to calculate the energy of a real physical system. As his excitement mounted he kept on making mistakes and correcting them, but finally he had it. When he looked at it he was struck with joy and astonishment. Out of the dance of calculations emerged an answer for the energy of the system, but crucially, this energy could only exist in a restricted set of values. In one fell swoop he had rediscovered Max Planck’s original formulation of quantum theory without explicitly using Planck’s energy formula. An answer this correct must be true. An answer this elegant must be true.

It was almost three o’clock in the morning. The night outside seemed to deepen into a deep chasm. He had hardly talked to anyone during his four days on the island, and now it seemed that all that silence was culminating in a full-throated expression of revolutionary insight. The hand of nature and his own dexterous mind had cracked the puzzle in front of him, just as invisible writing is suddenly revealed by the application of the right chemical solution. But the sheer multiplicity of applications that he now foresaw was startling. At first he was deeply alarmed. He had the feeling that, through the surface of atomic phenomena he was looking at a strangely beautiful interior, and now had to probe this wealth of mathematical structures that nature had so generously spread before him.

But that could wait. He now knew that he had a general scheme of quantum theory that could be used to solve any number of old and new problems. Bohr would be pleased, although he would still insist on several modifications to his formulation when it was time to publish. And of course he would show it to his friend Pauli who would provide the most stringent test of the correctness of his theory.


His hay fever seemed to have disappeared. He felt strong again. There did not seem much point in trying to fall asleep at this very late hour. He put on his boots and set out. There was a distant rocky outcrop, the northernmost tip of the island that he had not explored yet. He walked in the predawn light. Not a gull cried around him, not a leaf seemed to tremble. An hour later he was at the base of the rock and scaled it without much effort. There he sat for a long time until he saw the first rays of the sun penetrate the darkness. Photons of light falling on his eyes, stimulating electron transitions in atoms of carbon, nitrogen and oxygen. And at that moment he was the sole human being on earth who knew how this was happening.

Note: This is my latest column for 3 Quarks Daily. It's a piece of historical fiction in which I imagine 24-year-old Werner Heisenberg inventing quantum mechanics on the small island of Helgoland in the North Sea. Heisenberg's formulation was not the easiest to use and was supplanted by Schrödinger's more familiar wave mechanics, but it inaugurated modern quantum theory and was by any reckoning one of the most important discoveries in the history of physics.

Black holes and the curse of beauty: When revolutionary physicists turn conservative

This is my latest monthly column for 3 Quarks Daily.

On September 1, 1939, the leading journal of physics in the United States, Physical Review, carried two remarkable papers. One was by a young professor of physics at Princeton University named John Wheeler and his mentor Niels Bohr. The other was by a young postdoctoral fellow at the University of California, Berkeley, Hartland Snyder, and his mentor, a slightly older professor of physics named J. Robert Oppenheimer.

The first paper described the mechanism of nuclear fission. Fission had been discovered nine months earlier by a team of physicists and chemists working in Berlin and Stockholm who found that bombarding uranium with neutrons could lead to a chain reaction with a startling release of energy. The basic reasons for the large release of energy in the process came from Einstein's famous equation, E = mc², and were understood well. But a lot of questions remained: What was the general theory behind the process? Why did uranium split into two and not more fragments? Under what conditions would a uranium atom split? Would other elements also undergo fission?

Bohr and Wheeler answered many of these questions in their paper. Bohr had already come up with an enduring analogy for understanding the nucleus: that of a liquid drop that wobbles in all directions and is held together by surface tension until an external force that is violent enough tears it apart. But this is a classical view of the uranium nucleus. Niels Bohr had been a pioneer of quantum mechanics. From a quantum mechanical standpoint the uranium nucleus is both a particle and a wave represented as a wavefunction, a mathematical object whose manipulation allows us to calculate properties of the element. In their paper Wheeler and Bohr found that the uranium nucleus is almost perfectly poised on the cusp of classical and quantum mechanics, being described partly as a liquid drop and partly by a wavefunction. At twenty five pages the paper is a tour de force, and it paved the way for understanding many other features of fission that were critical to both peaceful and military uses of atomic energy.

The second paper, by Oppenheimer and Snyder, was not as long; only four pages. But these four pages were monumental in their importance because they described, for the first time in history, what we call black holes. The road to black holes had begun about ten years earlier when a young Indian physicist pondered the fate of white dwarfs on a long voyage by sea to England. At the ripe old age of nineteen, Subrahmanyan Chandrasekhar worked out that white dwarfs wouldn't be able to support themselves against gravity if their mass increased beyond a certain limit. A few years later in 1935, Chandrasekhar had a showdown with Arthur Eddington, one of the most famous astronomers in the world, who could not believe that nature could be so pathological as to permit gravitational collapse. Eddington was a previous revolutionary who had famously tested Einstein's theory of relativity and its prediction of starlight bending in 1919. By 1935 he had turned conservative.

Four years after the Chandrasekhar-Eddington confrontation, Oppenheimer became an instant revolutionary when he worked out the details of gravitational collapse all the way to their logical conclusion. In their short paper he and Snyder demonstrated that a star that has exhausted all its thermonuclear fuel cannot hold itself against its own gravity. When it undergoes gravitational collapse, it would present to the outside world a surface beyond which any falling object will appear to be in perpetual free fall. This surface is what we now call the event horizon; beyond the event horizon even light cannot escape, and time essentially stops flowing for an outside observer.

Curiously enough, the black hole paper by Oppenheimer and Snyder sank like a stone while the Wheeler-Bohr paper on fission gained wide publicity. In retrospect the reason seems clear. On the same day that both papers came out, Germany attacked Poland and started World War 2. The potential importance of fission as a source of violent and destructive energy had not gone unnoticed, and so the Wheeler-Bohr paper was of critical and ominous portent. In addition, the paper was in the field of nuclear physics which had been for a long time the most exciting field of physics. Oppenheimer's paper on the other hand was in general relativity. Einstein had invented general relativity more than twenty years earlier, but it was considered more mathematics than physics in the 1930s. Quantum mechanics and nuclear physics were considered the most promising fields for young physicists to make their mark in; relativity was a backwater.

What is more interesting than the fate of the papers themselves though is the fate of the three principal characters associated with them. In their fate as well as that of others, we can see the differences between revolutionaries and conservatives in physics.

Niels Bohr had pioneered quantum mechanics with his paper on atomic structure in 1913 and since then had been a founding father of the field. He had run an intellectual salon at his institute at Copenhagen which had attracted some of the most original physicists of the century; men like Werner Heisenberg, Wolfgang Pauli and George Gamow. By any definition Bohr had been a true revolutionary. But in his later life he turned conservative, at least in two respects. Firstly, he stubbornly clung to a philosophical interpretation of quantum mechanics called the Copenhagen Interpretation which placed the observer front and center. Bohr and his disciples rejected other approaches to quantum interpretation, including one named the Many Worlds Interpretation pioneered by John Wheeler's student Hugh Everett. Secondly, Bohr could not grasp the revolutionary take on quantum mechanics invented by Richard Feynman called the sum-over-histories approach. In this approach, instead of considering a single trajectory for a quantum particle, you consider all possible trajectories. In 1948, during a talk in front of other famous physicists in which Feynman tried to explain his theory, Bohr essentially hijacked the stage and scolded Feynman for ignoring basic physics principles while Feynman had to humiliatingly stand next to him. In both these cases Bohr was wrong, although the verdict is still out on the philosophical interpretation of quantum mechanics. It seems however that Bohr forgot one of his own maxims: "The opposite of a big truth is also a big truth". For some reason Bohr was unable to accept the opposites of his own big truths. The quantum revolutionary had become an old-fashioned conservative.

John Wheeler, meanwhile, went on to make not just one but two revolutionary contributions to physics. After pioneering nuclear fission theory with Bohr, Wheeler immersed himself in the backwater of general relativity and brought it into the limelight, becoming one of the world's foremost relativists. In the public consciousness, he will probably be most famous for coining the term "black hole". But Wheeler's contributions as an educator were even more important. Just like his own mentor Bohr, he established a school of physics at Princeton that produced some of the foremost physicists in the world; among them Richard Feynman, Kip Thorne and Jakob Bekenstein. Today Wheeler's scientific children and grandchildren occupy many of the major centers of relativity research around the world, and until the end of his long life that remained his proudest accomplishment. Wheeler was a perfect example of a scientist who stayed a revolutionary all his life, coming up with wild ideas and challenging the conventional wisdom.

What about the man who may not have coined the term "black holes" but who actually invented them in that troubled year of 1939? In many ways Oppenheimer's case is the most interesting one, because after publishing that paper he became completely disinterested in relativity and black holes, a conservative who did not think the field had anything new to offer. What is ironic about Oppenheimer is that his paper on black holes is his only contribution to relativity – he was always known for his work in nuclear physics and quantum mechanics after all – and yet today this very minor part of his career is considered to be his most important contribution to science. There are good reasons to believe that had he lived long enough to see the existence of black holes experimentally validated, he would have won a Nobel Prize.

And yet he was utterly oblivious to his creations. Several reasons may have accounted for Oppenheimer's lack of interest. Perhaps the most obvious reason is his leadership of the Manhattan Project and his fame as the father of the atomic bomb and a critical government advisor after the war. He also became the director of the rarefied Institute for Advanced Study and got saddled with administrative duties. It's worth noting that after the war, Oppenheimer co-authored only a single paper on physics, so his lack of research in relativity really reflects his lack of research in general. It's also true that particle physics became the most fashionable field of physics research after the war, and stayed that way for at least two decades. Oppenheimer himself served as a kind of spiritual guide to that field, leading three key postwar conferences that brought together the foremost physicists in the field and inaugurated a new era of research. But it's not that Oppenheimer simply didn't have the time to explore relativity; it's that he was utterly indifferent to developments in the field, including ones that Wheeler was pioneering at the time. The physicist Freeman Dyson recalls how he tried to draw out Oppenheimer and discuss black holes many times after the war, but Oppenheimer always changed the subject. He just did not think black holes or anything to do with them mattered.

In fact the real reason for Oppenheimer's abandonment of black holes is more profound. In his later years, he was afflicted by a disease which I call "fundamentalitis". As described by Dyson, fundamentalitis leads to a belief that only the most basic, fundamental research in physics matters. Only fundamental research should occupy the attention of the best scientists; other work is reserved for second-rate physicists and their graduate students. For Oppenheimer, quantum electrodynamics was fundamental, beta decay was fundamental, mesons were fundamental; black holes were applied physics, worthy of second-rate minds.

Oppenheimer was not the only physicist to be stricken by fundamentalitis. The malady was contagious and in fact had already infected the occupant of the office of the floor below Oppenheimer's – Albert Einstein. Einstein had become disillusioned with quantum mechanics ever since his famous debates with Bohr in the 1920s and his belief that God did not play dice. He continued to be a holdout against quantum mechanics; a sad, isolated, often mocked figure ignoring the field and working on his own misguided unification of relativity and electromagnetism. Oppenheimer himself said with no little degree of scorn that Einstein had turned into a lighthouse, not a beacon. But what is less appreciated is Einstein's complete lack of interest in black holes, which in some sense is even more puzzling considering that black holes are the culmination of his own theory. Einstein thought that black holes were a pathological example of his relativity, rather than a general phenomenon which might showcase deep mysteries of the universe. He also wrongly thought that the angular momentum of the particles in a purported black hole would stabilize its structure at some point; this thinking was very similar to Eddington's rejection of gravitational collapse, essentially based on faith that some law of physics would prevent it from happening.

Unfortunately Einstein was obsessed with the same fundamentalitis that Oppenheimer was, thinking that black holes were too applied while unified field theory was the only thing worth pursuing. Between them, Einstein and Oppenheimer managed to ignore the two most exciting developments in physics – black holes and quantum mechanics – of their lives until the end. Perhaps the biggest irony is that the same black holes that both of them scorned are now yielding some of the most exciting, and yes – fundamental – findings in cosmology, thermodynamics, information theory and computer science. The children are coming back to haunt the ghosts of their parents.

Einstein and Oppenheimer's fundamentalitis points to an even deeper quality of physics that has guided the work of physicists since time immemorial. That quality is beauty, especially mathematical beauty. Perhaps the foremost proponent of mathematical beauty in twentieth century physics was the austere Englishman Paul Dirac. Dirac said that an equation could not be true until it was beautiful, and he had a point. Some of the most important and universal equations in physics are beautiful by way of their concision and universal applicability. Think about E= mc², or Ludwig Boltzmann's equation relating entropy to disorder, S=klnW. Einstein's field equations of general relativity and Dirac's equation of the electron that marries special relativity with quantum mechanics are both prime examples of elegance and deep beauty. Keats famously said that "Beauty is truth and truth is beauty", and Dirac and Einstein seem to have taken his adage to heart.

And yet stories of Dirac and Einstein's quest for beauty are misleading. To begin with, both of them and particularly their disciples seem to have exaggerated the physicists' reliance on beauty as a measure of reality. Einstein may have become enamored of beauty in his later life, but when he developed relativity, he was heavily guided by experiment and stayed very close to the data. He was after all the pioneer of the thought experiment. As a patent clerk in the Swiss patent office at Bern, Einstein gained a deep appreciation for mechanical instrumentation and its power to reveal the secrets of nature. He worked with his friend Leo Szilard on that most practical of gadgets – a refrigerator. His later debates with Bohr on quantum mechanics often featured ingenious thought experiments with devices that he had mentally constructed. In fact Einstein's most profoundly emotional experience came not with a mathematical breakthrough but when he realized that his theory could explain deviations in the perihelion of Mercury, an unsolved problem for a century; this realization left him feeling that "something had snapped" inside him. Einstein's success thus did not arise as much from beauty as from good old-fashioned compliance with experiment. Beauty was a sort of secondary effect, serving as a post-facto rationalization for the correctness of the theory.

Unfortunately Einstein adopted a very different attitude in later years, trying to find a unified field theory that was beautiful rather than true. He started ignoring the experimental data that was being collected by particle physicists around him. We now know that Einstein's goal was fundamentally flawed since it did not include the theory of the strong nuclear force, a theory which took another thirty years to evolve and which could not have progressed without copious experimental data. You cannot come up with a complete theory, beautiful or otherwise, if you simply lack one of the key pieces. Einstein seems to have forgotten a central maxim of doing science, laid down by the sixteenth century natural philosopher Francis Bacon, one of the fathers of the scientific method: "All depends on keeping the eye steadily fixed upon the facts of nature and so receiving their images simply as they are. For God forbid that we should give out a dream of our own imagination for a pattern of the world". In his zeal to make physics beautiful, Einstein ignored the facts of nature and pursued the dreams of his once-awesome imagination.

Perhaps the biggest irony in the story of Einstein and black holes comes from the words of the man who started it all. In 1983, Subrahmanyan Chandrasekhar published a dense and authoritative tome called "The Mathematical Theory of Black Holes" which laid out the complete theory of this fascinating object in all its mathematical glory. In it Chandra (as he was called by his friends) had the following to say:

"In my entire scientific life, extending over forty-five years, the most shattering experience has been the realization that an exact solution of Einstein's equations of general relativity, discovered by the New Zealand mathematician, Roy Kerr, provides the absolutely exact representation of untold numbers of massive black holes that populate the universe. This shuddering before the beautiful, this incredible fact that a discovery motivated by a search after the beautiful in mathematics should find its exact replica in Nature, persuades me to say that beauty is that to which the human mind responds at its deepest and most profound."

Black holes and beauty had come full circle. Far from being a pathological outlier as believed by Einstein and Oppenheimer, they emerged as the epitome of austere mathematical and physical beauty in the cosmos.

Dirac seems to have been guided by beauty to an even greater extent than Einstein, but even there the historical record is ambiguous. When he developed the Dirac equation, he was very closely aware of the experimental results. His biographer Graham Farmelo notes, "Dirac tried one equation after another, discarding each one as soon as it failed to conform to his theoretical principles or to the experimental facts". Beauty may have been a criterion in Dirac's choices, but it was more a way of serving as an additional check rather than a driving force. Unfortunately Dirac did not see it that way. When Richard Feynman and others developed the theory of quantum electrodynamics – a framework that accounts for almost all of physics and chemistry except general relativity - Dirac was completely unenthusiastic about it. This was in spite of quantum electrodynamics agreeing with experiment to a degree unprecedented in the history of physics. When asked why he still had a problem with it, Dirac said it was because the equations were too ugly; he was presumably referring to a procedure called renormalization that got rid of infinities that had plagued the theory for years.

He continued to believe until the end that those ugly equations would somehow metamorphose into beautiful ones; the fact that they worked spectacularly was of secondary importance to him. In that sense beauty and utility were opposed in Dirac's mind. Dirac continued to look for beauty in his equations throughout his life, and this likely kept him from making any contribution that was remotely as important as the Dirac equation. That's a high bar, of course, but it does speak to the failure of beauty as a primary criterion for scientific discovery. Later in his life, Dirac developed a theory of magnetic monopoles and dabbled in finding formulas relating the fundamental constants of nature to each other; to some this was little more than aesthetic numerology. Neither of these ideas has become part of the mainstream of physics.

It was the quest for beauty and the conviction that fundamental ideas were the only ones worth pursuing that turned Einstein and Dirac from young revolutionaries to old conservatives. It also led them to ignore most of the solid progress in physics that was being made around them. The same two people who had let experimental facts serve as the core of their decision making during their youth now behaved as if both experiment and the accompanying theory did not matter.

Yet there is something to be said for making beauty your muse, and ironically this realization comes from the history of the Dirac equation itself. Perhaps the crowning achievement of that equation was to predict the existence of positively charged electrons or positrons. This discovery seemed so alien and unsettled Dirac so much at the beginning that he thought positrons had to be protons; it wasn't until Oppenheimer showed this could not be the case that Dirac started taking the novel prediction seriously. Positrons were finally found by Carl Anderson in 1932, a full three years after Dirac's prediction. This is one of the very few times in history that theory has genuinely predicted a completely novel fact of nature with no experimental basis in the past. Dirac would claim that it was the tightly knit elegance of his equation that logically ordained the existence of positrons, and one would be hard pressed to argue with him. Even today, when experimental evidence is lacking or absent, one has to admit that mathematical beauty is as good a guide to the truth as any other.

Modern theoretical physics has come a long way from the Dirac equation, and experimental evidence and beauty still guide practitioners of the field. Unfortunately physics at the frontiers seems to be unmoored from both these criteria today. The prime example of this is string theory. According to physicist Peter Woit and others, string theory has made no unique, experimentally testable prediction since its inception thirty years ago, and it also seems that its mathematics is unwieldy; while the equations seem to avoid the infinities that Dirac disliked, they also presents no unique, elegant, tightly knit mathematical structure along the lines of the Dirac equation. One wonders what Dirac would have thought of it.

What can today's revolutionaries do to make sure they don't turn conservative in their later years? The answer might come not from a physicist but from a biologist. Charles Darwin, when explaining evolution by natural selection, pointed out a profoundly important fact: "It is not the strongest of the species that survives, nor the most intelligent that survives. It is the one that is most adaptable to change". The principle applies to frogs and butterflies and pandas, and there is no reason why it should not apply to theoretical physicists.

What would it take for the next Dirac or Einstein to make a contribution to physics that equals those of Einstein and Dirac themselves? We do not know the answer, but one lesson that the lives of both these physicists has taught us – through their successes as well as their failures – is to have a flexible mind, to always stay close to the experimental results and most importantly, to be mindful of mathematical beauty while not making it the sole or even dominant criterion to guide your thought processes, especially when an "uglier" theory seems to agree well with experiment. Keep your eye fixed on the facts of nature, not just on the dream of your imagination.