Rutherford on tools and theories (and machine learning)

Ernest Rutherford was the consummate master of experiment, disdaining theoreticians for playing around with their symbols while he and his fellow experimentalists discovered the secrets of the universe. He was said to have used theory and mathematics only twice - once when he discovered the law of radioactive decay and again when he used the theory of scattering to interpret his seminal discovery of the atomic nucleus. But that's where his tinkering with formulae stopped.

Time and time again Rutherford used relatively simple equipment and tools to pull off seemingly miraculous feats. He had already won the Nobel Prize for chemistry by the time he discovered the nucleus - a rare and curious case of a scientist making their most important discovery after they won a Nobel prize. The nucleus clearly deserved another Nobel, but so did his fulfillment of the dreams of the alchemists when he transmuted nitrogen to oxygen by artificial disintegration of the nitrogen atom in 1919. These achievements justified every bit Rutherford's stature as perhaps one of two men who were the greatest experimental physicists in modern history, the other being Michael Faraday. But they also justified the primacy of tools in engineering scientific revolutions.

However, Rutherford was shrewd and wise enough to recognize the importance of theory - he famously mentored Niels Bohr, presumably because "Bohr was different; he was a football player." And he was on good terms with both Einstein and Eddington, the doyens of relativity theory in Europe. So it's perhaps not surprising that he pointed out an observation about the discovery of radioactivity attesting to the important of theoretical ideas that's quite interesting.

As everyone knows, radioactivity in uranium was discovered by Henri Becquerel in 1896, then taken to great heights by the Curies. But as Rutherford points out in a revealing paragraph (Brown, Pais and Pippard, "Twentieth Century Physics", Vol. 1; 1995), it could potentially have been discovered a hundred years earlier. More accurately, it could have been experimentally discovered a hundred years earlier.

Rutherford's basic point is that unless there's an existing theoretical framework for interpreting an experiment - providing the connective tissue, in some sense - the experiment remains merely an observation. Depending only on experiments to automatically uncover correlations and new facts about the world is therefore tantamount to hanging on to a tenuous, risky and uncertain thread that might lead you in the right direction only occasionally, by pure chance. In some ways Rutherford here is echoing Karl Popper's refrain when Popper said that even unbiased observations are "theory laden"; in the absence of the right theory, there's nothing to ground them.

It strikes me that Rutherford's caveat applies well to machine learning. One goal of machine learning - at least as believed by its most enthusiastic proponents - is to find patterns in the data, whether the data is dips and rises in the stock market or signals from biochemical networks, by blindly letting the algorithms discover correlations. But simply letting the algorithm loose on data would be like letting gold leaf electroscopes and other experimental apparatus loose on uranium. Even if they find some correlations, these won't mean much in the absence of a good intellectual framework connecting them to basic facts. You could find a correlation between two biological responses, for instance, but in the absence of a holistic understanding of how the components responsible for these responses fit within the larger framework of the cell and the organism, the correlations would stay just that - correlations without a deeper understanding.

What's needed to get to that understanding is machine learning plus theory, whether it's a theory of the mind for neuroscience or a theory of physics for modeling the physical world. It's why efforts that try to supplement machine learning by embedding knowledge of the laws of physics or biology in the algorithms are likely to work, while efforts blindly using machine learning to try to discover truths about natural and artificial systems using correlations alone would be like Rutherford's fictitious uranium salts from 1806 giving off mysterious radiation that's detected without interpretation, posing a question waiting for an explanation.

Ernest Rutherford: Master of simplicity, mentor without peer

It's Ernest Rutherford's birthday today. Along with Michael Faraday, Rutherford was the greatest experimentalist in modern history and the twentieth century's experimental counterpart to Albert Einstein.

Rutherford stands out because he defies the traditional notion of genius. He was a gruff, plainspoken man with a drooping mustache and booming voice who came from simple farming roots in New Zealand. He had little patience for philosophizing and even less for high-flying theorizing; he once said, "Theorists play games with their symbols while we discover truths about the universe". He admonished his pupils to not let him catch them merely talking about the universe in the hallway. And yet I don’t believe that Rutherford was against theory as a whole; after all he lived in age where the theoretical underpinnings of thermodynamics and electromagnetism and the whole edifice of Newtonian mechanics had already demonstrated spectacular successes. That Rutherford had an astute eye for theoretical talent is made clear by the fact that he trained and nurtured Niels Bohr; as dyed-in-the-wool a theoretician and philosopher as you could find. When asked why he adored Bohr in spite of his general disdain for theoreticians, he famously quipped, "Bohr is different. He's a football player". A grateful Bohr held Rutherford in immense regard all his life and even named one of his sons after the master.

Not just Bohr but many others benefited from Rutherford's tutelage; in a record that is unlikely to be ever topped, no fewer than 12 of his students and associates won Nobel Prizes. And many of them made highly consequential discoveries: for instance James Chadwick discovered the neutron, Francis Aston worked out the existence of isotopes and John Cockroft and Ernest Walton built the first particle accelerator and split the atom. The Cavendish Laboratory that he led was later bequeathed to another Nobel Laureate, Lawrence Bragg; Bragg in turn mentored scientists like Max Perutz, James Watson and Francis Crick who led a second revolution in biology. The sum total of Rutherford’s achievements and scientific lineage probably has no parallel in the history of experimental physics.

Rutherford could also inspire in other, odder ways. In September 1933, the Hungarian theoretical physicist Leo Szilard read an article in 'The Times' of London summarizing a lecture by Rutherford the previous night. Any hope of releasing the vast energy unlocked inside atoms was, in his considered opinion, "moonshine". The maverick thinker Szilard who was never thwarted by convention was annoyed by this prognostication of finality. He pondered the topic as he stepped off the curb at a traffic light. The rest is history.

As defining as these qualities were, to me Rutherford appears as a unique historical figure because in this age of Big Science and increasing complexity, he stands out as the undisputed master of the simple experiment. Throughout his career Rutherford made game-changing discoveries using equipment costing at best a few hundred dollars. The early twentieth century was an age of "string and sealing wax" physics and Rutherford harnessed this medium better than any other (Fermi came close). Whether he was bouncing alpha particles off nuclei, studying the disintegration of atoms or discovering radioactive transmutation, Rutherford's main goal was to illustrate deep physical principles in the simplest and most direct way possible.

It's not that he disdained fancy and expensive equipment; his students Walton, Cockcroft and Piotr Kapitza did construct giant machines accelerating particles to high speeds and generating intense magnetic fields. But in his own work Rutherford exemplified the very ideal of the minimalist. An illustration was his beautiful discovery of radioactive transmutation on which he collaborated with his student Ernest Marsden. Marsden had observed that when he bombarded hydrogen atoms in a sealed box with a scintillation screen with alpha particles, he predictably observed hydrogen atoms striking the scintillation screen and setting off flashes. Things got more interesting when he realized that he could observe scintillations even when the box was emptied of hydrogen. This is when Rutherford took over. He realized that the goal was basically one of exclusion; if he could systematically exclude possible suspects that could react with the alpha particle and produce scintillations, he could determine the culprit. It was with this intent that he pumped the box with pure oxygen, carbon dioxide and dry air. Only in the last case did the scintillations double, indicating that there was something in the air that was producing hydrogens. After eliminating every source of contamination, Rutherford realized the revolutionary interpretation of the experiment; the alpha particles were interacting with the nitrogen in air to create oxygen and spit out hydrogen atoms. In one fell swoop Rutherford had realized the dream of alchemists, in spirit if not in every detail; he had literally changed one element into another. There were few other experiments in the history of science that demonstrated mankind's awesome intellect and mastery over nature.

Rutherford's students all stood upon his shoulders when it came to making deep discoveries about the universe using simple approaches. Other researchers also followed suit. The Davisson-Germer experiment, the discovery of nuclear fission and the Pound-Rebka experiment are but three instances of the triumph of simple, small science. Perhaps the most notable of all of Rutherford's disciples was James Chadwick who, in 1932, discovered the neutron. The husband-wife team of Joliot-Curies and other physicists from Germany had observed a penetrating radiation arising from beryllium when it was bombarded by alpha particles. Everyone assumed this radiation to be gamma rays. To prove otherwise, Chadwick used an ingenious and simple method in which he interposed screens at different distances from the source of the mysterious radiation to determine its range and direction. Then, knowing the energy of the incoming particles and the outgoing particles and using classical physics, he determined that the emanating particles must be neutral and must have masses equal to protons. The discovery of the neutron was a watershed in the history of science; since it led directly to nuclear fission and atomic bombs, in one sense it separated history into two stark periods.

Already the age of simple science was diminishing by the time Rutherford died in 1937. By then another Ernest - Ernest Lawrence - was well on his way toward constructing cyclotrons with increasing sizes and energies, a development that led to even bigger and more expensive particle accelerators after World War 2 and culminated in the discovery of the Higgs boson at the LHC in 2012. Big Science has arrived with a resounding bang, not just in physics but in other fields like neuroscience, genetics and astronomy. One wonders what Rutherford and his counterparts would have thought of these endeavors comprising thousands of scientists, a mind-boggling array of instrumentation and the expenditure of billions of dollars. And you also have to ponder what the man who once said "If you need statistics to do science, then it's not science" would have had to say about particles like the Higgs whose existence hangs on the edge of five and six sigma tests of statistical significance rather than direct observation. He must of course have recognized that you cannot discover everything there is to discover about nature using a two-man team and equipment built in your lab's machine shed, but he also would have disapproved of the general shift toward Big Science from small science in ways that biases funding agencies' and individual scientists' priorities. On the other hand, I get the feeling that he would have lauded the success of Silicon Valley in which entrepreneurs building legendary companies out of their garages have kept alive the spirit of small experimentation.

Fortunately the age of simplicity is not dead. The 2010 Nobel Prize in physics went to physicists who discovered graphene using scotch tape. Discoveries in other disciplines like neuroscience are also being engineered using simple techniques; V. S. Ramachandran's work in determining important neural pathways is particularly notable in this regard. Going hand in hand with this kind of simple science is "amateur science", another endeavor that Rutherford would have approved of. Amateur astronomers have discovered extrasolar planets, amateur engineers have built computers and amateur DIY biologists are doing genetic engineering in their garages. In their own way these amateurs, using simple, inexpensive equipment bought off of eBay and pushing the boundaries of what's possible in garages and home labs, are keeping Rutherford's legacy alive. The man who revolutionized our understanding of the universe using little more than string and sealing wax would have watched with pride and hope.

This is a revised version of an older post.