What Oliver Sacks could teach us about the value of anecdotal information in drug discovery research

Oliver Sacks was a writer who elevated the art of anecdote to an art form. He did not write authoritative medical books filled with numbers, graphs and statistics. Instead he focused on individual cases and the human element in medicine. A 2015 Wired magazine profile of Sacks highlighted his signature achievement in resurrecting the value of anecdote in medical research.

By restoring narrative to a central place in the practice of medicine, Sacks has regrafted his profession to its roots. Before the science of medicine thought of itself as a science, at the crux of the healing arts was an exchange of stories. The patient related a confusing odyssey of symptoms to the doctor, who interpreted the tale and recast it as a course of treatment. The compiling of detailed case histories was considered an indispensable tool of physicians from the time of Hippocrates. It fell into disrepute in the 20th century, as lab tests replaced time-consuming observation, merely “anecdotal” evidence was dismissed in favor of generalizable data, and the house call was rendered quaintly obsolete. 

...By exiling the clinical anecdote to the margins of medical practice—to stories passed down in hallways from attending physician to resident—the culture of medicine had blinded itself, forgetting things it had once known. Sacks calls these knowledge gaps “scotomas,” the clinical term for blind spots or shadows in the field of vision. 

...Sacks immersed himself in the neglected anecdotal literature of migraine, feeling that every one of his patients “opened out into an entire encyclopedia of neurology.” 

By studying individual patients with unusual or bizarre neurological symptoms or personality changes, not only the infamous sleeping sickness in "Awakenings" but insomnia, migraines and Tourette's syndrome all revealed their intricacies under Sacks's skillful and empathetic dissection.

Sacks's explorations of anecdotal data were on my mind as I contemplated the pros and cons of anecdotal data in drug discovery, and especially in molecular modeling. It's a topic that is the subject of much discussion, and occasional pugnacious debate. One of the smartest people I know in the field often scorns individual stories of successes in modeling chemical and biological data. If you read about or listen to case studies in the field, you will often find scientists attributing success in a particular drug design study to a particular method, software algorithm or physicochemical factors like electrostatic effects or hydrophobic effects. 

My statistically enlightened colleague has a tendency to dismiss these individual stories: Where, he asks, are the comparisons, often with simpler techniques? Where are the controls? How do you know for certain that there is a causal relationship between a particular method or factor and the success of a particular drug design?

My colleague is absolutely right about the difficulty of extrapolating to causal explanations or general rules based on individual studies. Not a day passes when I don't hear the words "Method X worked for us" or "Method X failed abysmally for us". "Method X" could be a particular assay, chemical reagent, cell line or computational algorithm. The problem is when these statements are extrapolated to "Therefore Method X works" or "Method X does not work". The difference between "works for me" and simply "works" can be the difference between storytelling and actual science.

And yet I cannot help but think, partly based on Oliver Sacks's career, that storytelling has a unique place in drug discovery. And this is not just because of the cultural and community-building role of storytelling that sociologists often extol. It's because I see anecdotes not as data but as starting points for gathering data. And this is because of the sheer complexity of biology, and of drug discovery by extension. The problem is that when you are dealing with a very complex and multifactorial system, one in which the variance of success is large, it's often impossible to find general rules even by running well-designed statistical studies. Do you then completely dismiss individual data points? I think that would be a mistake. 

If a colleague tells me that using a method for optimizing electrostatic interactions worked well for his system, I would try to use that method for my system within time and resource constraints, especially if his study is well documented and carefully done. I would certainly try it out if both of us were working on similar systems (say, kinases) but even if the systems were different I would give it a shot. That's because we know for a fact that similar laws of physics and chemistry operate even in very different systems; often it's a matter of teasing apart which laws among them are dominant and which ones are weaker.

Then there's also Sacks's quote about each one of his patients providing openings into entire encyclopedias of discovery. What Sacks is really talking about here are model systems which highlight a particular feature that is present in other systems. For instance, a patient with memory loss may not be representative of the population as a whole, but he can shed some very valuable insights into the workings of the mechanisms of memory (sadly although illuminatingly, it's only by observing certain damaged neurological mechanisms that you can know more about normal, undamaged ones). 

Model systems are similarly invaluable in fundamental drug discovery research. A good example is a hydrophobic protein cavity in which a single engineered polar amino acid residue can provide information on the role of electrostatics in small molecule binding. There also "virtual" model systems; for instance a molecular mechanics force field in which the electrostatic interactions are turned off to study only the influence of the Van der Waals interactions. Synthetic chemists can also tell you about scores of model systems that are constructed to demonstrate the feasibility of certain reactions or conformational effects. These model systems are not infallible, and they are certainly not data by themselves. But what they are are invitations to discovery, curious starting points, hints that point the way to interesting phenomena. Whether one uses them depends to some extent on time and resource availability, but we would be naive to simply discard them because they represent isolated or anecdotal cases.

That brings me to the value of outliers. Outliers are interesting in almost any investigation, but they are especially important in a paradigm like biology or medicine where the emergent, non-linear nature of the system leads to significant variance. In fact one could argue that medicine is currently being revolutionized by the study of outliers in clinical trials. Outliers can point the way to selecting subgroups of patient populations, so-called 'extraordinary responders" who elicit an unusual response to certain drugs. A great recent example is the anticancer drug Iressa which was withdrawn from the market because of lack of efficacy. However, a few years later AstraZeneca could introduce the drug back into the market when it found out that patients with EGFR mutations responded much better to it than the general population. It was thus the outliers that could help repurpose a drug, one which is saving lives today. And as the genetic basis of medicine is unraveled more and more, we can be sure that specific outlier patients who are distinguished by unique genetic signatures will be critical not only in targeting specific drugs but also in advancing basic science; a great example along these lines is the aerobics instructor in Texas who had a mutation in a protein called PCSK9, now a lucrative target for heart disease drugs. It would have been a shame if her particular case had simply been dismissed as a statistical anomaly.

That then is the great value of anecdotes, not as markers of scientific laws in themselves but as starting points for experiment and theory, as signposts leading us to intellectual forests and mountaintops. Whether these forests and mountaintops contain anything of value is a separate question, but without the anecdotes we may never know about their existence.