A while ago I wrote about the dominance of foxes over hedgehogs in
chemistry. Hedgehogs love to drill deep into one topic; foxes love to leap over
interdisciplinary fences. Both creatures have been key to the progress of
science: Einstein was a hedgehog, von Neumann was a fox; Darwin was a hedgehog,
Crick was a fox. Many chemists are both hedgehogs and foxes, but my reading of
the recent history of chemistry told me that in chemistry hedgehogs have had a
greater impact.
I was reminded of hedgehogs and foxes again, this time in
the context of drug discovery, as I was reading psychologist Philip Tetlock’s
book on prediction, “Superforecasting: The Art and Science of Prediction”. The book describes a giant prediction
project called the Good Judgment Project which Tetlock created to test people’s
powers of intuiting the future. The project recruited thousands of people, both renowned
experts and amateurs, and asked them to make important predictions related to
finance, politics and human society in general. Participants were asked to
predict the fate of particular currencies or countries, or asked to predict
terrorist attacks or asked to predict the major effects of climate change.
The results of the study were published in 2005 and they
caused quite a stir because they found that several amateurs handily beat experts
at their own game. But another conclusion that the study found, and one that
caught my eye, was that the best predictors were all foxes rather than
hedgehogs. That’s because hedgehogs are more likely to fall in love with one
big idea and try to apply it to most problems while foxes are more likely to be more
modest and try several ideas to pick the ones that work. Hedgehogs may be deep,
but foxes are nimble, and it seems that when it comes to complicated
predictions flexibility is more important than depth.
It struck me that this is exactly how it should be in drug
discovery, and how it often isn’t. The reason that foxes outperformed hedgehogs
in the Good Judgment Project was because multifactorial, complex systems like
the climate and the stock market are seldom amenable to a select few Big Ideas.
What they benefit most from instead are modest but useful ideas from several
different approaches – rational thinking, seat-of-your-pants intuition, rock-solid experience – that are cobbled together to produce a workable recipe.
Now drug discovery should be the poster boy for a multifactorial, complex system whose fruits are ripe for plucking by foxes, but the history of the field actually demonstrates how
hedgehogs have often held it back, sometimes by their own designs but most
often by blind acceptance by their followers. Consider all the Big Ideas that
have permeated drug discovery in the past three decades – structure-based
design, molecular modeling, HTS, combinatorial chemistry, DOS, Rule-of-5. In
each of these cases the relevant Big Idea usually came from a single hedgehog or a few
hedgehogs. Sometimes these individuals highlighted the utility of the idea
beyond its modest but narrow reach, but often it was other scientists or
business leaders that hyped the idea way beyond its domain of applicability (remember that "Designing Drugs Without Chemicals" headline from 1981?).
The Big Idea escaped from its ground reality of bounded rationality and entered
the stratospheric domain of Six Sigma efficiency and business mantras. The same
goes for many other smaller but still influential ideas in the history of drug
discovery, including druglike metrics, molecular dynamics and animal models.
None of these hedgehog-based ideas is useless, and in some
cases they can be very useful, but the key thing to realize is that they have
been the product of hedgehog minds which sought to revolutionize the process of
drug discovery and fold its problems under one big tent. Ironically however, the very
reason these ideas have found use is because they were largely adopted by foxes
who then used them as part of a flexible and versatile toolkit. Equally ironically, while the hedgehogs have undoubtedly led to an understanding of the field,
often that understanding has come from foxes who have illuminated the
limitations of the hedgehogs’ ideas (the Ro5 is a notorious example).
It's also interesting to note that sometimes being a fox or hedgehog can be a matter of time. This is because as our understanding of complex systems grows with time we could transition a bit from foxdom to hedgehogdom. Consider all the attempts to correlate toxicity - a very complex and heterogenous variable - with simple metrics like molecular lipophilicity, flexibility, number of sp3 carbons etc. These attempts represent classic hedgehog-like strategies to usher a complex variable into the tent of a simple rule; no wonder they have been found to be woefully lacking in utility. However we might conjecture that as our understanding of the determinants of toxicity grows in the next few decades, we may potentially reach a tipping point where we may be able to come up with general rules that are at least somewhat broadly applicable. The time then may be better for hedgehogs to make general predictions.
It's also interesting to note that sometimes being a fox or hedgehog can be a matter of time. This is because as our understanding of complex systems grows with time we could transition a bit from foxdom to hedgehogdom. Consider all the attempts to correlate toxicity - a very complex and heterogenous variable - with simple metrics like molecular lipophilicity, flexibility, number of sp3 carbons etc. These attempts represent classic hedgehog-like strategies to usher a complex variable into the tent of a simple rule; no wonder they have been found to be woefully lacking in utility. However we might conjecture that as our understanding of the determinants of toxicity grows in the next few decades, we may potentially reach a tipping point where we may be able to come up with general rules that are at least somewhat broadly applicable. The time then may be better for hedgehogs to make general predictions.
Consider the big names in the history of drug discovery –
James Black, Gertrude Elion, Paul Janssen, Albert Hoffman, Paul Ehrlich, Robert
Koch – and you will find that they were either exclusively or mostly foxes. What they stressed was the plurality of judgment and the exploration of alternative ideas. The
same quality applies to some leading drug hunters who I personally know. Most of
these individuals don’t have one Big Thought attached to their name; rather they
conceived several ideas that led to an integrated approach to drug discovery.
They will not go down in history as the originators of rules and principles,
but they will go down in history as actual drug inventors with names on several key
patents.
My guess is that the reason these people worked so well as
foxes rather than hedgehogs was because they realized how complex biological systems are and how
ignorant we all are when it comes to perturbing these systems with small molecules. I
think that the feature that distinguishes a hedgehog mindset from a fox mindset
is an appreciation of the complexity of the system under consideration. When
you know how incomplete your understanding of biological targets or the
properties of successful drugs is, you quickly realize how hard it would be to
apply a hedgehog-like, one size fits all approach to any problem in drug
discovery. It is far better to diversify your portfolio of tools and ideas and improvise. The best drug hunters are foxes not always because they want to, but
because they realize that they have to. While some of them understand the time-dependence of hedghogish predictions noted above, they are also sober enough to realize that we aren't there yet.
As the future of drug discovery unfolds before our eyes, I do not see hedgehogs eclipsing foxes in relevance. Both are necessary and hedgehogs will continue to come up with big ideas, but these ideas can likely mislead the field if foxes do not reveal their strengths and limitations. Hedgehogs and foxes can both co-exist and thrive in drug discovery, but only if they let each other explore their favorite haunts.
As the future of drug discovery unfolds before our eyes, I do not see hedgehogs eclipsing foxes in relevance. Both are necessary and hedgehogs will continue to come up with big ideas, but these ideas can likely mislead the field if foxes do not reveal their strengths and limitations. Hedgehogs and foxes can both co-exist and thrive in drug discovery, but only if they let each other explore their favorite haunts.
How are you reading the book when it is not yet released? For someone who read the book, you seem to be a little vague on what it says.
ReplyDelete"The results of the study were published in 2005 and they caused quite a stir because they found that handily beat experts at their own game."
No. That was Tetlock's book "Expert Political Judgment" that was published in 2005. The study it was based on DID NOT involve "several amateurs" (in fact, any amateurs). The Good Judgment Project took place in 2011-2015 and some of the results were published in three academic papers. With more to come.
"asked them to make important predictions related to finance, politics and human society in general"
Not really. Because GJP was funded by CIA's research arm IARPA, almost all of the predictions dealt with geopolitical forecasts. The few financial and climate questions were there because of there obvious ties to geopolitics.
BTW, if your company has any use for "superforecasting foxes", it can order some forecasts from a commercial spin-off: http://www.goodjudgment.com/
Not in drug design per se, of course, but potentially as an independent check on issues related to marketing and regulatory issues. Plenty of "superforecasters" are also professionals with backgrounds that allow evaluation of technical issues.
Thanks for reminding of the distinction between the two projects! It would be interesting to see how foxes do when it comes to drug approvals and discovery.
DeleteI haven't read Tetlock, but your description reminds me of some of the examples in Surowiecki's Wisdom of Crowds. Perhaps wise crowds make better predictions because they behave like foxes, incorporating multiple perspectives on the same problem. I also feel like the information cascades that Surowiecki warns against would lead a crowd toward hedgehogdom by causing one given perspective to overshadow the many others.
DeletePhil, your view is basically true. The good thing is, if you read "Superforecasters...", you'll see that Tetlock is not that heavy on that fox/hedgehog metaphor anymore. It's just not particularly useful. To the extent that it is, it basically says that hedgehog's ideological commitment is a detriment. Which is of course true - but relatively trivial. What remains of fox's advantage in what psychologists evaluating the Good Judgment Project called "open-minded thinking" - something that they seem to view as an inherent personality trait. So, simplifying a bit, the biggest "superforecaster" advantage may be in resisting groupthink on all levels.
DeleteGroupthink is certainly the enemy. It's frustrating to see it take over time and time again in politics, business, and science. Scientists are taught to challenge preconceptions, but it doesn't protect them completely from the lure of prevailing notions.
ReplyDelete