Jim Simons: "We never override the computer"

Billionaire-mathematician Jim Simons has been called the most successful investor of all time. His Renaissance Technologies hedge fund has returned an average of 40% returns (after fees) over the last 20 years. The firm uses proprietary mathematical algorithms and models to exploit statistical asymmetries and fluctuations in stock prices to leverage price differentials and make money. 

Simons had made groundbreaking contributions to algebraic topology before founding Renaissance, and his background enabled him to recruit top mathematicians, computer scientists, physicists and statisticians to the company. In fact the company actively stays away from recruiting anyone with a financial or Wall Street background.

I've been enjoying the recent biography of Simons, "The Man Who Solved the Market", by Gregory Zuckerman. But there's an interesting video of a Simons talk at San Francisco State University from 2014 in which he says something very intriguing about the models that Renaissance builds:

"The only rule is that we never override the computer. No one ever comes in any day and says the computer wants to do this and that’s crazy and we shouldn’t do it. You don’t do it because you can’t simulate that, you can’t study the past and wonder whether the boss was gonna come in and change his mind about something. So you just stick with it, and it’s worked."

It struck me that this is how molecular modeling should be done as well. As I mentioned in a previous post, a major problem with modeling is that it's mostly applied in a slapdash manner to drug discovery problems, with heavy human intervention - often for the right reasons, because the algorithms don't work great - obscuring the true successes and failures of the models. But as Simons's quote indicates, the only way to truly improve the models would be to simply take their results at face value, without any human intervention, and test them. At the very minimum, "simulating" historical human intervention is going to be pretty hard. So the only way we'll know what works and what doesn't is if we trust the models and let them rip through. As I pointed out though, in most organizations experimenters are simply not incentivized, nor are there enough resources, to carry out this comprehensive testing. 

Jim Simons and Renaissance can do it because 1. They have the wisdom to realize that that's the only way in which they can get the models to work and 2. They have pockets that are deep enough so that even model failures can be tolerated. Most drug discovery organizations, especially smaller ones, presumably can't do 2. But they could still do it in a limited sense in a handful of projects. What's really necessary though is 1. and my concern is that we'll be waiting for that even if we have the resources to do 2.

The human problems with molecular modeling

Molecular modeling and computational chemistry are the neglected stepchildren of pharmaceutical and biotech research. In almost every company, whether large or small, these disciplines are considered "support" disciplines, peripheral to the main line of research and never at the core. At the core instead are synthetic chemistry, biology and pharmacology, with ancillary fields like formulations and process chemistry becoming increasingly important as the path to a drug progresses.

In this post I will explore two contentions:

1. Unless its technical and human problems are addressed, molecular modeling and simulation will remain peripheral instead of core fields in drug discovery.

2. The overriding problem with molecular modeling is the lack of a good fit between tools and problems. If this problem is addressed, molecular modeling stands a real chance of moving from the periphery to, if not the very core, at least close to the core of drug discovery.

There are two kinds of challenges with molecular modeling that practitioners have known for a long time - technical and human. The technical problems are well known; although great progress has been made, we still can't model the details of biochemical systems very accurately, and even key aspects of these systems like protein motion, water molecules and - in case of approaches like machine learning - lack of adequate benchmarks and datasets continue to thwart the field. 

However, in this piece I will focus on the human problems and explore potential ways of mitigating them. My main contention is that the reason modeling often works so poorly in a pharmaceutical setting is because the incentives of modelers and other scientists are fundamentally misaligned. 

In a nutshell, a modeler has two primary objectives - to make predictions about active, druglike molecules and to validate the models they are using. But that second part is actually a prerequisite for the first - without proper validation, a modeler cannot know if the exact problem space they are applying their models to is actually a valid application of their techniques. For proper validation, two things are necessary:

1. That the synthetic chemist actually makes the molecules they are suggesting.

2. That the synthetic chemistry does not make molecules which they aren't suggesting.

In reality the synthetic chemist who takes up the modelers' suggestions has little to no interest in model validation. As anyone who has done modeling knows, when a modeler suggests ten compounds to a synthetic chemist, the synthetic chemist would typically pick 2 or 5 out of those 10. In addition, the synthetic chemist might pick 5 other compounds which the modeler never recommended. The modeler typically also has no control and authority over ordering compounds themselves.

The end result of this patchwork implementation of the modeler's predictions is that they never know whether their model really worked. Negative data is especially a problem, since synthetic chemists are almost never going to make molecules that the modeler thinks will be inactive. You are therefore left with a scenario in which neither the synthetic chemist nor the modeler knows or is satisfied with the utility of the models. No wonder the modeler is relegated in the back of the room during project discussions.

There is another fundamental problem which the modeler faces, a problem which is actually more broadly applicable to drug discovery scientists. In one sense, not just modeling but all of drug discovery including devices, assays, reagents and models can be considered as a glorious application of tools. Tools only work if they are suited to the problem. If a practitioners thinks the tool will be unsuited, they need to be able to say so and decline using the tool. Unfortunately, incentive structures in organizations are rarely set up for employees to say "no". Hearing this is often regarded as an admission of defeat or an unwillingness to help out. This is a big mistake. Modelers in particular should always be rewarded if they decline to use modeling and can gives good reasons for doing so. As it stands, because they are expected to be "useful", most modelers end up indiscriminately using their tools on problems, no matter what the quality of the data or the probability of success is. This means that quite often they are simply using the wrong tool for the wrong problem. Add to this the aforementioned unwillingness of synthetic chemists to validate the models, and it's little surprise that modeling so often fails to have an impact and is relegated to the periphery.

How does one address this issue? In my opinion, the issue can be mitigated to a significant extent if modelers know something about the system they are modeling and the synthesis which will yield the molecules they are predicting. If a modeler can give sound reasons based on assays and synthesis - perhaps the protein construct they are using for docking is different from one in the assay, perhaps the benchmarks are inadequate or perhaps the compounds they are suggesting won't be amenable to easy synthesis because of a weird ring system - other scientists are more likely to both take their suggestions more seriously as well as respect their unwillingness to use modeling for a particular problem. The overriding philosophy that a modeler utilizes should be captured not in the question, "What's the best modeling tool for this problem?" but "Is modeling the right tool for this problem?". So, the first thing a modeler should know is whether modeling would even work, but if not, he or she will go a long way in gaining the respect of their organization if they can say at least a few intelligent things about alternative experimental approaches or the experimental data. There is no excuse for a computational chemist to not be a chemist in the first place.

More significantly, my opinion is that this mismatch will not be addressed until modelers themselves are in the driver's seat, until they can ensure that their predictions are tested in their entirety. Unfortunately there's little control modelers have over testing their models; much of it simply depends on how much the synthetic chemists trust the modelers, a relationship driven as much by personality and experience as modeling success. Even today, modelers can't usually simply order their compounds for synthesis from internal or external teams.

Fortunately there are two very significant recent developments that promise modelers a degree of control and validation that is unprecedented. One is the availability of cheap CROs like WuXi and Enamine which can make many of the compounds that are predicted by modeling. These CROs have driven the cost down so significantly that even negative predictions can now importantly be tested. In general, the big advantage of external CROs relative to internal chemists is that you can dictate what the external CROs should and shouldn't make - they won't make compounds which you don't recommend and they will make every compound that you do; the whims of personal relationships won't make a difference in a fee-for-service structure.

More tantalizingly, there have been a few success stories now of fully computationally-driven pipelines, most notably Nimbus and Morphic Therapeutic and, more recently, Silicon Therapeutics. When I say "fully computationally driven" I don't mean that synthetic chemists don't have any input - the inaccuracy of computational techniques precludes fully automated molecule selection from a model - what I mean is that every compound is a modeled compound. In these organizations the relationship between modeling and other disciplines is reversed, computation is front and center - at the core - and it's synthetic chemistry and biology in the form of CROs that are at the periphery. These organizations can ensure that every single prediction made by modelers is tested and made, or conversely, that no molecule that is made and tested fails to go through the computational pipeline. At the very least, you can then keep a detailed bookkeeping record of how designed molecules perform and therefore validate the models; at best, as some of these organizations showed, you can discover viable druglike leads and development candidates.

Computational chemistry and modeling have come a long way, but they have a long way to go both in terms of technical and organizational challenges. Even if the technical challenges are solved, the human challenges are significant and will hobble the influence computation has on drug discovery. Unless incentive structures are aligned the fields will continue to have poor impact and be at the periphery. The only way for them to progress is for computation to be in the driver's seat and for computational chemists to be as informed as possible. Fortunately with the advent of the commodification of synthesis and the increased funding and interest in computationally driven drug pipelines, it seems there may be a chance for us to find out how well these techniques work after all.

Image source

What Oliver Sacks could teach us about the value of anecdotal information in drug discovery research

Oliver Sacks was a writer who elevated the art of anecdote to an art form. He did not write authoritative medical books filled with numbers, graphs and statistics. Instead he focused on individual cases and the human element in medicine. A 2015 Wired magazine profile of Sacks highlighted his signature achievement in resurrecting the value of anecdote in medical research.

By restoring narrative to a central place in the practice of medicine, Sacks has regrafted his profession to its roots. Before the science of medicine thought of itself as a science, at the crux of the healing arts was an exchange of stories. The patient related a confusing odyssey of symptoms to the doctor, who interpreted the tale and recast it as a course of treatment. The compiling of detailed case histories was considered an indispensable tool of physicians from the time of Hippocrates. It fell into disrepute in the 20th century, as lab tests replaced time-consuming observation, merely “anecdotal” evidence was dismissed in favor of generalizable data, and the house call was rendered quaintly obsolete. 

...By exiling the clinical anecdote to the margins of medical practice—to stories passed down in hallways from attending physician to resident—the culture of medicine had blinded itself, forgetting things it had once known. Sacks calls these knowledge gaps “scotomas,” the clinical term for blind spots or shadows in the field of vision. 

...Sacks immersed himself in the neglected anecdotal literature of migraine, feeling that every one of his patients “opened out into an entire encyclopedia of neurology.” 

By studying individual patients with unusual or bizarre neurological symptoms or personality changes, not only the infamous sleeping sickness in "Awakenings" but insomnia, migraines and Tourette's syndrome all revealed their intricacies under Sacks's skillful and empathetic dissection.

Sacks's explorations of anecdotal data were on my mind as I contemplated the pros and cons of anecdotal data in drug discovery, and especially in molecular modeling. It's a topic that is the subject of much discussion, and occasional pugnacious debate. One of the smartest people I know in the field often scorns individual stories of successes in modeling chemical and biological data. If you read about or listen to case studies in the field, you will often find scientists attributing success in a particular drug design study to a particular method, software algorithm or physicochemical factors like electrostatic effects or hydrophobic effects. 

My statistically enlightened colleague has a tendency to dismiss these individual stories: Where, he asks, are the comparisons, often with simpler techniques? Where are the controls? How do you know for certain that there is a causal relationship between a particular method or factor and the success of a particular drug design?

My colleague is absolutely right about the difficulty of extrapolating to causal explanations or general rules based on individual studies. Not a day passes when I don't hear the words "Method X worked for us" or "Method X failed abysmally for us". "Method X" could be a particular assay, chemical reagent, cell line or computational algorithm. The problem is when these statements are extrapolated to "Therefore Method X works" or "Method X does not work". The difference between "works for me" and simply "works" can be the difference between storytelling and actual science.

And yet I cannot help but think, partly based on Oliver Sacks's career, that storytelling has a unique place in drug discovery. And this is not just because of the cultural and community-building role of storytelling that sociologists often extol. It's because I see anecdotes not as data but as starting points for gathering data. And this is because of the sheer complexity of biology, and of drug discovery by extension. The problem is that when you are dealing with a very complex and multifactorial system, one in which the variance of success is large, it's often impossible to find general rules even by running well-designed statistical studies. Do you then completely dismiss individual data points? I think that would be a mistake. 

If a colleague tells me that using a method for optimizing electrostatic interactions worked well for his system, I would try to use that method for my system within time and resource constraints, especially if his study is well documented and carefully done. I would certainly try it out if both of us were working on similar systems (say, kinases) but even if the systems were different I would give it a shot. That's because we know for a fact that similar laws of physics and chemistry operate even in very different systems; often it's a matter of teasing apart which laws among them are dominant and which ones are weaker.

Then there's also Sacks's quote about each one of his patients providing openings into entire encyclopedias of discovery. What Sacks is really talking about here are model systems which highlight a particular feature that is present in other systems. For instance, a patient with memory loss may not be representative of the population as a whole, but he can shed some very valuable insights into the workings of the mechanisms of memory (sadly although illuminatingly, it's only by observing certain damaged neurological mechanisms that you can know more about normal, undamaged ones). 

Model systems are similarly invaluable in fundamental drug discovery research. A good example is a hydrophobic protein cavity in which a single engineered polar amino acid residue can provide information on the role of electrostatics in small molecule binding. There also "virtual" model systems; for instance a molecular mechanics force field in which the electrostatic interactions are turned off to study only the influence of the Van der Waals interactions. Synthetic chemists can also tell you about scores of model systems that are constructed to demonstrate the feasibility of certain reactions or conformational effects. These model systems are not infallible, and they are certainly not data by themselves. But what they are are invitations to discovery, curious starting points, hints that point the way to interesting phenomena. Whether one uses them depends to some extent on time and resource availability, but we would be naive to simply discard them because they represent isolated or anecdotal cases.

That brings me to the value of outliers. Outliers are interesting in almost any investigation, but they are especially important in a paradigm like biology or medicine where the emergent, non-linear nature of the system leads to significant variance. In fact one could argue that medicine is currently being revolutionized by the study of outliers in clinical trials. Outliers can point the way to selecting subgroups of patient populations, so-called 'extraordinary responders" who elicit an unusual response to certain drugs. A great recent example is the anticancer drug Iressa which was withdrawn from the market because of lack of efficacy. However, a few years later AstraZeneca could introduce the drug back into the market when it found out that patients with EGFR mutations responded much better to it than the general population. It was thus the outliers that could help repurpose a drug, one which is saving lives today. And as the genetic basis of medicine is unraveled more and more, we can be sure that specific outlier patients who are distinguished by unique genetic signatures will be critical not only in targeting specific drugs but also in advancing basic science; a great example along these lines is the aerobics instructor in Texas who had a mutation in a protein called PCSK9, now a lucrative target for heart disease drugs. It would have been a shame if her particular case had simply been dismissed as a statistical anomaly.

That then is the great value of anecdotes, not as markers of scientific laws in themselves but as starting points for experiment and theory, as signposts leading us to intellectual forests and mountaintops. Whether these forests and mountaintops contain anything of value is a separate question, but without the anecdotes we may never know about their existence.

W B Yeats's "A Poet to his Beloved": Molecular modeling version

Hit picking parties, eviscerating weaknesses and other vistas from the world of molecular docking

Here’s a good review of the pitfalls and promises of molecular docking by John Irwin and Brian Shoichet (UCSF) which is worth your time, especially if you are a non-specialist who wants a summary of what’s happening in the field. As the review notes, there are many first principles-based reasons why docking should not work – poor calculation of ligand conformations, poor treatment of protein and ligand electrostatics and desolvation, non-existent consideration of protein movement, wishful treatment of water molecules, sloppy representation of x-ray structures...the list just goes on. When docking 10⁷ or so ligands involving 10¹³ total configurations, any one of these “maddening details” can doom your study.

And yet as the authors note both through general considerations and a few case studies, incremental but steady improvements in docking methodology have now made the technique respectable in most structure-based drug design campaigns (which interestingly have provided more drug candidates than HTS). Several reasons have contributed to this respectability. The first is the sheer throughput; no experimental technique can possibly screen 10 million ligands in a few days or weeks, so even with its flaws docking rises up at least as a potential complement to experiment. As the review notes, even a 10% success rate in finding new binders to a good target would be an improvement, and with well-defined binding sites the success rate can surpass high throughput screening (thus, the correct question to ask is not whether your method gives you false positives and negatives but whether this error rate is enough to overwhelm the experimental discovery rate for true positives). The second factor is the existence of massive databases like ZINC and ChEMBL containing millions of annotated ligands which could serve as starting points for ligand discovery.

Thirdly, while the holy grail of docking would be to correctly predict the absolute affinities of your ligands or at least to rank them, the more modest goal is to try to separate binders from non-binders and to discover novel chemotypes. Docking has been reasonably successful in meeting this goal, and the review presents several case studies that discovered interesting ligands which were dissimilar to known ones. In turn however, what really makes the discovery of novel chemotypes interesting is that it could lead to novel biology. It is this ability to potentially “break out of medicinal chemistry boxes” that makes docking attractive. For instance you could potentially find agonists by docking when you only found antagonists before, or – in what is one of the more interesting examples illustrated in the article – you could ‘deorphanize’ an enzyme by docking potential substrates to it and predicting its reaction profile. I still find this evidence anecdotal, but it's at least a good starting point for trying out things.

The rest of the review is also useful, not in the least because - given that it's from the Shoichet lab - there's also an instructive checklist of caveats to keep in mind while experimentally screening (PAINS, aggregators etc.) ligands. There is also a discussion of using homology models for docking. Using homology models is tricky even for lead optimization, so I would be wary in applying them too widely for high-throughput docking. While the review does illustrate an interesting case involving a GPCR, I want to note that I did blog about this case when it was published and described how – new ligands notwithstanding – the calculation seemed to use enough computing power and models to light up a startup, along with copious expert input.

It’s this last point in the review which is really the crux of the matter. When the servers have cooled down and the electrons have stopped flowing, the most important equipment that one can bring to bear on a docking study is a good pair of eyes connected to an experienced brain. Even with small error rates "the scum can rise to the top" ("The scum is out there" could be a good tagline for a X-Files episode about HTS). There is little substitute for careful inspection of top docked hits and looking at things like strain, abnormal charged interactions and wrong tautomeric states; otherwise staring down a computer screen would present the same risks as staring down a gun barrel.

The Shoichet lab has occasional "hit-picking parties" where teams of medicinal and computational chemists examine docked structures, and I suspect these parties are more common in other places than you think (although probably not as common as they should be). It’s only when the high throughput-low accuracy domain of docking meets the low throughput-high accuracy domain of the human mind that docking will continue to be successful. Given what we have seen so far I think there are grounds for hope.

Here's that prospective drug design study you asked for...

Improving ligand binding selectivity by
electrostatic optimization

Here's a rare beast: a set of ten prospective molecular design studies from Roche, mostly made possible by a combination of modeling and crystallography. It's a valuable contribution to the field in my opinion, especially since people are always asking about prospective successes of molecular modeling and because companies are reluctant to divulge that kind of data. I especially liked the authors' emphasis on qualitative rather than quantitative aspects of modeling approaches; this is a subtlety not always appreciated by critics and is in fact one that goes to the heart of chemistry as a predictive science.

The review deals with ten early discovery projects involving diverse targets where a variety of modeling techniques were used to improve affinity, selectivity, solubility, pharmacokinetic properties and a bunch of other desired druglike characteristics.

Some of the applications (filling hydrophobic pockets with small aromatic substituents or designing 'steric bumps' to get selectivity against other protein subtypes) are relatively straightforward while others (scaffold hopping, getting affinity by generalized electrostatic optimization, homology modeling) are more challenging and interesting. Here is a table displaying target type, approach and impact of the various protocols.

Most of the projects benefited from early crystallographic data and in fact make a case for getting this kind of data as early as possible, even when you have relatively weak hits (as I have found out through experience, you can get a perfectly reasonable co-crystal structure with a 10 µM hit). At the same time, crystallographic data can sometimes actually surprise and tell you where you went wrong; for instance there is an example of a tryptase inhibitor whose scaffold was redesigned and found to be favorable through modeling, only to realize from the crystal structure that the scaffold was in fact flipped through 180 degrees. That particular example illustrates that occasionally you can get the right answer through the wrong process, although knowing this fact as early as possible itself is quite useful.

Homology models pose a particular challenge for modeling; as I described in a previous post, a small change in the torsional angle of even a single residue can impact your ligand binding prediction. In this review the authors make a good case for using homology models even as crude aids. The crux of the matter is generating good hypotheses, and even crude models can help us do that. In this particular case, based on an initial lead, the model was used to predict a position to add polarity to the molecule. This led to a sulfonamide being replaced by an amide and a spiro ring system which retained potency and good properties.

The last few cases deal with ligand-based optimization in which the lack of 3D protein structural information required the use of 3D ligand overlays. The authors make another important point here: in one of their case studies they used a simple 'shape envelope' instead of detailed QSAR analysis to guide ligand design. As they point out, doing ligand overlaps for detailed QSAR analysis can be very tricky; the devil is in the molecular details and small differences in atom placement might throw you off. There have been several articles bemoaning the limitations of QSAR in recent years, so this sounds like a safe thing to do.

There are some obvious limitations to using such techniques which I am sure the authors are well aware of. Their list features hits, not misses, and many of the techniques which may have worked in these particular cases may not have worked in others. In addition, the review does not explore whether there is in fact a causal relationship between the technique used and the result obtained since other hypotheses aren't always explored. Nevertheless, it is unrealistic to expect researchers to try out every single hypotheses in a project, and what I find most useful in any case about this article is that it provides us with a checklist of things to try and conjectures to test. Science is about ideas, not answers, and as with anything else in drug discovery, if one thing fails you just hold your head high and try another.

The review concludes with a set of lessons which I think are valuable guidelines to think about in any molecular design project. The importance of the first lesson cannot be overemphasized: qualitative statements can often be more useful than quantitative analysis. I can't say this statement doesn't make me beam with pleasure. It is an antidote to those who think of chemistry as physics and expect quantitative predictions. As the case studies here demonstrate, not only are quantitative predictions often a fool's errand but in many cases they aren't even necessary. 

This is a point that I think is often lost on the critics of molecular modeling. The goal of modeling is not just to make detailed predictions; it is to cull unnecessary directions of inquiry, save labor, guide researchers into previously unexplored areas of thinking and generate hypotheses that can be quickly made and tested. It is to help think about molecular design in the broadest possible way. This is value that goes far beyond being able to rank order your latest deck of hits, and it's something to keep in mind the next time an experimentalist asks you whether you can do that.

The other lessons are also worth remembering: use molecular design to shape medicinal chemistry space, employ the principle of parsimony (and Occam's razor), annotate whenever possible (even simple visualization of molecular interactions can be eye-opening), realize the domain of applicability of your techniques (occasionally by stress-testing them) and perhaps most importantly, stay close to experiment. That last part is something all good modelers should know; don't use quantum chemical torsional calculations when you can look up features in the CSD, don't use homology models when you can convince the crystallographer to get you even a low-resolution crystal structure, don't use fancy scaffold morphing software when the medicinal chemist tells you that he or she can rapidly make alternative scaffolds. 

As the review concludes:

"Best practice in molecular design is best practice in all sciences: a relentless focus on clarity, simplicity and good experimental design. What is special about molecular design is the need to build solid hypotheses and to simultaneously foster creative thinking in medicinal chemistry. If we accept this, our focus may shift from the many semi-quantitative prediction tools we have to methods supporting this creative process. Further improvements in computational methods may then have less to do with science than with good software engineering and interface design. The tools are a just means to an end. Good science happens when they are appropriately employed."

A means to an end indeed. Modeling is a poor master but can be a very useful servant.

What do we need? A Jarvis for molecular modeling. When do we need him? Yesterday.

"Jarvis, is this model statistically validated or am I just
making stuff up?"

One of the major and somewhat underappreciated characters in the Iron Man franchise of movies is Jarvis, Tony Stark’s loyal AI system and indispensable assistant. Jarvis in fact may be the principal character in Iron Man apart from Iron Man himself, since he has saved Tony Stark's life more than once. For our purposes though, Jarvis’s key function is in reducing Iron man’s ideas to practice. In the first Iron Man movie, after Tony Stark cobbles together a primitive iron man suit from a bunch of scraps in a cave, it’s Jarvis who helps him turn his newfound ideas into something far more sophisticated. Jarvis has two key capabilities that help him help Stark. One is a superior natural language processing capability that allows him to understand exactly what his creator wants. The other is access to a vast repository of data regarding specs, blueprints, system hybrids and other paraphernalia which he can summon on demand.

What I find most interesting though is Jarvis’s highly interactive nature. He seems to anticipate much of Tony Stark’s thinking, asking questions like “Are you sure you don’t want to use carbon nanotubes instead of titanium, Sir?” or “Do you want me to pull up [system X] which is similar to [system Y] that you are trying to build?”. This interactive capability not only speeds up the progress of projects that Stark is working on, but it also opens up new avenues that he himself may not have thought about.

Why was I thinking about Jarvis? Here’s the thing: I thought about Jarvis when I realized how woefully, drastically uninteractive all of our molecular modeling software is. This criticism does not apply to a specific program or set of tools; it permeates the entire panoply of structure and ligand based drug design tools employed by computational and medicinal chemists. One can debate the pros and cons of specific algorithms for molecular dynamics or docking or QSAR, but I think the one thing we should be able to agree on is that none of these tools anticipate our needs or talk to us even in simple ways. In the sense of being interactive, our modeling software is as primitive as transportation was in the eighteenth century; it sits there, listless and passive, waiting for us to push the buttons and pull the levers.

This is an odd state of affairs. Today we expect most of our electronic devices and software to interact with us; Microsoft Office had in fact implemented their primitive version of Jarvis – the unfortunate and doomed ‘Clippy’ – in Windows back in the 90s. Clippy did not stay around forever, but in principle he was asking the right questions (“It seems you are writing a letter”). What we need is a more sophisticated form of Clippy for our modeling software.

What would a Jarvis or advanced Clippy for modeling look like? For one thing, it would be able to look at a protein or ligand of interest and immediately have at hand a list of similar systems drawn from the academic, industrial and patent literature. Its speed and efficiency assumes ready – instant in fact – access to databases like the PDB, GVK and ChemBL. This task by itself shouldn’t be a problem since it only involves an upfront investment of effort. Once this data has been acquired, our hypothetical Jarvis should then be able to identify the tasks we are embarking on and suggest enhancements, automation or modifications to those tasks. For instance, we may want to identify or probe binding sites in a protein which we want to inhibit. In that case, once we start to run a program like Schrodinger’s SiteMap which accomplishes this, Jarvis should immediately be able to chime in, identify what we are doing, and then retrieve homologous proteins with similar binding sites. It should similarly be able to parse a ligand on the screen and identify similar ligands depending on what we are doing with it. For instance if we were docking that ligand, it would draw up a list of shape-based binding pocket pharmacophores which are similar to the ones in our protein, telling us what the probability of our ligand inhibiting those other proteins might be. This would give us some idea of the protein off-target interactions which our ligand might be expected to have. All this would be displayed as attractively as the graphics in Tony Stark’s basement lab.

In the ligand-based design sphere, a Jarvis for modeling would be especially useful in building QSAR models. One of the biggest pitfalls of QSAR - and in fact of all of computational chemistry - is the existence of spurious or artifactual correlations with biological activity. There are innumerable case studies where people have correlated biological activity of molecules with any number or combination of chemically impenetrable and mathematically dense parameters without first checking whether the activity correlates equally well or better with very simple parameters such as molecular weight, hydrophobicity (logP) or polar surface area. A Jarvis for modeling would make sure that whenever you start building any kind of a QSAR or similar model, he calculates and flashes in front of you a few simple correlations that allow you to make sure that you are not missing simple relationships; only once you are sure that the simple correlations don’t hold up would it make sense to grab a non-linear combination of your favorite ten-dimensional topological index and a parameter from a relativistic quantum chemical calculation. Visualization of this data in a clean, comprehensive and attractive format would again be a key attribute of such a Jarvis.

The desirability of having a Jarvis for molecular modeling dovetails with similar general thoughts I have had about the lack of sophistication in modeling software. I always find it odd that we expect a lot of interactive sophistication in our iPhones and our Samsung watches, and yet we somehow seem to be content in dealing with software for molecular modeling that simply waits for us to push buttons. Generally speaking we don’t apply the same standards to molecular modeling software which we apply to our smartphones, tablets and other devices. We take Siri and Cortana for granted, and yet we don’t demand Siris, Cortanas and Jarvisis in our docking programs. It would be a while before an entity as intelligent as Jarvis is able to help us do better modeling, but that does not mean we don’t start trying right now. I think that demanding this kind of sophistication from scientists, developers and vendors will do a lot of good for the entire community. It’s high time we did.

Beware of von Neumann's elephants using bulldozers to model quarks

The other day I wrote about the late physicist Leo Kadanoff who captured one of the key caveats of models with a seriously useful piece of advice - "Do not model bulldozers with quarks". Kadanoff was talking about the problems that arise when we fail to use the right resolution and tools to model a specific system. While reading Kadanoff's warnings I also remembered one of John von Neumann's equally witty portents for flawed modeling - "With four parameters I can fit an elephant to a curve. With five I can make him wiggle his trunk".

It strikes me that between them Kadanoff and von Neumann capture almost all the cardinal sins of modeling. The other day I was having a conversation about modeling with a leading industrial molecular modeler, and he made the very cogent point that it is imperative to keep the resolution of a particular system and the data it presents in mind when modeling it. My colleague could well have been channeling Kadanoff. This point is actually simple enough to understand (although hard enough to always keep in mind when obeying institutional mandates in a shortsighted environment which thrives on unrealistic short-term goals). 

If you are doing structure based drug design for instance, it's dangerous to try to read too much atomic detail into a 3 angstrom protein-ligand structure. Divining fine details of halogen substitutions, amide flips and water molecules from such a structure can always get you in trouble. If a 3 angstrom structure is the best you have, your optimum strategy would be to try rough designs of molecules - a hydrophobic extension here, a basic amine there - without getting too fine-grained about it. What you should aim for is maximum diversity accessible with minimal synthetic effort - libraries of small peptides might be suitable candidates in such cases. After that let the chemical matter guide you. Once you have a hit, that's when you want to get more detailed, although even then the low resolution of the structure may be at odds with the high resolution of your thinking.

An equally good or even better strategy to adopt in such cases might be a purely ligand-based assault on the structure. There might be similar ligands hitting similar proteins which you might be aware of, or even in case of de novo ligand design you might want to push for purely ligand-based diversity. But this is where you now have to start listening to von Neumann. You may try to fit potential activities of ligands to a few parameters, or build a QSAR model. What you might really be doing however is building not a QSAR model but a house of cards supporting a castle in the air - in other words an overfit model with scant connection to chemically intuitive reality. In that case rest assured - von Neumann's elephant would be quite willing to crash his way in and tear apart your castle.

Kadanoff's admonition to not model bulldozers with quarks is a good admonition for structure-based design. Von Neumann's elephants are good portents to keep in mind for ligand-based drug design. Together the two can hopefully keep you from falling into the abyss and getting crushed under the elephant and the bulldozer.

(Ir)rational drug design and the history of 20th century science

Here is an excellent overview of the hopes and foibles of "rational" drug design by Brooke Magnanti (Hat tip: Pete Kenny) which touches on several themes and names that would be familiar to those in the field: Ant Nicholls and OpenEye, Dave Weininger and Daylight fingerprints, Barry Werth's "The Billion Dollar Molecule" and Vertex, the inflated hopes of structure-based design, cheminformatics and screening etc. 

Those who are heroic survivors of that period would probably start with looking back with dewey eyes, followed by groans of disappointment. The bottom line in that article and several similar ones is that rational drug design and all that it entails (crystallography and molecular modeling in particular) has clearly not lived up to the hype. It's also clear that the swashbuckling scientists portrayed by Werth in his book for instance were more brilliant than successful. It's a tape of hope and woe that has played before, over and over again in fact.

It's clear that much of the faith in rational drug design until now has had a healthy component of irrational exuberance to it. Looking back at the inflated expectations of the 1980s and early 90s for designing drugs atom by atom, followed by the disappointing failures and massive attrition which rapidly succeeded these expectations, makes me wonder what it was exactly that got everyone into trouble. There was a constellation of factors of course, but the historian of science in me thinks that a major part of at least the psychological (and by extension, organizational) aspects of the issue have to deal with the stupendous successes of twentieth century science in generating a mountain of optimism which skeptics are still trying to chip away at.

It's quite clear that as far as scientific progress goes, the 20th century was the mother of all centuries. Very significant scientific advances (Newton, Maxwell, Darwin, Mendel) had undoubtedly occurred in earlier times, but the sheer rate at which science advanced in the last one hundred years far outstripped scientific progress in all previous centuries. Just consider the roster of both idea-based and tool-based scientific revolutions that we witnessed in the past century: x-rays, the atomic nucleus, relativity, quantum mechanics, nuclear fission, the laws of heredity, the structure of biomolecules, particle physics, lasers, computers, organic synthesis, gene editing...and we are just getting warmed up here.

By the 1980s this amazing collection of scientific gems had reached a crescendo, especially in the biomedical sciences. The rise of recombinant DNA technology, protein structure determination, and improved hardware, software and visualization virtually ensured that scientists started feeling very good indeed about designing drugs to block particular proteins at the molecular level. Philosophically too they were highly primed by the astounding reductionist successes of the past one hundred years. After all reductionism had uncovered the cosmic microwave background radiation from the Big Bang, given us the structure of elemental life proteins like hemoglobin and the photosynthetic complex, split the atom, doubled the number of transistors on a chip in eighteen months and taught us how to copy and paste genes. Designing drugs would be a natural extension, if not a job for graduate students, after all this success.

But what happened instead was that both scientifically and philosophically we ran into a wall. What we found out scientifically was that we still understand only a fraction of the complexity of biological systems that we need to for perturbing them with the fine scalpels of small organic molecules. Philosophically we found out that biological systems are emergent and contingent, so all the reductionist success of the past century is still not enough to understand them. In fact beyond a certain point reductionism would fundamentally put us on the wrong track. The past hundred years made us believers in Moore's Law, but what we got instead was Eroom's Law. Moore's Law is what reduces my running time from 12 mins/mile to 8:30 mins/mile in a year. Eroom's Law is what keeps it from reducing much further. Exponential technological success is not axiomatic and self-fulfilling.

I thus see a very strong influence of the success of twentieth century science in steering the wildly optimistic hopes of drug discovery scientists beginning in the 1980s. Hopefully we are wiser now, but institutional forces and biases still keep us from improving on our failures. As Pete Kenny says in his post for instance, obsession with specific technologies rather than a combined application of several technologies still biases scientists and managers in biotech and pharmaceutical organizations. The rise and ebb (did you just say "rise"?) of economic forces makes the job environment unstable and discourages scientists from pushing bold ideas that promise to break free from reductionist approaches. And much of our science is still based on sloppy theorizing without proper recourse to statistics and controls, not to mention an unbiased look at what the experiments truly are and are not telling us. 

Santayana told us that we are condemned to relive history if we forget it. But when it comes to the promises of rational drug design, what we should do perhaps is to purge our minds of the successes of the 20th century and remember Francis Bacon's exhortation from the 16th century instead: "All depends upon keeping the eye steadily fixed on the facts of nature. For God forbid that we should give out a dream of our own for a pattern of the world."

Image: "Cognition enhancer" (Source: Brooke Magnanti, Garrett Vreeland)

What would be a "non-intuitive" prediction in medicinal chemistry?

Over the last two decades when computer-aided drug design was in development, one of the most common refrains you heard from medicinal chemists about its utility was that it did a poor job predicting  “non-intuitive” structural modifications to molecules. But the term is not always easy to define, and while the charge is often valid, it’s also sometimes unfair since what’s non-intuitive can be highly subjective and constitute a moving target. Also, the bar for non-intuitive ideas can be rather high based on the experience of particular medicinal chemists; it’s not always fair to hold a simple computational method up to the same standards as a chemist with thirty years experience (that’s not exactly what the software has been designed for…).

Nonetheless, it’s rather refreshing to hear modelers level the same charge against themselves, which is perhaps a sign that the entire field is now seeking higher standards than before. I was pleased to hear this sentiment noted several times in the ACS meeting in Boston which I just attended. But the question still stands: What prediction could a modeler make that would be deemed ‘non-intuitive’ or 'novel' by a fairly experienced medicinal chemist? There are at least a few cases that come to mind:

Binding and solution conformation prediction: Chemists are used to looking at 2D structures, and even a highly experienced chemist won’t be able to predict most of the time how a complicated-looking molecule will bind in a protein binding pocket. That is what docking is for, and it's one of the few areas of modeling which can claim a modest but solid degree of success. What is still a non-trivial problem is to predict the ensemble of conformations in solution which converge to a single conformation in the protein pocket. I worked on this problem myself in grad school, and it took up the majority of my half-decade or so spent there. The problem is in both determining the population of solution conformations and estimating the binding energy going from multiple to one conformation, and the general solution is still tedious and complicated.

The prediction of conformational changes in general is a problem that cannot be easily visualized by medicinal chemists without some kind of computational or experimental (especially NMR) support. Subtle structural additions like methyl groups or halogens can sometimes cause significant changes in solution conformational populations, which in turn may impact the binding conformation. Generally speaking it is impossible to understand these effects using intuition alone. Intramolecular hydrogen bonds which can stabilize conformations and improve membrane permeability are also hard to visualize or predict without some kind of computational analysis, especially for larger molecules like macrocycles.

Scaffold hopping: Another attractive idea which is not obvious to medicinal chemists. Scaffold hopping involves essentially locating the binding pharmacophore for a molecule and then finding (ideally) a completely different set of bonds and connectivities that would map on to the same pharmacophore. It is especially useful for transforming ring systems to one another or constraining an acyclic system in a ring. One utility of scaffold hopping is to locate bioisosteres. Computational techniques can be very useful here in principle, although pharmacophore detection can be spotty because of problems with false positives and negatives. Scaffold hopping is not just non-intuitive but is also a boon to getting around intellectual property which is usually every chemist’s nemesis.

Calculating desolvation penalties: An experienced medicinal chemist may be able to look at a compound and make a guess about its size or lipophilicity, but guessing desolvation penalties is intuitively quite hard except in obvious cases (as in the case of a positively or negatively charged group – even then, guessing the sum of all the interactions is challenging). One of the reasons is that desolvation being a point charge-dipole interaction, its energy goes up as the square of the charge (instead of just inversely as in Coulomb's law): small changes in heteroatom distributions can thus have significant and non-obvious effects on solvation/desolvation. 

Unfortunately calculating solvation energies is also still hard in a general sense for computational chemists, but progress continues to be made. One of the most successful predictions of modeling would be a case where a highly charged group compensates for all its desolvation by making perfectly formed hydrogen bonds with a protein - this is a very hard thing to predict as of now. Generally speaking though, desolvation penalties would, at least in principle, fall into the category of things that medicinal chemists wouldn’t often be able to guess.

Calculating strain energies: This is another case where even experienced medicinal chemists may not be able to make intuitive statements. Sometimes it’s obvious in an x-ray crystal structure that ligands are strained (manifested for instance in the form of bent amides, bent phenyl rings or any kind of non-planar conjugated systems). But other times the effects of strain can be invisible to the naked eye. The problem is that bond length changes of as little as tenths of an angstrom can translate into significant strain energies of several kcals/mol, and it is hard if not impossible for even seasoned medicinal chemists to actually see these strain-inducing elements without some kind of calculation. That’s where modeling can help.

Water molecules: We know well by now how complicated the behavior of water molecules in protein binding sites can be. Sometimes the kinds of predictions that modelers make about easily and productively displaced water molecules are rather obvious, such as when they are talking about a water molecule in a nice hydrophobic cavity. The subtle cases are harder to intuitively predict. For instance crystallographic waters may be firmly bound and therefore may have good enthalpy but may still be unhappy and displaceable because of an unfavorable entropy. Similarly as detailed in the link above, water molecules at ligand-solvent interfaces may have unexpected thermodynamic features. Unhappy water prediction methods like WaterMap and SZMAP are promising, but only when they can predict non-intuitive scenarios that are refractory to easy analysis by medicinal chemists.

Data analysis: Generally speaking the word ‘non-intuitive’ may also mask more mundane but useful goals like being able to analyze large amounts of data and suggest useful trends. In fact that’s a task that’s usually quite unsuited to the skills of a medicinal chemist because of its reliance on numbers and statistics and opacity to easy structural visualization.

Feel free to note others in the comments section which I might have left out. Even better are cases where modelers can make suggestions that aren't just non-intuitive but counterintuitive. For instance, if you can predict that a methyl group filling a pocket would lead to a drop in potency (steric reasons? trapped water?) that would be a good counterintuitive prediction. Or if you could predict that cyclization of a molecule would actually increase conformational flexibility because of alleviation of syn-pentane interactions (as I found out in my comparison of cyclic dictyostatin with acyclic discodermolide), that prediction would also fall into the same category. Counterintuitive predictions also provide acid tests of any model because of their emphasis on falsifiability.

I don’t claim that all the goals listed above are well within the purview of molecular modeling. What I am claiming is that there are several challenging tasks on which modeling has started to make inroads. And a good number of these could be called “non-intuitive” or even "counterintuitive". A decade or two down the line I don't think such predictions from modeling will be as rare as we currently think they are, and that's something that we all should look forward to.

The problem with molecular modeling is not just molecular modeling

I am attending the Gordon Conference on Computer-Aided Drug Design (CADD) in the verdant mountains of Vermont this week, and while conference rules prohibit me from divulging the details of the talks, even the first day of the meeting reinforces a feeling that I have had for a while about the field of molecular modeling: the problems that plague the field cannot be solved by modelers alone.

This realization is probably apparent to anyone who has been working the field for a while, but its ramifications have become really clear in the last decade or so. It should be obvious by now to many that while modeling has seen some real and solid progress in the last few years, the general gap between promise and deliverables is still quite big. The good news is that modeling has been integrated into the drug discovery process in many small and sundry ways, ranging from getting rid of duplicates and "rogue" molecules in chemical libraries to quick similarity searching of new proposed compounds against existing databases to refinement of x-ray crystal structures. These are all very useful and noteworthy advances, but they don't by themselves promise a game changing impact of modeling on the field of drug discovery and development.

The reasons why this won't happen have thankfully been reiterated several times in several publications over the last fifteen odd years, to the extent that most reasonable people in the field don't get defensive anymore when they are pointed out. There's the almost complete lack of statistics that plagued the literature, leading people to believe that specific algorithms were better than what they actually were and continuing to apply them (this aspect was well emphasized by the last GRC). There's the constant drumbeats about how badly we treat things like water molecules, entropy, protein flexibility and conformational flexibility of ligands. There are the organizational issues concerning the interactions between modelers and other kinds of scientists which in my opinion people don't formally talk about with anywhere near the level of seriousness and the frequency which they deserve (although we are perfectly happy to discuss them in person).

All these are eminently legitimate reasons whose ills must be exorcised if we are to turn modeling into not just a useful but consequential and even paradigm-shifting part of the drug discovery process. And yet there is one other aspect that we should be constantly talking about that really puts a ceiling on top of even the most expert modeler. And this is the crucial reliance on data obtained from other fields. Because this is a ceiling erected by other fields it's not just something that even the best modelers alone can punch through. And breaking this ceiling is really going to need both scientific and organizational changes in the ways that modelers do their daily work, interact with people from other disciplines and even organize conferences.

The problem is simply of not having the right kind of data. It's not a question of 'Big Data' but of data at the right level of relevance to a particular kind of modeling. One illusion that I have felt gradually creeping up the spines of people in modeling-related conferences is that of somehow being awash in data. Too often we are left with the feeling that the problem is not that of enough data, it's only of tools to interpret that sea of information.

The problem of tools is certainly an important one, but the data problem has certainly not been resolved. To understand this, let's divide the kind of data that is crucial for 'lower level' or basic modeling into three categories: structural, thermodynamic and kinetic. It should be obvious to anyone in the field that we have made amazing progress in the form of the PDB as far as structural information is concerned. It did take us some time to realize that PDB structures are not sacrosanct, but what I want to emphasize is that when serious structure-based modeling like docking, homology modeling and molecular dynamics really took off, the structural data was already there, either in the PDB or readily obtained in house. Today the PDB boasts more than a hundred thousand structures. Meticulous tabulation and analysis of these structures has resulted in high-quality datasets like Iridium. In addition there is no dearth of publications pointing out the care which must be exercised in using these structures for actual drug design. Finally, with the recent explosion of crystallographic advances in the field of membrane protein structure, data is now available for virtually every important family of pharmaceutically relevant proteins.

Now consider where the field might have been in a hypothetical universe where the PDB was just getting off of the ground in the year 2015. Docking, homology modeling, protein refinement and molecular dynamics would all have been in the inky backwaters of the modeling landscape. None of these methods could have been validated in the absence of good protein structure and we would have had scant understanding of water molecules, protein flexibility and protein-protein interactions. The Gordon Conference on CADD would likely still be the Gordon Conference on QSAR.

Apply the same kind of thinking to the other two categories of data - thermodynamic and kinetic - and I think we can see some of the crucial problems holding the field back. Unlike the PDB there is simply no comparable database of tens of thousands of reliable thermodynamic data points that would aid the validation of methods like Free Energy Perturbation (FEP). There is some data to be found in repositories like PDBbind, but this is still a pale shadow of the quantity and (curated) quality of structures in the PDB. No wonder that our understanding of energies - relative to structure - is so poor. When it comes to kinetics the situation is much, much worse. In the absence of kinetic data, how can we start to truly model the long residence times in protein-ligand interactions that so many people are talking about these days? The same situation also applies to what we can call 'higher order' data concerning toxicology, network effects on secondary targets in pathways and so on.

The situation is reminiscent of the history of the development of quantum mechanics. When quantum mechanics was formulated in the twenties, it was made possible only by the existence of a large body of spectroscopic data that had been gathered since the late 1870s. If that data had not existed in the 1920s, even wunderkinder like Werner Heisenberg and Paul Dirac would not have been able to revolutionize our understanding of the physical world. Atomic physics in the 1920s was thus data-rich and theory poor. Modeling in 2015 is not exactly theory-rich to begin with, but I would say it's distinctly data-poor. That's a pretty bad situation to be in.

The reality is very simple in my view: unless somebody else - not modelers - generates the thermodynamic, kinetic and higher-order data critical to advancing modeling techniques the field will not advance. This problem is not going to be solved by a bunch of even genius modelers brainstorming for days in a locked room. Just like the current status of modeling would have been impossible to imagine without the contributions of crystallographers, the future status of modeling would be impossible to imagine without the contribution of biophysical chemists and biologists. Modelers alone simply cannot punch through that ceiling.

One of the reasons I note this problem is because even now, I see very few (none?) meetings which serve as common platforms for biophysical chemists, biologists and modelers to come together and talk not just about problems in modeling but how people from these other fields can address the problem. But as long as modelers think of Big Data as some kind of ocean of truth simply waiting to spill out its secrets in the presence of the right tools, the field will not advance. They need to constantly realize the crucial interfacing with other disciplines that is an absolute must for progress in their own field. What would make their own field advance would be its practitioners knocking on the doors of their fellow kineticists, thermodynamicists and network biologists to get them the data that they need.

That last problem suddenly catapults the whole challenge to a new level of complexity and urgency, since convincing other kinds of scientists to do the experiments and procure the data that would allow your field to advance is a daunting cultural challenge, not a scientific one. Crystallographers were busy solving pharmaceutically relevant protein structures long before there were modelers, and most of them were doing it based on pure curiosity. But it took them fifty years to generate the kind of data that modelers could realistically use. We don't have the luxury of waiting for fifty years to get the same kind of data from biophysical chemists, so how do we incentivize them to speed up the process?

There are no easy ways to address this challenge, but a start would be to recognize its looming existence. And to invite more scientists from other fields to the next Gordon Conference in CADD. How to get people from other fields to contribute to your own in a mutually beneficial relationship is a research problem in its own right that deserves separate space at a conference like this. And there is every reason to fill that space if we want our field to rapidly progress.

Want to bind small molecules? Get a backbone

Here’s a paper from the Shoichet lab at UCSF that illustrates one of the major problems that drug designers encounter – predicting conformational changes (“entropy” to a physicist). What the study does is to plug a series of eight very simple congeneric ligands – benzene, methyl, ethyl and propyl benzene all the way to hexyl benzene - into a model protein cavity, in this case a lysozyme mutant, and observe the corresponding changes in protein conformation by solving the crystal structures. And the results aren’t exactly heartwarming for early phase drug discovery scientists.

Synthesizing congeneric series of ligands is a standard process in lead optimization and the elephant in the room which is often banished out of sight by drug designers is the possibility of large conformational changes in the protein caused by small changes in ligand structure (the other assumption is constancy in ligand binding orientation, and even that doesn’t always hold). The assumption is that any minor change in structure in the ligand would be accommodated by equivalent, small amino acid side chain movements in the protein.

This study shows that at best that assumption is a faith-based assumption which should always be considered provisional. What the authors observe is that instead of a smooth transition of amino acid side chain movements, you see a discrete and far more significant change in protein backbone movement, resulting in a subtle population of different states which bind the ligands. The difference in binding energy going from benzene to hexyl benzene is not too large – about 1.5 kcal/mol – but you are already seeing backbone movements. What is perhaps a bit more reassuring than this observation is that some of the discrete states are mirrored in lysozyme structures found in the PDB - but the authors looked at 121 structures to substantiate the result. Not the kind of numbers you would expect to find in the PDB for your typical novel drug discovery target.

The conclusions of the paper are a bit discomforting for at least two reasons. Firstly as mentioned above, drug designers often assume constancy or smooth and minor side chain changes in protein conformation when testing congeneric ligands in lead optimization. It’s quite clear that this is always a bit of a gamble: if something as simple as a change in molecular weight could lead to such divergent changes, what would small but important changes or reversals in polarity do? And then one also starts wondering how much weird or divergent SAR could potentially be explained by such unexpected backbone conformational changes.

Secondly, these kinds of changes pose a real problem for molecular modelers. As the paper says, you would need to go to pretty long MD (molecular dynamics) simulations or more radical protein modeling to look at backbone changes; even today, modeling backbone changes by either physics-based methods (like MD) or knowledge-based techniques (like Rosetta) is both less validated and more computationally expensive.

Lastly though, this study is another example of why drug discovery is hard even at a basic scientific level. Countless factors thwart the best intentions of drug designers at every stage, and uncertainty in predicting protein backbone conformational changes must rank pretty high on that list.

A molecular modeler to his beloved (medicinal chemist)

With all apologies to W. B. Yeats

Original: "I BRING you with reverent hands
The books of my numberless dreams;
White woman that passion has worn
As the tide wears the dove-gray sands,
And with heart more old than the horn
That is brimmed from the pale fire of time:
White woman with numberless dreams
I bring you my passionate rhyme." Corrupted: "I BRING you with reverent hands Bioactive among my numberless conformations Unstrained structures that my protein has worn As the slide showcases the top-ranked ligands, And with common sense more than docking score                              That is scraped from the muddy waters of sampling Ascendant among my numberless failures You better now hand me my dime."