Field of Science

Datasets in Virtual Screening: starting off on the right foot

ResearchBlogging.org
As Niels Bohr said, prediction is very difficult, especially about the future. In case of computational modeling, the real grist of value is in prediction. But for any methodology to predict it must first be able to evaluate. Sound evaluation of known data (retrospective testing) is the only means to proceed to accurate prediction of new data (prospective testing).

Over the last few years, several papers have come out involving the comparisons of different structure-based and ligand-based methods for virtual screening (VS), binding mode prediction and binding affinity prediction. Every one of these goals if accurately achieved could lead to the saving of immense amounts of time and money for the industry. Every paper concludes that some method is better than other. For virtual screening for example, it has been concluded by many that ligand-based 3D methods are better than docking methods, and 2D ligand-based methods are at least as good if not better.

However, such studies have to be conducted very carefully to make sure that you are not biasing your experiment for or against any method, or comparing apples and oranges. In addition, you have to use the correct metrics for evaluation of your results. Failure to do either of these and other things can lead to erroneous or/and inflated or artificially enhanced results leading to fallible prediction.

Here I will talk about two aspects of virtual screening; choosing the correct dataset, and choice of evaluation metric. The basic problems in VS are false positives and false negatives and one wants to minimize the occurrence of these. Sound statistical analysis can do wonders for generating and evaluating good virtual screening data. This has been documented in several recent papers, notably one by Ant Nicholls from OpenEye. If you have a VS method, it's not of much use randomly picking a random screen of 100,000 compounds. You need to choose the nature and number of actives and inactives in the screen judiciously to avoid bias. Here are a few things to be remembered that I got from the literature:

1. Standard statistical analysis tells you that the error in your results depends upon the number of representatives in your sample. Thus, you need to have an adequate number of actives and inactives in your screening dataset. What is much more important is the correct ratio of inactives to actives. The errors inherent in choosing various such ratios have been quantified; for example, with an inactive:active ratio of 4:1, the error incurred is 11% more than that incurred by a theoretical ratio of infinite:1. For a ratio of 100:1 it's only 0.5% more than with infinite. Clearly we must use a good ratio of inactives to actives to reduce statistical error. Incidentally you can also increase the number of actives to reduce this error. But this is not compatible with real-life HTS where actives are (usually) very less, sometimes not more than 0.1% of the screen.

2. Number is one thing. The nature of your actives and decoys is equally important; simply overwhelming your screen with decoys won't do the trick. For example, consider a kinase inhibitor virtual screen in which the decoys are things like hydrocarbons and inorganic ions. In his paper, Nicholls calls distinguishing these decoys the "dog test", that is, even your dog should be able to distinguish them from actives (not that I am belittling dogs here). We don't want a screen that makes it too easy for the method to reject actives. Thus, simply throwing a screen of random compounds at your method might make it too easy for your method to screen actives and mislead.

We also don't want a method that rejects chemically similar molecules on the basis of some property like logP or molecular weight. For example consider a method or scoring function that is sensitive to logP, and suppose it is supplied with two hypothetical molecules which have an identical core and a nitrogen in the side chain. If one side chain has a NH2 and another one is N-alkylated where the alkyl is butyl, then there will be a substantial difference in logP between the two, and your method will fail to recognise them as "similar", especially from a 2D perspective. Thus, a challenging dataset for similarity based methods is one in which the decoys and actives are property-matched. Just such a dataset has been put together by Irwin, Huang and Shoichet- this is the DUD (Directory of Useful Decoys) dataset of property-matched compounds. In it, 40 protein targets and their corresponding actives have been selected. 36 property-matched decoys for every active have been chosen. This dataset is much more challenging for many methods that do well on other random datasets. For more details, take a look at the original DUD paper. In general, there can be different kinds of decoys; random, drug-like, drug-like and property-matched etc. and one needs to know exactly how to choose the correct dataset. With datasets like DUD, there is an attempt to provide possible benchmarks for the modeling community.

3. Then there is the extremely important matter of evaluation. After doing a virtual screen with a well-chosen dataset and well-chosen targets, how do you actually evaluate the results and put your method in perspective? There are several metrics but until now, the most popular way of doing this is by calculating enrichment and this is the way it has been done in several publications. The idea is simple; you want your top ranked compounds to contain the most number of actives. Enrichment is simply the fraction of actives found in a certain fraction of screened compounds. Ideally you want your enrichment curve to shoot up at the beginning, that is you want most (ideally all) of the actives to show up in the first 1% or so of your ranked molecules. Then you compare that enrichment curve to a curve (actually a straight line) that would stem from an ideal result.
The problem with enrichment is that it is a function of the method and the dataset, hence of the entire experiment. For example, the ideal straight line depends on the number of actives in the dataset. If you want to do a controlled experiment, then you want to make sure that the only differences in the results come from your method, and enrichment introduces another variable that complicates interpretation. Other failings of enrichment are documented in this paper.

Instead, what's recommended for evaluating methods are R.O.C curves.

Essentially, R.O.C curves can be used in any situation where one needs to distinguish signal from noise and boy, is there a lot of noise around. R.O.C curves have an interesting history; they were developed by radar scientists during World War 2 to distinguish the signal of enemy warplanes from the noise of false hits and other artifacts. In recent times they have been used in diverse fields; psychology, medicine, epidemiology, engineering quality control, anywhere where we want to pick the bad apples from the good ones. Thus, R.O.C curves simply plot the false positive (FP) rate against the true positive (TP) rate. A purely random result gives a straight line at 45 degrees implying that for every FL you get a TP- dismal performance. A good R.O.C curve is a hyperbola that shoots above the straight line, and a very useful measure of your method's performance is the Area Under the Curve (AUC). The AUC needs to be prudently interpreted; for instance an AUC of 0.8 means that you can discriminate a TP by assigning a higher score to it than to a FP in 8 out of 10 cases. Here's a paper discussing the advantages of R.O.C curves for VS and detailing an actual example.

One thing seems to be striking. The papers linked here and at other places document that R.O.C curves may currently be the single-best metric for measuring performance of virtual screening methods. This is probably not too surprising given that they have proved so successful in other fields.

Why should modeling be different? Just like in other fields, rigorous and standard statistical metrics need to be established for the field. Only then will the comparisons between different methods and programs commonly seen these days be valid. For this, as in other fields, experiments need to be judiciously planned (including choosing the correct datasets here) and their results need to be carefully evaluated with unbiased techniques.

It is worth noting that these are mostly prescriptions for retrospective evaluations. When confronted with an unknown and novel screen, which method or combination of methods does one use? The answer to this question is still out there. In fact some of the real-life challenges run contrary to the known scenarios. For example consider a molecular screen from some novel plant or marine sponge. Are the molecules in this screen going to be drug-like? Certainly not. Is this going to have the right ratio of actives to decoys? Who knows (the whole point is to find the actives). Is it going to be random? Yes. If so, how random? In all actual screenings, there are a lot of unknowns out there. But it's still very useful to know about "known unknowns" and "unknown unknowns", and retrospective screening and the design of experiments can help us unearth some of these. If nothing else, it indicates attention to sound scientific and statistical principles.

In later posts, we will take a closer look at statistical evaluation and dangers in pose-prediction including being always wary of crystal structures, as well as something I found fascinating- bias in virtual screen design and evaluation that throws light on chemist psychology itself. This is a learning experience for me as much or more than it is for anyone else.


References:

1. Hawkins, P.C., Warren, G.L., Skillman, A.G., Nicholls, A. (2008). How to do an evaluation: pitfalls and traps. Journal of Computer-Aided Molecular Design, 22(3-4), 179-190. DOI: 10.1007/s10822-007-9166-3

2. Triballeau, N., Acher, F., Brabet, I., Pin, J., Bertrand, H. (2005). . Journal of Medicinal Chemistry, 48(7), 2534-2547. DOI: 10.1021/jm049092j

3. Huang, N., Shoichet, B., Irwin, J. (2006). . Journal of Medicinal Chemistry, 49(23), 6789-6801. DOI: 10.1021/jm0608356

Expelled! No nuance allowed

I was reminded of the title of Ben Stein's ludicrous new movie on creationism when I came across BBC's recalling the Three Mile Island which took place on this day in 1979. The reporting makes it clear how reporters sweetly shy away from any subtleties or scientific nuance, which unfortunately turn out to be details that matter. I do understand that almost everyone was more ignorant or fearful of anything in nuclear in 1979 and to be fair the BBC does note later that nobody died from the accident, but what strikes me is how they used the blanket-word "radiation" so many times in the article without in any way qualifying what it means. This is quite similar to the irrational gut reaction many people have when they hear the word. To recapitulate:

1. "Radiation" bathes us from head to toe throughout our life. Background radiation is hundreds of times more than any radiation accrued from living near a nuclear reaction. It's even more than radiation possibly escaped from a nuclear reaction in an accident if the reactor has a containment structure.

2. There is no proof that low-level "radiation" causes cancer; in fact there is proof that it may be generally good for life. Plus, almost everybody who reports such studies fails to consider the relative risks from "radiation" compared to other causes. As the well-known scientist James Lovelock notes in his The Revenge of Gaia (2006), it is misleading to say that 40,000 extra people will die earlier because of some radiation. The question is, how much earlier? As he says, if people are going to die on an average a week earlier because of some radiation, compare that to hundreds of thousands that would die instantaneously if the giant dam they live next to bursts open? How many people die years earlier because of heart disease? How many lives are prematurely cut short because of road accidents? Yet we pristinely accept these risks in daily life. People have no problem living near dams on the Yangtze when the risk they pose is much higher than that from "radiation".

3. And of course, the simple scientific error of not noting what the radiation consists of is commonplace. Every college kid knows that radiation can consist of many different particles- alphas, betas, gammas, neutrons- that each have a vastly different effect on living tissue. Plus, the isotope that emits the radiation is crucial; uranium is vastly preferable to strontium. But strontium has a smaller half life....and so on.

In case of TMI, it was immensely bad timing since the accident was preceded by the scare-mongering movie The China Syndrome starring Jane Fonda, a well-known irrational anti-nuclear spokesman. In it, the reactor core is feared to be melting away through the earth to China, a preposterous scenario even by fictional Hollywood standards. (although some of the Amazon reviewers don't seem to get this) The point is, it is pure fear-mongering to kick around words like "radiation" and "radioactive steam". Sadly, the scenario has not changed too much, and I doubt if most people will do a much more responsible job of reporting such an accident if it happens today. I feel miffed in thinking that a similar accident today will essentially impact the public's perception of nuclear energy almost the same as TMI. I do hope I am wrong. But then, the media has a proven track record of not caring about subtlety and nuance when reporting on science (or many other things for that matter). Unfortunately, they are the "respectable" sources who reach the most people and who most people rely on for their daily dose of "reality".

On anonymous blogging

There's a fair amount of drama going on at The Chem Blog, with "Kyle Finchsigmate" announcing first that he was going to retire, and then demoting (advancing?) himself to "emeritus blogger" to continue writing in his rather inimitable style.

I enjoy reading Kyle's blog and have always done so, and I doubt how many people would have the gumption to express themselves in the way that he does and still preserve a significant amount of intellectual substance and solid information. At the same time, I do find myself cringing once in a while at his novel use of expletives. That said, at least for me, the entertainment gains from reading his blog outweigh the displeasure I may sense in myself on reading what my inner "offense-meter" thinks is a disproportionate amount of profanity or irreverence. But that's not the point. The point is that on the whole, Kyle has always been honest about what he thinks, and that is something that everybody should value.

However, being a star does have its problems and I think Kyle is suffering from typical celebrity troubles. First of all, when you write in the risque language that he does, there is going to be a relatively higher proportion of people feeling offended; it's an inevitable collateral effect. Secondly and more simply, as Kyle's celebrity status improved and hundreds of people started reading his blog, the sheer statistical number of people that could be offended went up. With Kyle's cover blown, this became a particular problem.

Anonymous blogging does have its pitfalls. The advantage of being gung-ho and writing whatever you want about whoever you want, even if done honestly, can be outweighed by the possible consequences if discovered. In this information age, I think it's hard to stay completely anonymous for long. As your blog becomes more and more popular, the chances of getting outed naturally increase (Consider the 'Fake Steve Jobs' who was actually surprised that his his outing took that long).

That's one reason I decided to have a non-anonymous blog. Right at the beginning, I decided that I would not write about other scientists, my own work or my own advisor. Of course it's not very easy because it's human nature to be constantly tempted to point fingers at others and engage in gossip, as well as put in plugs for one's own work. At the very least, it's quite easy to be full of praise for someone, but in the world of science even such an action can have unintended personal consequences. It also seems clear to me that you face a larger risk even with run-of-the-mill writing if you work for someone who is high-profile and famous. Speaking for myself, the only time I have criticised someone is in case of blatant fraud or obviously questionable work where many others have raised similar issues.

But blogging non-anonymously and responsibly and yet effectively is nevertheless possible, and is demonstrated by one of the best examples of such writing that I know- Derek's In The Pipeline which is one of the most widely read scientific blogs on the internet. Especially being in industry, I would think that Derek carries a larger than usual burden of making sure he does not become a liability. I believe he has succeeded admirably in always writing entertainingly and quite provocatively, and yet avoided getting personal or offensive. His posts stimulate and inform without offending (at least largely; there is always going to be somebody who can get offended by anything). Paul Bracher's now sadly defunct blog was another example of provocative and yet gentle writing; it was interesting how even that gentle writing got Paul into trouble with commentators sometimes.

In any case, that's Derek. Kyle's different, and why not? In the end blogging is a personal activity and personal choice. Even if I am writing for others, it's my blog, and I should have the freedom to express my views the way I want to. It's Kyle's choice how much risk to bear. It's inevitable that he shoulders more risk with his particular style, and in my personal opinion, he could sometimes tread a little more softly as a small price to pay for less risk. But again, it's his blog, and if he changes himself too much for the sake of propriety, then it wouldn't be Kyle Finchsigmate's blog anymore, would it?

The Gate

Image Hosted by ImageShack.us


Image Hosted by ImageShack.us


There it was, completely nondescript. Nobody could ever tell what the room was used for more than half a century ago. But then I thought that that sounded apt; after all, nobody was supposed to know in the first place what went on in there. To that extent, it perfectly served its intended purpose. An unexceptional but heavy iron gate right next to it probably was the only object that possibly enforced the gravity of the situation.

I looked around and saw the usual tourists ambling along and taking photos of the road on which it was located, of the plaza just a block away with its numerous Indian craftsmen peddling their pretty artistic wares on the road, and of the impressive church that seemed like an anomaly among the low-lying, colorful adobe stores and restaurants. Nobody was taking photos of the door and the small room inside, and there wasn't any reason why anybody should have. What was so special about it? It blended perfectly into its surroundings. The room hosted a shop that sold pretty dresses, bed-sheets and candles. It was right next to a well-known local restaurant and a winery. The entire set of shops and restaurants resided along a contiguous structure under one roof, with a courtyard inside with clear signs that the buildings had seen many such evenings; a sign located inside the courtyard indicated that the buildings were constructed in the early 1600s, sometime when this small town was one among many provincial seats of the vast Spanish Empire then straddling the globe.

The place was so nondescript that in 1943 and 1944, many young men missed it and walked straight past by. These were young men from diverse backgrounds. Many had been picked right out of universities for their particular talents. They were from every part of the country, from Princeton to Berkeley, from Chicago to New York. They were men and women of different dispositions, religious sentiments or the lack thereof, married or single, with an average age of 25 years. Many of them spoke English with a heavy accent. But all of them had one thing in common; all had been asked to report to 109 East Palace Avenue in downtown Santa Fe, where I was now standing. None of them knew what would happen next. All they had been asked to do was to take a train to Lamy and report to 109 East Palace. There, they would be given further instructions by Dorothy McKibben. She would send them to a place that no one had ever heard of.

Dorothy McKibben herself had never wildly guessed that she would be at 109 East Palace. A remarkably plucky, courageous and determined woman, she had come to Santa Fe in the 1920s stricken with TB. At that time, the clear air and bright sunlight were deemed to be salutary for TB patients, and many affected by the terrible ailment came to the historic town with the expectation that the disease would either break them or make them. It did make Dorothy, who had already lost two sisters to TB. But fate had more in store for the Smith college graduate. She fell in love and married a former World War 1 soldier who got stricken with Hodgkin's disease, then an incurable condition. He died, and the grief-stricken Dorothy with her baby son decided to come to her beloved Santa Fe again to spend the rest of her days. There, she seamlessly blended into the town life and became close friends with most of the townsfolk. When World War 2 began, she lost her job as an accountant at an Indian trading company due to personnel shortage. She had been offered another job and was seriously considering to take it, when a friend of hers asked her whether she would be interested in working for the government as a secretary. The job would pay a little better, and it would last at least as long as the war.

Dorothy was summoned to the Hotel La Fonda, a couple of blocks from East Palace, to discuss the job with her friend. There in the lobby she was still undecided about it, when a man in a porkpie hat and the bluest eyes she had ever seen ambled over and talked to her about it for a few minutes. She was astonished when she found herself accepting the job after 10 minutes of talking to the man. That was probably not surprising. J. Robert Oppenheimer had that effect on everyone, whether janitor or Nobel laureate; his powers of persuasion were legendary. The job he offered Dorothy was supposed to be top-secret and she could not tell anyone about it. She would be an important person for a very important government project. It would be run out of 109 East Palace. It would be Dorothy's job to manage personnel.

But over time, her job description expanded. She became much more than a secretary. She would be secretary, personnel manager, mother goose for depressed souls, officiator of marriages and agony aunt for couples in love, friend and confidant of some of the most brilliant minds of the century, and in the end, gentle but firm and efficient supervisor for the front office of the Manhattan Project at Los Alamos, a place that did not exist on the map. Dorothy McKibben became the gatekeeper of the atomic bomb. For three years, she ran the front end of the most secret scientific project in history from 109 East Palace, from inside that small room in front of which I stood. It was her job to guide the brilliant and clueless young men and women to Los Alamos after they arrived in Santa Fe, often dazed and lost, not knowing where to go. Everyone- and everything including material- without exception who worked on the bomb passed through the doors of 109 East Palace, because it was Dorothy's job to issue them the secret passes that would open the gates of the secret lab high up in the mountains.

It would also be her job to make sure the project remained secret in Santa Fe. To this end, she would always be careful never to address anyone as "Professor" or "Doctor" or even mention the word "scientist". With the deluge of European accents from world-famous European émigré scientists that flooded the town, it was hard to do this to say the least. But Dorothy managed it very efficiently. Her calm demeanor and absolute dedication won her the respect and endearment of Oppenheimer and his band of prima donnas, along with a dozen top military officials not known for exuding warmth. 109 East Palace and her home became focal points for social occasions, where famous scientists could let off steam and spill their frustrations about living in an inaccessible, cloistered, alternately freezing and scorching place that nobody knew about. Even if she was not officially told, by the end, Dorothy became aware of what was going on in the mountains through her close association with the thousands that worked there. Her home and 109 became places where scientists and their spouses could find some solace from the tortuous implications of what they were doing. Domestic spats, complaints about living conditions, strained relationships, baby epidemics, shopping troubles and gossip about other scientists' wives were all directed towards Dorothy's sympathetic ear.

Now, as I stood in front of the small enclosure, my mind wandered back more than half a century to what went on there. For the first few weeks, Oppenheimer worked out of 109 East Palace with Dorothy. As he chain smoked and pondered the stark reality of what he was making, his dazzlingly erudite mind must have straddled enormous moral and philosophical dilemmas. Enrico Fermi would visit the project and make calls to Los Alamos from this small room; later he moved permanently to the secret lab. He would stand there, twirling a pencil in his hands while he offered advice on some obscure calculation, his eyes twinkling, sometimes staring quizzically at Dorothy as if he expected her to pipe in with a clever suggestion. Prankster Richard Feynman would have certainly stepped foot in there more than once. He would have engaged in his usual tomfoolery while all the time shouldering a tragic burden as his wife lay dying in a sanitarium in Albuquerque. The wise and great Niels Bohr who, wresting with this unusual paradox of creating a weapon so powerful that it might abrogate war, visited Dorothy several times and endeared himself to her. The steadfast Hans Bethe, the volatile Edward Teller, the intrepid Otto Frisch who worked out nuclear fission with his aunt Lise Meitner, even the spy Klaus Fuchs, all had to pass through the gates of 109 East Palace on their way to Los Alamos. With these extraordinary souls, Dorothy undertook a journey that nobody would ever forget, a journey that would change history and the future. A journey that fundamentally changed the nature of man's animosity towards his fellow human beings.

But then I was suddenly jolted back to reality. I had been standing in front of 109 East Palace for almost an hour. It was dark. I became painfully aware of the cold, biting, clear air of New Mexico. High up in the mountains, the ponderosas must have been casting silhouettes, shadows that Robert Oppenheimer must have retreated to in moments of quiet introspection. Around me, the tourists had started dwindling. The shop owner closed the room and lit up the pretty dresses in the windows with a soft, glowing light. I took some more photos and started walking back to my hotel. Beside 109, 111 beamed with restaurant visitors engaged in casual and lively banter.

At that moment, 109 East Palace looked like 111's and the other rooms' poor cousin. But it occupies a unique and special place in history of singular value, one that should be commemorated at that spot but sadly is not. It is a testament to a remarkable woman, a remarkable group of men and women, and a truly remarkable time that changed our world. Legendary names- Oppenheimer, Bohr, Fermi, Bethe, Rabi, Teller, Feynman to name only a few- passed through that door. All of them sincerely believed that their work would save the world, a world gone half mad in the throes of inhumanity. Their fond hope was that the weapon they were creating would be so terrifying that it would, in Bohr's words, propel humanity into a completely new situation that could not be resolved by war. The implications of their work woefully turned out to be more complicated. But one thing was for sure. Among other things, none of them would ever forget 109 East Palace.

Neither will I.

Southwestern interlude

"Prediction is very difficult...especially about the future" - Niels Bohr

Finally back from The Land of Enchantment. The guys at OpenEye are awesome and immensely smart. I learnt more in this mentally exhausting conference than in any until now. There was much modeling, crystallography, screening, mathematical similarity, statistics and whatnot. In the past few weeks, I have had to deal with previously arcane (to me) statistical concepts like ROC curves which are fascinating. The conference indicated that there is much success we still have by chance, and one of the best ways to tackle molecular complexity in all its forms is by starting simple, for example in trying to predict solvation energies for simple organic molecules, still a highly challenging endeavor. Prediction is difficult indeed, especially about the simple things.

On the whole, New Mexico is indeed very special (and those who say that only Santa Fe is nice haven't really been around Albuquerque). I lived in a splendid studio apartment in a lively and charming old-age home, ate enough green chile to last a lifetime, visited the National Atomic Museum, admired turkey quilts in the Pueblo Indian museum, kicked around dust in the Santa Fe Plaza, bought a Sidney Harris cartoon t-shirt, and got a free one from OpenEye with the quote above, a favourite.

The air is thin, dry and crystal clear. One of the consequences of this is that...let me just say that a little alcohol goes a long way, especially for someone who hardly ever drinks. The sun is brighter and lights up your thoughts. The miles upon miles of pinon-covered hillocks and the snow-capped Sandia and Sangre de Cristo mountains in the distance are marvelous. The ski slopes of Santa Fe make you happily giddy. The landscape is like nothing I had seen before, and it seems to be wholesome for arty, existentialists types. Robert Oppenheimer once said that his two great loves were physics and desert country. He thought it was a pity they could not be combined, until he discovered Los Alamos. New Mexico seems to provide one of the few avenues for such a heady combination.

SAMPLing out in the desert

I am in the absolutely charming Santa Fe, NM for the CUP IX OpenEye meeting. OpenEye's ROCS and related programs have become quite popular with the modeling and drug discovery community in the last couple of years. This year, as I had written about before, they have a challenge named SAMPL for all groups across the country involved in drug discovery modeling- provided with a few ligands and targets, do virtual screening, pose prediction and binding affinity prediction. It was a great opportunity for our small group of four grad students and postdocs to take part in the challenge.

The OpenEye meeting is always a spirited, extremely informal and funny meeting. The titles of talks makes it clear what kind of atmosphere enlivens the event. There are some really smart and interesting people here, heavily involved with all aspects of modeling and drug discovery. The Hotel La Posada, with its quaint spread-out low-lying adobe hut-like rooms adds to the charm.

The talks have been enlightening, and especially today's talks on crystallography and structure were very good. The most useful were those which focused on errors in crystal structures and crystallographic models- and there are a lot of them out there. The most terrific talk was by Gerhard Kleywegt from Uppsala. He talked about the myriad number of errors existing in crystal structures in the PDB, including misplaced ligands in little or no electron density (protein crystallographers can be especially negligent about ligand fitting, which unfortunately is of greatest interest to medicinal chemists), neglect of water molecules, incorrect conformations, and of course the rash of strained structures, poor resolutions and bad B-factors and R-factors. There are programs such as Afitt which can refine these structures. But a lot of people use crystal structures as they are and don't take account of such inadequacies. Some crystal structures are completely wrong, and yet exist in the PDB. Kleywegt's talk was very funny and informative, and gave a very good reason for why anyone who uses PDB crystal structures should be more than cautious in using them as they are.

Tomorrow there are some great talks on lead optimization and statistical techniques in methods evaluation. And Wednesday is going to be devoted to a brainstorming session on the SAMPL challenge. It would be very interesting to compare our own work with that of others. All in all, it is getting to be a refreshing scientific experience.

Otherwise the city of Santa Fe is very scenic and historical. Buildings have to be built to certain historical Spanish-Indian construction guidelines. And just a short walk from our hotel is 109 East Palace, which was the Manhattan Project's front office, where new arrivals were "inducted" and passes were issued to them so that they could travel to the secret city of Los Alamos, a place that did not exist on the map. Today the room is just a shop with no inkling of what went on there. Little do people strolling around know the great men that passed through those doors and walked that street- Oppenheimer, Fermi, Bethe, Teller, Frisch, Bohr...the list goes on.

A rash of molecular personalities

ResearchBlogging.org
Just like human beings, molecules have personalities. And just like human beings, they display those personalities best when they react to a stimulus. For a medicinal chemist, one such stimulus is HTS where one can identify different flavors of molecules through their interaction with protein targets. But this is not always done, and quantitative analysis of molecules in HT screens is lacking. Clearly such analyses will help to identify compositions of such screens and give insight into future screens.

In his latest offering, Brian Shoichet does just that. He and his group set out to identify essentially every molecular character from a colorful screen of about 70000 molecular personalities applied to ampicillin resistant beta-lactamase. Their results are surprising.

Out of 70000, about 1274 showed activity. Shoichet has already extensively documented the alarming frequency of aggregate-forming molecules in common HTS screens. It's a very substantial contribution from his laboratory. In this case, 1204 (95%) of the 1274 turned out to be inhibiting the enzyme through non-specific aggregation. This can be found out by adding detergent, which breaks up the aggregates and gets rid of the spurious activity.

So now there were 70 detergent-insensitive compounds. How many of these were true, reversible binders? 25 of these were beta-lactams, and since they are covalent modifiers of the enzyme and known chemical scaffolds, they were not considered further. So out of the remaining ones, 25 were re-synthesized and were found to be false positive through lack of reproducible activity. There were now 20 non beta-lactams. Out of these 9 were again found to be aggregators- the earlier screen had skipped them because of low detergent concentration.

That left 12 molecules. After some more scrutiny, these were all found to be covalent, irreversible modifiers of the enzyme. A neat and simple trick can be used to identify covalent modification; mass spectra of the modified enzyme are clearly different from the apo enzyme.

So how many non-covalent, reversible inhibitors of beta-lactmase were found? Zero.

To shed some more light on this strange phenomenon, the authors turned to docking with DOCK. To make sure the program can identify reversible binders, some known binders were seeded among the unknown binders. After docking and observing that the first 500 hits contained the known binders, 16 out of these 500 compounds were selected based on structural diversity and then assayed. Interestingly, two among these compounds were found to inhibit the enzyme at IC50 values of >100 µM. No wonder the initial screen had missed these phthalimide culprits- the highest concentration in the screen was 30 µM.

In other studies, they also did some SAR on the hits and verified the docking poses by obtaining crystal structures. There are other interesting details in the paper.

But even if the study did not unearth reversible, potent, novel binders, it is of course still very instructive. It tells us about the variety of beasts existing in HTS. It also again sheds light on docking as a valuable complement to HTS. In this case, 70000 compounds may been too less for assaying, and 30 µM must have been two low a threshold for finding hits. In any case, higher thresholds for testing are limited by practical difficulties, including material availability and solubility. But what is valuable is that given due effort, we can identify compounds that give false positive results in screens through novel mechanisms- in this case by aggregation (detected by detergent addition) and by covalent modification (detected by mass spec)

There are clearly some notorious and dirty candidates in HTS screens- more than everyone would be comfortable with- and this study provides a good model for being on one's guard and seeking to identify them as thoroughly as possible. When we lay down the red carpet, we want only the cream of the crop, not asses disguised as lions.

Babaoglu, K., Simeonov, A., Irwin, J.J., Nelson, M.E., Feng, B., Thomas, C.J., Cancian, L., Costi, M.P., Maltby, D.A., Jadhav, A., Inglese, J., Austin, C.P., Shoichet, B.K. (2008). Comprehensive Mechanistic Analysis of Hits from High-Throughput and Docking Screens against ÃŽ²-Lactamase. Journal of Medicinal Chemistry DOI: 10.1021/jm701500e

Space: The Violent Frontier?

A couple of days ago, the US sent up a missile to blow up a "rogue" satellite. China had also done the same thing a year or so ago. These actions seem like ominous preludes to a possible arms race in space, the last thing the world wants.

In the latest issue of Pragati, Adityanjee has an article that exhorts India to develop its own ASAT (anti-satellite) system in response to these actions by the US and China. While developing such a system might be good insurance and a future bargaining chip, the first and most important thing we all need to do is keeping pushing for an international treaty to ban weapons in space. Not only will a failure to do this lead to a possible new Cold War, but it can also render space inhospitable for peaceful technologies, an event that will be disastrous for countries that currently use satellites for weather forecasting and precipitation for example.

Mike Moore, who is a previous editor of the Bulletin of the Atomic Scientists, has written a new book on the history and current status of attempts by the US to weaponise and militarise space. As in many other cases, the US- no friend of treaties for quite some time now- is unique in having vetoed and blocked attempts to forge international treaties to ban weapons in space. This is probably not too surprising considering the attempts by the US since the 19060s (when they were developing a purported system to oust Chinese ballistic missiles) to the 1980s (age of the infamous Star Wars) to the 2000s (the Bush administration's obsession with National Missile Defense). True to other traditions, the US has constantly underscored its sovereign right to exceptionalism when the rest of the world thinks otherwise. The US satellite blowup comes close on the heels of renewed efforts of China and Russia to push for an international treaty to ban space weapons.

In his book, Moore documents how President Eisenhower made spirited efforts to stop an arms race in space. However, every administration since the Reagan administration has vetoed attempts by other space-faring countries to negotiate such treaties. The Pentagon's love affair with GPS-guided precision weapons in the 90s fueled ambitions to weaponise space. Again, it's the US that is leading the world into a dangerous era and is interested in unilaterally pursuing belligerent aims. It seems to be still living in Cold War mode. As Moore says, China and Russia (and presumably India) have many more important problems to tackle and spend money on than building a space weapons capability. However, they can, and will, build this capability if they see the US constantly trying to do so.

The US in fact has a golden opportunity right now to preserve its superiority in weapons technology. The situation reminds me of the early days of the nuclear arms race, when an exceptional opportunity to preserve the US superiority over Russia in nuclear arms was lost because of politicking by right wing hawks and threat inflation specialists. After that, Russia soon caught up and it was too late. Similarly, now is the time for the US to talk to other nations and sign a space-weapons ban, thus preserving and possibly sealing its current technological advantage.

One of the central points of missile defense that I have often made in other posts, is that it is almost assuredly going to fail against ballistic missiles, a point which should have been emphasized in the Pragati article. This points needs to be constantly emphasized because like some annoying virus, it keeps infecting and enamoring the minds of US and world officials in every successive administration, in spite of its proven lack of feasibility. Shooting down ballistic missiles realistically has always been a pipe dream harboured by zealous government officials, and the fallibility of this has been demonstrated time and time again by distinguished scientists and other officials. Any attempt to build an anti-ballistic missile system is a huge waste of money, time and talent, and as an added insidious side-effect, it breeds hostility in other nations, something that has already happened because of the US National Missile Defense system. Missile defense should rankle the hearts of democracy and peace-lovers, libertarians and economic conservatives.

Shooting down satellites is another matter, and unfortunately easier than shooting down ballistic missiles. But as Moore points out in his book, one of the many effects of such an exchange will be an amplification of debris in low-earth orbit, debris that will likely make it impossible to use satellites for peaceful purposes, including missions to other planets in the solar system. And of course, it will add perhaps irreversibly to the hubris-laden image that the US has in the world right now.

Every attempt should be made by all space-faring countries to push for an international treaty banning any kind of weapons in space. But sadly, it's the US again that is posing the biggest impediment to the forging of such a consensus. The next President should make it a priority to sign such a treaty, and Obama has indicated that he might be interested. Any attempt by the US to develop a space weapons capability will lead to a dangerous arms race with Russia, China, India and others, involving huge expenditures and wasted efforts. It will contribute to an already deeply dividing feeling of international resentment and animosity. But perhaps most importantly, it will send out a signal that space, that ultimate refuge that is supposed to be the equal sovereign right of every human being on the planet, can be belligerently conquered and manipulated by a few nations. After that, everything will be up for grabs.

"The name's Inhibitor. Kinase Inhibitor"

ResearchBlogging.org
Since we were on the topic of kinases a few days back, it is worth pointing out a recent paper by Alex Aronov, Mark Murcko and others at Vertex on kinase-likeness.

One of the most common questions asked by medicinal chemists is about how to identify "privileged scaffolds", either generally among molecules that could bind to proteins, or among a subset of molecules. It would be useful indeed to identify, for example, common scaffolds among kinase inhibitors, which could help to pick out putative kinase inhibitors from a large dataset of molecules.

Aronov's group sets out to do just this. They do a statistical study in which they identify rings and linkers that are commonly found among kinase inhibitors. For this, they looked at three different databases. The important one was a database of molecules obtained from GVK Biosciences, which contains inhibitors from J. Med. Chem. published between 1959-2003. Naturally, there is a preponderance of kinase inhibitor-like molecules in the later publications. As controls, they also used two other databases which were underrepresented in kinase inhibitors.

Now any kinase lover worth his or her salt should not be surprised at their database-fishing results- they find that amino (NH) linkers are among the most common, along with rings containing one or two nitrogens. Cyano groups are also pretty common in side chains. They find that biaryl amines are extremely common among kinase inhibitors, another not very surprising conclusion. Anilines, pyridines and pyrimidines as well as pyrroles are also better represented among kinase inhibitors.

But the authors do something more valuable and generally applicable. After making these observations, they come up with what they call the "2-0 rule" for kinase likeness prediction. The rule needs putative kinase inhibitors to have an aromatic pyridine like (N) or pyrrole like (NH) nitrogen, or an aniline NH and/or a cyano group. Simply put, the rule for kinase-likeness is represented as:
∑ (N aromatic) + ∑ (NH aromatic) > 2
OR
∑ (Ar-NH) + ∑ (R-CN) > 0
When they applied this simple rule to the GVK database, they found that four out of five kinases inhibitors were flagged with this rule. In case of the kinase-inhibitor underrepresented database obviously, there were very few compounds that emerged through this rule.

The paper also went one step further and documented the locations in the ATP-binding pocket of kinases where these commonly identified fragments fit and found most probable locations for the fragments. However, it should be remembered that allosteric kinase inhibitors can have some of the most valuable functions for selectively targeting kinases, and this rule may not help in identifying them. But in any case, not too many allosteric inhibitors are known relative to ATP-site binding inhibitors.

Such a rule, while not completely exhaustive, can serve some valuable purposes. For one thing as the authors demonstrate, it can be used to fish out kinase inhibitor-like molecule hits/leads from a database of existing drugs. This is of course a well-known strategy, to use a drug prescribed for one ailment to treat a different one, and one of the big advantages of this is that because the drug has been on the market its PK/ADME-TOX properties are already well-established. But it's not easy to do in general, and it's nice to have a structural rule that could weed out such drugs and "redeploy" them for targeting kinases. In this case, the fished-out drugs that originally bound to totally different targets did show broad-spectrum kinase activity experimentally.

In general, such a rule can be used both ways. More commonly, it can be used to fish out kinase inhibitors. But in a more novel application, it can be used to exclude kinase inhibitors and then screen other moelcules for novel scaffolds, albeit with a lower hit rate.

There is a caveat in this study which the authors note; the scaffolds for kinase inhibitors probably also reflect the ease of access to these inhibitors through existing synthetic chemistry. For example, one of the reasons anilines and biarylanilines are predominant among kinase inhibitors is because they are easy to synthesize through coupling reactions like Suzuki coupling. Ready availability of starting materials also drives the structures of kinase inhibitors. Thus, the existing structures should not be taken as representative of the most commonly possible scaffolds for kinase inhibition. The search should certainly continue.

Aronov, A.M., McClain, B., Moody, C.S., Murcko, M.A. (2008). Kinase-likeness and Kinase-Privileged Fragments: Toward Virtual Polypharmacology. Journal of Medicinal Chemistry DOI: 10.1021/jm701021b

Jim Watson, 'seed'y guy

So it seems that some people are visibly outraged at Jim Watson's appointment on the SEED group board of directors, which among other things runs the well-respected Scienceblogs, featuring many well-known blogs like Pharyngula. According to these people, Watson should be fired and not allowed to sit on any such committee because of his racist remarks.

With all due respect to the sentiments of these folks, I have to say I disagree.

I completely agree with what a ditzy old fool Watson was, and I have disparaged his actions in a previous post. But no matter how much you may hate the man, there are two things; firstly, that should still not take anything away from his past achievements and his capabilities (something that some have unnecessarily tried to do), and secondly, it also does not preempt him from offering his services in some valuable way to other organizations. Relatively few science administrators have the kind of experience that Watson has.

In fact, I now think that removing him from his age-old post at Cold Spring Harbor Laboratory itself was uncalled for. This was not because he does not deserve to be marginalised, but we have to agree that his direction of CSHL has nothing to do with his racist remarks. I completely agree how difficult it must be to have such a man running your institution, but to be honest, I believe that keeping him in his position and subjecting him to daily scorn might have been a better punishment for him than just removing him from front of everyone's eyes.

And no matter what he has said, I think it sounds a little totalitarian to say that Jim Watson should not be allowed to sit on any committee or on any board at any time in his life, ever. He is going to get his due ridicule and punishment from the way people will be treating him from now on. But how about this- we can give him that due punishment and also take advantage of his knowledge of science.

Two birds with one stone I say.