Humans beat computers in predicting protein structures

I was going to first describe Rosetta in a post, but a rather cool paper related to the program which appeared in Nature yesterday makes me jump the gun.

In a nutshell, Rosetta tries to predict the structure of proteins from amino acid sequence by inserting fragments from known protein structures and doing many rounds of side chain torsional angle and rigid-body energy optimization. It uses a scoring function to rank the resulting structures that uses empirically derived hydrogen bonding, hydrophobic burial and desolvation terms. Detailed description will have to await the next post since yesterday's paper is not about Rosetta per se.

Instead the paper talks about a program named FoldIt which essentially asks relatively untrained computer gamers to address the protein folding problem. Gamers are asked to tweak, pull, freeze and rotate parts of an incorrectly folded structure to try to twist it into the correct structure. The interface looks like the picture above. Data from a total of 57,000 gamers was pooled. The gamers were driven to solve the problem by the usual incentives of competition and co-operation. Each set of movements would lead to an increase or decrease in a score, with the goal being to find the correct folded structure corresponding to the minimum score. The corresponding set of operations in Rosetta would involve hydrophobic burial, hydrogen bond formation and breakage, helix rotation and other related movements. The project essentially pitted Rosetta versus the gamers.

The results were striking. In a significant number of cases, the gamers actually outdid Rosetta. The reasons are very intriguing and- in an age where computers seem to have unlimited power over our lives- generally testify to the advantages of humans being over computers. For instance in one case, the gamers had to first unravel significant parts of the protein leading to a sharply unfavorable score and then again re-fold it, leading to a correct structure. Rosetta would not attempt the first operation because of the sharp increase in score. This is a classic example of long-term strategy. Unlike computers, humans can make seemingly bad short-term decisions that ultimately lead to good results; we observe this process in many aspects of daily life, from stock market traders taking risks because they see favorable returns later, to politicians making unpopular choices because they think these choices will eventually lead to a popular outcome. Unlike humans though, it is very difficult for a computer program to do long-term planning, and this example illustrates not only the advantages that human intuition can have but also identifies gaps in a program like Rosetta which can possibly be filled.

Another example where the humans outdid the computers was when presented with a set of 10 incorrect structures. Humans generally chose the structure closest to the given structure, whereas Rosetta picked another structure. The main point here is that simple visual clues can sometimes trump complicated decision-making (although they can also mislead). More generally, the results underscored the fact that gut feelings and mere inspection can sometimes lead to successful results.

The one case where the humans did not do as well as Rosetta was in addressing the "classic" protein folding problem, where the challenge was to predict 3D structure from sequence alone. In this case, the sheer amount of conformational space to be searched thwarts success, and there are also no visual cues to guide the process unlike before. The key value of computer approaches which can rapidly pare down the conformational space becomes evident in this example.

So since humans outdid the computer in many cases on the basis of intuition, this must be one super-smart group of biochemists, right? Au contraire! One of the most compelling facts was that most of the gamers in fact not only lacked a formal background or PhD. in biochemistry, but also lacked a formal background in science. Relatively few had college degrees, let alone more advanced ones. For instance there is a profile of a woman in the video below who works in a physical therapist's office, who says that after coming home she feels like a different person when she plays the game. This is great. The examples strikingly illustrates that even untrained humans can possess skills that may be difficult to program into a computer.

It remains to be seen if these results can be extrapolated to large-scale trials, but this very intriguing study perhaps illustrates the general principle that cracking a problem as complex as protein folding is going to require a diverse set of skills, from Monte Carlo searching to gut feelings.



Cooper, S., Khatib, F., Treuille, A., Barbero, J., Lee, J., Beenen, M., Leaver-Fay, A., Baker, D., Popović, Z., & players, F. (2010). Predicting protein structures with a multiplayer online game Nature, 466 (7307), 756-760 DOI: 10.1038/nature09304