Jim Simons: "We never override the computer"

Billionaire-mathematician Jim Simons has been called the most successful investor of all time. His Renaissance Technologies hedge fund has returned an average of 40% returns (after fees) over the last 20 years. The firm uses proprietary mathematical algorithms and models to exploit statistical asymmetries and fluctuations in stock prices to leverage price differentials and make money. 

Simons had made groundbreaking contributions to algebraic topology before founding Renaissance, and his background enabled him to recruit top mathematicians, computer scientists, physicists and statisticians to the company. In fact the company actively stays away from recruiting anyone with a financial or Wall Street background.

I've been enjoying the recent biography of Simons, "The Man Who Solved the Market", by Gregory Zuckerman. But there's an interesting video of a Simons talk at San Francisco State University from 2014 in which he says something very intriguing about the models that Renaissance builds:

"The only rule is that we never override the computer. No one ever comes in any day and says the computer wants to do this and that’s crazy and we shouldn’t do it. You don’t do it because you can’t simulate that, you can’t study the past and wonder whether the boss was gonna come in and change his mind about something. So you just stick with it, and it’s worked."

It struck me that this is how molecular modeling should be done as well. As I mentioned in a previous post, a major problem with modeling is that it's mostly applied in a slapdash manner to drug discovery problems, with heavy human intervention - often for the right reasons, because the algorithms don't work great - obscuring the true successes and failures of the models. But as Simons's quote indicates, the only way to truly improve the models would be to simply take their results at face value, without any human intervention, and test them. At the very minimum, "simulating" historical human intervention is going to be pretty hard. So the only way we'll know what works and what doesn't is if we trust the models and let them rip through. As I pointed out though, in most organizations experimenters are simply not incentivized, nor are there enough resources, to carry out this comprehensive testing. 

Jim Simons and Renaissance can do it because 1. They have the wisdom to realize that that's the only way in which they can get the models to work and 2. They have pockets that are deep enough so that even model failures can be tolerated. Most drug discovery organizations, especially smaller ones, presumably can't do 2. But they could still do it in a limited sense in a handful of projects. What's really necessary though is 1. and my concern is that we'll be waiting for that even if we have the resources to do 2.