Making speculation official: More on the conservatism of leading science journals

I want to thank everyone for the interesting comments on the last post; I thought it would be best to address them in a new one since my response got too long and spawned too many thoughts for the comments section.

I want to enumerate what I think are the benefits of having a separate 'Speculations' section in journals like Science and Nature because that point perhaps did not come across very clearly. In the context of the present controversial paper on arsenic-associated life, here's what would happen in a world which reveled in speculation. The authors submit the paper to Science. The Science reviewers and editors say that the paper is interesting but that the extraordinary claims are not supported by extraordinary evidence. Nonetheless, they would be quite happy to publish it in their brand new 'Imaginings' section as food for thought for other researchers. They ask the authors to tone down their conclusions and present the paper as a set of observations with some possible interpretations; either in a preliminary findings section or a speculation section. Now someone in the comments section suggested that the authors would have rejected Science's offer in such a case. But let's give them the benefit of doubt. While by no means ecstatic, the authors grudgingly accept Science's offer. The paper now looks much more tentative and its conclusions are much more modest. It proudly features as one of the first inaugural articles in 'Imaginings'. But here's the other good thing that happens: NASA and the authors now resist the temptation to present and sensationalize the work in a press conference before publication because of course it's a little embarrassing to hype a paper explicitly marked as speculative. Everyone is happy; the reviewers, the authors, Science and the public. The media will of course still hype the paper but that's pretty much a constant anyway. Generally speaking, the evil stepmother disintegrates in a blinding flash of light, the princess marries the prince in a glade surrounded by furry creatures and everybody lives happily ever after. The End, for now at least.

I agree that this is an ideal scenario. But it has a much higher probability of being played out if Science sported an explicit section on speculation. When work being presented is speculative, both the public and the reviewers are more forgiving of its incompleteness and the authors and sponsoring agencies don't (or at least should not) feel as tempted to hype it. It appears in print exactly the way it should; as a very intriguing set of observations and experiments that deserves closer scrutiny and nothing more. Any possible earth-shattering implications can wait.

There was a thought that journals should actually become more conservative because of the increasing instances of fraud that have been reported during the last few years. The general direction of this kind of thinking is sound, but I don't think it will help scientific progress at all. Fraud in scientific publishing will continue at a minimum ambient level irrespective of whether journals are conservative or not; it's just human nature. The only way journals could significantly crack down on fraud is if they become ultra-conservative. But this would be a disaster since along with fraud it would lead to the filtering out of too many promising novel ideas. The occasional admission of fraud is a burden we have to bear for publishing the boldest flights of imagination. The best thing however is that we don't have to worry too much about the problem at all; as I mentioned earlier, the beauty of science is that it is usually an incredibly efficient self-correcting process. Unlike the rogue agent from The Matrix, fraud does not stick around for too long to cause havoc. If anything, the universal presence of blogs and online information sharing now ensures that fraud is much more swiftly recognized and dealt with than before; many recent cases can attest to this fact. If the world earlier depended on one Neo to save itself, we now have several who are up to the task.

This brings us to another point in the comments section. Some people pointed out that the proliferation of blogs and other online avenues have now actually provided more opportunities than ever to speculate, and we need not depend on elite journals for doing this. While this is undoubtedly true, I wish it solved the problem. I wish that speculation on blogs was as respected as speculation in Nature. But we don't live in that ideal world yet. For whatever reason, journals like Science and Nature are now worshipped even more than what they were before. In my own field of organic chemistry for instance, you would find many pathbreaking papers published in relatively low-impact journals in the sixties and seventies, but hardly any more. Sadly, the obsession of impact factors and the constant pressure to publish and perish have put the premier journals on a pedestal. There are unfortunately many who think that only papers in these journals are worth taking seriously. This is extremely regretful (and is definitely a topic for a separate post) but sadly it's reality. Unless this reality changes, speculation would become respectable only if it's published by Nature and Science. As was pointed out, the Annals of Improbable Research has published ideas that first make us laugh and then make us think. If you look at some of the papers in the journal which have bagged the notorious IgNobel prize, they are actually quite well-supported by data and statistical analysis. Yet regrettably, we will have to wait for at least a few generations before anyone takes the Annals as seriously as Cell or PNAS. One of my main points in the last post was that we need to make speculation not just easier but more respectable and official again. And for better or worse, for now it's going to become respectable and official only if the top journals give it a public platform.

Ultimately, what purpose will all this serve? Many of the benefits have been described; as a commentator succinctly mentioned in the earlier post, it would give the publication of preliminary ideas an official sounding board. The way the present system is set up- and the recent example makes it clear- scientists are just going to be dissuaded from publishing tentative, bold observations and ideas because of the impending public backlash. But the commentator also pointed out another important dividend; the process would perhaps make the true nature of science clear to a public which is too often fed information in black and white sound bytes.

There are rules for doing, interpreting and publishing science, just like there are rules for how to raise children. And just as the rules for raising children wonderfully break down in the face of reality, so do the rules of actual scientific research. Real science is as messy as real child rearing. It's only fair that the public knows about this process.

The beauty of it is that it all comes together in the end. The baby turns into a fine young man or woman, and science continues to flourish.

Note: The comments section makes it clear that we need to distinguish between two kinds of articles, those suitable as "Preliminary Results" and those suitable as "Speculation". The two kinds may certainly overlap; the arsenic paper would thus be primarily in a "Preliminary Results" section but the hypothesis about arsenated DNA backbones would put it into a "Speculations" section and in this case the speculation would not be toned down but kept in.