On the impact of social media and Twitter on scientific peer review

I am very pleased to note that an my article on the impact of social media and especially of blogs and Twitter on peer review in chemistry in particular and science in general has just come out in a special issue of the journal 'Accountability in Research'. This project has been in the works for almost a year and I have spent quite a bit of time on it. The whole issue is open access and it was made possible by the dedicated and generous efforts of my colleague and friend, the eminent historian of chemistry Jeff Seeman. I am privileged to have my article appear along with those by Roald Hoffmann, William Schulz, Jeffrey Kovac and Sandra Titus. All their papers are highly readable.

Here in a nutshell is what I say. I have had a very dim view of Twitter recently as a vehicle for cogent science communication and rational debate, but in this article I find myself full of praise for the medium. This sentiment has been inspired by the use of Twitter in recent times for demolishing careless science and questioning shoddy or controversial papers in the scientific literature. In my opinion the most spectacular use of Twitter to this effect was Nature Chemistry editor Stuart Cantrill's stark highlighting of 'self-plagiarism' in a review article published by Ronald Breslow in JACS in 2012 (I hold forth on the concept of self-plagiarism itself in the article). As I say in my piece, to my knowledge this is the first and only instance I know in which Twitter - and Twitter alone - was used to point our errors in a paper published in a major journal. If Cantrill's analysis was not a resounding example of peer review in the age of social media, I don't know what is.

I have had a much more consistent and positive views of blogs as tools for instant and comprehensive peer review, and thanks to the vibrant chemistry blogosphere that I have been lucky to be a part of for almost eleven years, have witnessed the true coming of age of this medium. There is no doubt that peer review on blogs is here to stay, and in my article I address the pitfalls and promises inherent in this development. One of the most important concerns that a naive observer would have regarding the use of blogs or Twitter for peer review is the potential for public shaming and ad hominem attacks - and such an observer would find plenty of recent evidence in the general Twittersphere to support their suspicions. Yet I argue that, at least as far as the limited milieu of chemistry blogs is concerned, the signal to noise ratio has been very high and the debate remarkably forward-thinking and positive; in fact I think that, by and large, chemistry blogs could serve as models of civil and productive debate for blogs on more socially or politically contentious topics like evolution and climate change. I am proud to be part of this (largely) civil community.

What I aim to do in this piece is to view the positive role of Twitter and blogs in effecting rapid and comprehensive peer review through the lens of three major case studies which would be familiar to informed observers: the debacle of 'arsenic life', the fiasco of hexacyclinol and the curious case of self-plagiarism in the Breslow 'space dinosaurs' review. In each case I point out how blogs and Twitter were responsible for pointing out mistakes and issues with the relevant material far faster than official review ever could and how they circumvented problems with traditional peer review, some obvious and some more structural. The latter part of the review raises questions about the problems and possibilities inherent in the effective use of these tools, and I muse a bit about how the process could be made fairer and simpler.

Due to the sheer speed with which blogs and social media can turn our collective microscopes on the scientific literature and the sheer diversity of views which can be instantly brought to bear on a contentious topic, there is no doubt in my mind that this new tier of scientific appraisal is here to stay. In my opinion the future of completely open peer review is bright and beckons. How it can complement existing modalities of 'official' peer review is an open question. While I raise this question and offer some of my own thoughts I claim to provide no definitive answers. Those answers can only be provided by our community.

Which brings me to the crux of the article: although my name is printed on the first page of the piece it really is of, by and for the community. Hope there will be something of interest to everyone in it. I welcome your comments.

Angewandte Chemie retracts hexacyclinol paper. Sort of

So it seems that the infamous hexacyclinol saga has been finally put to rest and Angewandte Chemie has retracted the paper. For those chemists who might still be unfamiliar with it, it's not hard to explain: Total synthesis paper published in 2006 with more holes than the vacuum of deep space. Multiple blog postings and papers demolish the claim within months. Journal does not retract the paper for six years.

Well, now the journal has published the retraction. Here's what it has to say:

The following article from Angewandte Chemie International Edition, “Total Syntheses of Hexacyclinol, 5-epi-Hexacyclinol, and Desoxohexacyclinol Unveil an Antimalarial Prodrug Motif” by James J. La Clair, published online on February 9, 2006 in Wiley Online Library (http://onlinelibrary.wiley.com), has been retracted by agreement between the author, the journal Editor in Chief, Peter Gölitz, and Wiley-VCH Verlag GmbH & Co. KGaA. The retraction has been agreed due to lack of sufficient Supporting Information. In particular, the lack of experimental procedures and characterization data for the synthetic intermediates as well as copies of salient NMR spectra prevents validation of the synthetic claims. The author acknowledges this shortcoming and its potential impact on the community.

What I find disappointing about this retraction is that it's just not strong enough in denouncing the paper. It's not just that the procedures were irreproducible or that the supporting information was incomplete, it's that the whole synthesis was essentially...make believe. This was made clear by papers published later (re-synthesizing the natural product and calculating and comparing NMR spectra) which demonstrated beyond any shade of reasonable doubt that whatever was supposedly synthesized in the paper simply couldn't correspond to the structure of hexacyclinol as we know it. 

I think this is an important difference that the retraction does not acknowledge; it's the difference between saying "we think this could be wrong but we can't be sure since we can't reproduce the data" and "we are almost certain this is wrong since independent studies have convincingly demonstrated its utter implausibility".

Update: Carmen Drahl from C&EN has a superb Storify summary of the hexacyclinol saga over the last six years which features some of the blog posts commenting on the debacle. Carmen was also kind enough to post a picture of my cherished hexacyclinol t-shirt which I am still eager to break out; as I said in my email to her, I am still waiting to wear it at a big party where fellow t-shirters get together, laugh with sadistic glee, and mock the scattered bones of hexacyclinol's atomic constituents.

Hexacyclinol: Case Closed

Seems that hexacyclinol is finally dead and buried and the tomb has been sealed. If anyone had any doubt that there was something wrong with the original structure, this paper should resolve it. For those who have had the good fortune of not hearing about hexacyclinol, see the links at the end.

In any case, the present authors from Italy simply calculate H1 chemical shifts and coupling constants for the original disputed structure (1) and the newly proposed and calculated structure (2). Recall that one of the things James La Clair claimed when John Porco synthesized what Scott Rychnovsky felt (Yes, it does sound like one of those convoluted Shakespearean chains of people) was the correct structure was that the two different structures could possibly give rise to the same or extremely similar NMR spectra.

Yes, it's probably true that in the infinitely large universe of organic compounds one could possibly locate two compounds which could give the same NMR spectra. In real life, such a possibility for two complex compounds that look so different is very unlikely. However, the assertion that this could be so is a testable and falsifiable hypothesis and therefore a good one.

In the present paper, the Italian group finally tackles the problem using a set of high-level calculated NMR spectra that include H1 chemical shifts, coupling constants, and even COSY plots at the DFT level. The answers are unambiguous and clear; Porco and Rychnovsky's hexacyclinol gives much smaller errors between measured and calculated parameters compared to La Clair's "hexacyclinol". At the same time, there is some striking similarity between many of the NMR parameters. In the end though, a competent organic chemist would be able to tell that the spectra belong to different structures. As the authors put it rather clearly,

The structure of hexacyclinol is confirmed to be (2). Furthermore, if (1) had been synthesized or was formed from an unforeseen reaction, its NMR spectra are sufficiently different from those of (2) as to guarantee their distinction.

Short of a miracle that the famed Bionic Brothers could spring now, I don't see what Dr. La Clair can do to validate his original proposal and synthesis. This should also again be a question for Angewandte Chemie who published the paper; I am sure all of us are curious to know what happened there.

I have talked on this blog several times before (see below) about computationally calculated NMR parameters coming of age, and this paper should reinforce their tremendous power in resolving confusion between organic structures. H1 chemical shifts can now be predicted to within a remarkable 0.1 ppm accuracy. Hopefully these studies should now encourage the widespread use of the relevant tools in organic chemistry.

Previous posts on hexacyclinol and computational prediction of NMR data: here, here, here, here and here

Update: Also discussed at In The Pipeline and The Chemistry Blog. As mentioned in the comments, I too find it remarkable that Angew Chem has not published some kind of explanatory note by now, after all the blog and news publicity that the issue got.


Reference:
Giacomo Saielli, Alessandro Bagno (2009). Can Two Molecules Have the Same NMR Spectrum? Hexacyclinol Revisited Organic Letters DOI: 10.1021/ol900164a

Hexacyclinol as a test case: ab initio C13 chemical shift prediction

Anybody heard of this natural product called hexacyclinol and how doubts were raised about its synthesis and structure? Kidding obviously. I am going to assume that any organic or related chemist who has not heard of hexacyclinol has not heard of Robert Burns Woodward by default.

Well, in any case, recall that the high point of that deb(acle)ate was Scott Rychnovsky's demonstration by using quantum chemical prediction of C13 chemical shifts that a structure quite different from hexacyclinol fit the C13 NMR data much better compared to JJLC's structure. To do this Rychnovsky used DFT methods and the mpw1pw91 functional which was tried, tested and proven to be a reliable tool for C13 chemical shift prediction by Bifulco and others. (excellent general review here which deals with calculation of both shifts and 2 and 3 bond homo and heteronuclear coupling constants)

The point of value for the organic chemist from the whole exercise was the fact that C13 chemical shift prediction could not just be used to distinguish regioisomers whose identity might be ambiguous but, based on Rychnovsky's analysis, also can be used to correctly assign misassigned C13 peaks. To me this is the greatest benefit of the analysis for the practicing organic chemist.

Henry Rzepa and Christopher Braddock at Imperial College in London have now demonstrated the application of this increasingly valuable method to the correct assignment of some interesting halogenated natural products called obtusallenes. In this case there was ambiguity about the positions of a chlorine and a bromine. The proton chemical shifts were very similar and could not be used to assign the positions. Rzepa and Braddock used the mpw1pw91 functional not just for the chemical shift calculation but also for the optimization. Fast computational power has made this possible now. The bottom line is that average C13 shift deviations are much more for the incorrect regioisomer. Using the method, the authors also re-assigned two ambiguous peaks. In addition, they determine that the 6-31G (d,p) basis set gives some errors for certain functional groups while using the aug-cc-pVDZ basis set (all that's left to say is "warp speed" now) basis set eliminates these errors.

A short, neat demonstration of the increasing value of quantum chemical NMR prediction methods for the practical organic chemist.

Braddock, D.C., Rzepa, H.S. (2008). Structural Reassignment of Obtusallenes V, VI, and VII by GIAO-Based Density Functional Prediction. Journal of Natural Products, 71(4), 728-730. DOI: 10.1021/np0705918