When a distinguished journal is caught unawares in its editorial judgment, others will cheer at the burning house. The academic business is a tough one, and at its core is an exaggerated virtue that often conceals core defects.
Plagiarists, like discreet, discerning murderers, do slip away. Charlatans are celebrated as offering something original when, in truth, the material is merely repackaged. Data can be fabricated, as the journal Science found out in 2006 regarding erroneous claims on stem cell research.
The Lancet has been responsible for some remarkable publications. But it has also had a few frightful slipups. The latest was associated with the drug hydroxychloroquine, of greater interest for the fact that this particular antimalarial drug has refused to leave the tickertape of COVID-19 gossip. At first, it was a great hope, supposedly a shield and cure against the novel coronavirus. US President Donald Trump embraced it, describing it as ‘a gift from heaven if it works’; doctors, certainly in the US, prescribed it, despite having little idea of its efficacy. In Brazil, it was endorsed by President Jaire Bolsonaro even as he expressed doubt about the dangers of COVID-19.
Two other possibilities were also likely: the drug would have no effect at all, or be harmful. This tortured path wound its way to the Lancet when, on 4th June, the journal retracted a paper published the previous month. The premise of that publication was that hydroxychloroquine and chloroquine, its analogue, increased the mortality rate in hospitals for those taking it with COVID-19. With publication still fresh, statistician James Watson commented on the effect size in question. ‘Not many drugs are that good at killing people.’
The New England Journal of Medicine replicated the move regarding a separate study, dealing with cardiovascular disease and the claimed disproportionate effects of COVID-19 upon them. Both articles had relied on data from the same Chicago-based firm, Surgisphere, a company that prides itself on the use of artificial intelligence, machine learning and bid data to assist hospitals in making more informed decisions. ‘We can no longer vouch for the veracity of the primary data resources', claimed the primary authors Mandeep Mehra of Brigham and Women’s Hospital, Frank Ruschitzka of University Hospital Zurich and Amit Patel of the University of Utah.
The Lancet episode is an object lesson in how faith can, at points, substitute hardnosed checking. Surgisphere, run by CEO Sapan Desai, had bold claims, supposedly gathering data from 617 hospitals on six continents including a figure close to 100,000 patient records. This should have been the first warning.
It then transpired that a degree of academic intimacy existed behind the publication: Mehra and Desai were introduced by Dr. Patel ‘through academic and medical circles’. Academic objectivity had been ladled with personal ties. Patel subsequently noted that he was related to Dr. Desai by marriage.
'But to blame the Lancet exclusively is ultimately missing the point in the industry. The entire peer review process is simply not as thorough as it ought to be.'
Such connections subsequently troubled both journals. But the main problem remained the opaque operations of Surgisphere. Despite defending the integrity of their data, the company refused to share it with the independent institute Mehra had solicited for auditing reasons.
Through this, it should be remembered that the peer review process in the academy can be treacherous at the best of times. The Lancet has fallen foul at points on the matter, publishing material considered defective in terms of methodology. This has led to conservative commentators lobbing the odd grenade at its editor-in-chief Richard Horton.
In 2006, the journal published a piece on mortality rates in Iraq after the 2003 invasion using a cross-sectional cluster sample survey. One graph was distinctly inflated, with a number of violent deaths spiked in a manner at odds with previous estimates. The impression given was that such numbers seemed to compare favourably with those such as the Iraq Body Count, giving a false impression of accurate trends. In truth, it was a mess, with the use of two Y axes in the relevant graph creating, as Michael Spagat noted, ‘the illusion that two curves moving in the same direction at different speeds are in fact moving at the same speed.’ The article was but one facet of a molehill of consternation, with the lead author, Gilbert Burnham, censured by the American Association for Public Opinion Research for being evasive about his methodology and suspended by Johns Hopkins University’s Bloomberg School of ‘privileges to serve as a principal investigator on projects involving human subjects research’.
But to blame the Lancet exclusively is ultimately missing the point in the industry. The entire peer review process is simply not as thorough as it ought to be. During the COVID-19 pandemic, another phenomenon has also taken hold: the idea of preprint publications, rushed off the scientific production line to satisfy a desperate audience keen to grasp the latest trends, the possible cures for a global pandemic. These lack the traditional rigour in so far as they are not peer-reviewed, but as the Lancet process suggests, this is no guarantee against fallibility.
Peer review, be it in the sciences or humanities, is marked by problems. But the Lancet saga, sparking in a time when the experts are ridiculed by sceptical populists keen to diminish danger, has done little to restore confidence. It has led to the necessary observation that science is not free of error, being a process of constant self-correction. ‘It never finds absolute truth,’ reflects science journalist Faye Flam, ‘and it sometimes trips, but it can right itself and move on.’ But the desperation of understanding what best works in combating COVID-19 is so pronounced that the antimalarial angle may still have its day, if only a short one.
Dr Binoy Kampmark is a former Commonwealth Scholar who lectures at RMIT University, Melbourne.
Main image: Journals in a library (Maarten van den Heuvel/Unsplash)