Lancet and the perils of peer review

5 Comments

 

When a distinguished journal is caught unawares in its editorial judgment, others will cheer at the burning house. The academic business is a tough one, and at its core is an exaggerated virtue that often conceals core defects. 

Journals in a library. Photo (Maarten van den Heuvel/Unsplash)

Plagiarists, like discreet, discerning murderers, do slip away. Charlatans are celebrated as offering something original when, in truth, the material is merely repackaged. Data can be fabricated, as the journal Science found out in 2006 regarding erroneous claims on stem cell research.

The Lancet has been responsible for some remarkable publications. But it has also had a few frightful slipups. The latest was associated with the drug hydroxychloroquine, of greater interest for the fact that this particular antimalarial drug has refused to leave the tickertape of COVID-19 gossip. At first, it was a great hope, supposedly a shield and cure against the novel coronavirus. US President Donald Trump embraced it, describing it as ‘a gift from heaven if it works’; doctors, certainly in the US, prescribed it, despite having little idea of its efficacy. In Brazil, it was endorsed by President Jaire Bolsonaro even as he expressed doubt about the dangers of COVID-19.

Two other possibilities were also likely: the drug would have no effect at all, or be harmful. This tortured path wound its way to the Lancet when, on 4th June, the journal retracted a paper published the previous month. The premise of that publication was that hydroxychloroquine and chloroquine, its analogue, increased the mortality rate in hospitals for those taking it with COVID-19. With publication still fresh, statistician James Watson commented on the effect size in question. ‘Not many drugs are that good at killing people.’

The New England Journal of Medicine replicated the move regarding a separate study, dealing with cardiovascular disease and the claimed disproportionate effects of COVID-19 upon them. Both articles had relied on data from the same Chicago-based firm, Surgisphere, a company that prides itself on the use of artificial intelligence, machine learning and bid data to assist hospitals in making more informed decisions. ‘We can no longer vouch for the veracity of the primary data resources', claimed the primary authors Mandeep Mehra of Brigham and Women’s Hospital, Frank Ruschitzka of University Hospital Zurich and Amit Patel of the University of Utah.

The Lancet episode is an object lesson in how faith can, at points, substitute hardnosed checking. Surgisphere, run by CEO Sapan Desai, had bold claims, supposedly gathering data from 617 hospitals on six continents including a figure close to 100,000 patient records. This should have been the first warning.

It then transpired that a degree of academic intimacy existed behind the publication: Mehra and Desai were introduced by Dr. Patel ‘through academic and medical circles’. Academic objectivity had been ladled with personal ties. Patel subsequently noted that he was related to Dr. Desai by marriage.

 

'But to blame the Lancet exclusively is ultimately missing the point in the industry. The entire peer review process is simply not as thorough as it ought to be.'

 

Such connections subsequently troubled both journals. But the main problem remained the opaque operations of Surgisphere. Despite defending the integrity of their data, the company refused to share it with the independent institute Mehra had solicited for auditing reasons.

Through this, it should be remembered that the peer review process in the academy can be treacherous at the best of times. The Lancet has fallen foul at points on the matter, publishing material considered defective in terms of methodology. This has led to conservative commentators lobbing the odd grenade at its editor-in-chief Richard Horton.

In 2006, the journal published a piece on mortality rates in Iraq after the 2003 invasion using a cross-sectional cluster sample survey. One graph was distinctly inflated, with a number of violent deaths spiked in a manner at odds with previous estimates. The impression given was that such numbers seemed to compare favourably with those such as the Iraq Body Count, giving a false impression of accurate trends. In truth, it was a mess, with the use of two Y axes in the relevant graph creating, as Michael Spagat noted, ‘the illusion that two curves moving in the same direction at different speeds are in fact moving at the same speed.’ The article was but one facet of a molehill of consternation, with the lead author, Gilbert Burnham, censured by the American Association for Public Opinion Research for being evasive about his methodology and suspended by Johns Hopkins University’s Bloomberg School of ‘privileges to serve as a principal investigator on projects involving human subjects research’.

But to blame the Lancet exclusively is ultimately missing the point in the industry. The entire peer review process is simply not as thorough as it ought to be. During the COVID-19 pandemic, another phenomenon has also taken hold: the idea of preprint publications, rushed off the scientific production line to satisfy a desperate audience keen to grasp the latest trends, the possible cures for a global pandemic. These lack the traditional rigour in so far as they are not peer-reviewed, but as the Lancet process suggests, this is no guarantee against fallibility.

Peer review, be it in the sciences or humanities, is marked by problems. But the Lancet saga, sparking in a time when the experts are ridiculed by sceptical populists keen to diminish danger, has done little to restore confidence. It has led to the necessary observation that science is not free of error, being a process of constant self-correction. ‘It never finds absolute truth,’ reflects science journalist Faye Flam, ‘and it sometimes trips, but it can right itself and move on.’ But the desperation of understanding what best works in combating COVID-19 is so pronounced that the antimalarial angle may still have its day, if only a short one.

 

 

Binoy KampmarkDr Binoy Kampmark is a former Commonwealth Scholar who lectures at RMIT University, Melbourne.

Main image: Journals in a library (Maarten van den Heuvel/Unsplash)

Topic tags: Binoy Kampmark, Lancet, peer review, academic publishing

 

 

submit a comment

Existing comments

But was Dr Horton’s blunder due to a faulty peer review process, the rush to find a COVID-19 cure, or politics? The corruption of science for financial/political gain is not new. The influence of drug companies on researchers/universities led Dr Marcia Angell, editor-in-chief of the New England Medical Journal, to state: “It is simply no longer possible to believe much of the clinical research that is published, or to rely on the judgment of trusted physicians or authoritative medical guidelines.” Horton himself said in 2015: “Much of the scientific literature, perhaps half, may simply be untrue…science has taken a turn towards darkness.” Stephen Pinker wrote: “The analysis of ideas is commonly replaced by political smears and attacks. This poisoning of the intellectual atmosphere…” Jonathan Haidt recognized how a “small range of political outlooks…causes a system-wide problem”, so he founded Heterodox Academy to try to overcome the narrowing of political viewpoints. Horton enthusiastically endorsed a study that refuted the benefits of hydroxychloroquine, a drug which Donald Trump had endorsed, and he urged Americans to “put a president in the White House…who will understand…” Naturally the mainstream media, 90% of whom promoted the Trump/Russia collusion hoax, jumped at Horton’s, ultimate fake, story.
Ross Howard | 17 June 2020


How far-seeing was Juvenal ("Satires", VI)?! In days where we find ourselves frequently exposed to "fake news", the rapid and wide dissemination of which calls much research and media reporting into question, there he was nearly 2,000 years ago posing the question :"Who guards the guardians?" His pertinence was not lost on the school teacher of Cologne's Heinrich Boll, author of "The Lost Honour of Katharina Blum" and "Murke's Collected Silences", who displayed his rejection of fascist authoritarianism and propaganda by ensuring his pupils were well versed in the scathing Roman satirist's politically incorrect poems.
John RD | 17 June 2020


Ross Howard correctly points out that corruption within the scientific research community is not new. It is driven, first, by the lust in some for fame and the benefits it brings such as the doting adulation of others, research grants and invitations with paid 5-star accommodations and international travel and second, by the lust for money. This latter was the great pollutant of virtue and integrity spawned in America with the patenting of medical discoveries and the use of litigation if anyone dared adopt another's ":advance in the care of the sick". Drinker's iron lung was a perfect example of this when the iron lung was not available to many who died with polio in the big epidemics of the 1930s because of Drinker's suing of anyone who tried to apply its principles without paying royalties. The great doctors like Florey with his penicillin and Jonas Salk with his polio vaccine saved countless millions of human lives and didn't earn a solitary penny or cent from their discoveries and wore their fame with genuine humility. Then came the advent of the big Pharmaceuticals paying doctors and researchers for promoting their products and charging the sick outrageous rates for their products, something which in some cases benefitted only those in the ranks of the very wealthy. The death of virtue at the hands of greed and the lust for power has characterised modern Western Civilisation and it is no wonder the world is in the state we see today. Hands on hearts ladies and gentlemen. Altogether now. God love America!!!!
john frawley | 18 June 2020


Imitation (and learning from the mistakes of others) is the sincerest form of flattery. I hope this Country can live up to the name of Clever rather than Lucky and produce our own commensals of Surgisphere and Cambridge Analytica to increase our intellectual and other forms of capital and benefit our economy.
roy chen yee | 18 June 2020


I missed this article and the quality of its comments, all of which I enthusiastically endorse, except to say there's a similar principle at stake that hasn't yet been employed in the discussion about the Synod and which ought to be. Just imagine getting the Synod back on track were we to employ the good offices of an unbiased referee, who, coming from another Church or, as a religious ethicist of note, could offer an opinion that might break the impasse between conservatives and radicals. In these ecumenical times why should that not be possible?
Michael Furtado | 01 July 2020


x

Subscribe for more stories like this.

Free sign-up