Data in dozens of clinical trials may have been ‘deliberately falsified’, says senior doctor

A doctor from Devon has suggested that data in some clinical trials, published in respected anaesthetic and general medical journals, could have been “deliberately falsified”. 

John Carlisle, consultant anaesthetist at Torbay Hospital, developed a screening tool to assess the consistency of baseline data in more than 5,000 randomised controlled trials.

The results of his study, published in Anaesthesia
^[1]
, showed that more than 80 trials had extreme P values, suggesting there was a problem with the data.

Using the tool — essentially a computer programme that looks at whether the distribution of data is too wide or too narrow to be accurate – Carlisle analysed the distribution of 72,261 means of 29,789 variables in 5,087 randomised controlled trials published in eight journals between January 2000 and December 2015.

The journals were Anaesthesia (399), Anesthesia and Analgesia (1288), Anesthesiology (541), British Journal of Anaesthesia (618), Canadian Journal of Anesthesia (384), European Journal of Anaesthesiology (404), Journal of the American Medical Association (JAMA) (518), and the New England Journal of Medicine (935).

Carlisle said: “Randomised controlled trials have been retracted after the data distributions have been calculated as improbable. Most retracted trials have been written by anaesthetists and published by specialist anaesthetic journals. I wanted to explore whether the distribution of baseline data in trials was consistent with the expected distribution.”

In JAMA, there were 12 potentially problematic papers and in the New England Journal of Medicine there were nine.

Carlisle reported a much lower retraction rate in the general medical journals than in the anaesthesia journals.

“This screening tool raises questions about data in some studies, which on full investigation may turn out to involve misinterpretation, statistical error or plain simple mistakes,” he says.

“However, on the basis of previous studies it is likely that some of the data highlighted in this latest research may have been deliberately falsified.”

He adds: “As a minimum it is clear that the reporting of some randomised controlled trials may be seriously flawed.”

The Carlisle screening tool has been used previously to look at the work of specific authors, but this is the first time it has been used to check a large sample of medical papers.