Seriously Misleading Network Meta-analysis in Lancet of Acceptability of Depression Pills

In 2018, Andrea Cipriani et al. published a network meta-analysis of trials of depression pills in The Lancet. The authors included both head-to-head comparisons of drugs and comparisons of drugs with placebo. They ranked the drugs according to their effect and acceptability.

The acceptability was measured as drop-out for any reason. This is a highly relevant outcome measure in placebo-controlled trials of depression pills. When the patients decide whether it is worthwhile to continue in the trial, they make a judgment about if the benefits they perceive exceed the harms.

Cipriani et al. showed the odds ratios for drop-out for each of the 21 drugs they included in their review but did not provide a summary estimate. Based on their data, I calculated a summary odds ratio of 0.98 (95% confidence interval 0.95 to 1.01, P = 0.22, fixed effect model). Thus, on average, they did not find a significant difference in drop-out between active drug and placebo.

However, when my research group analyzed the placebo-controlled trials, the result was markedly different. Significantly more patients dropped out on the pills than on placebo, risk ratio 1.08 (1.03 to 1.13). After exclusion of three particularly flawed trials, the risk ratio for the remaining 26 trials was 1.12 (1.07 to 1.18). Thus, we found that 12% more patients dropped out on a depression pill than on placebo (P < 0.00001).

We included trials of duloxetine, fluoxetine, paroxetine, sertraline, and venlafaxine. When I restricted Cipriani et al.’s meta-analysis to these five drugs, the odds ratio was similar to the one for all 21 drugs, 0.97 (0.92 to 1.01, P = 0.15).

There are two main reasons why Cipriani et al.’s meta-analysis is seriously misleading. Firstly, they mainly based their data on published trial reports, whereas we exclusively used the voluminous clinical study reports drug companies have submitted to drug regulators. We discovered that some patients lost to follow up early in the trials were not accounted for and that some patients with missing efficacy or harms data were not included in the analyses. We included these patients.

Secondly, as I and others have demonstrated, Cipriani et al.’s meta-analysis is essentially a garbage-in, garbage-out exercise. I called my criticism “Rewarding the Companies that Cheated the Most in Antidepressant Trials” and showed, based on other data, that some of the results could simply not be true.

Other researchers from my center demonstrated that Cipriani et al. had ignored or underestimated the biases in the trials. Only one of the 522 trials they included had a low risk of bias. Furthermore, the outcome data they reported differed from those in the clinical reports in two-thirds of the 19 trials they examined.

Other reviews based mainly on published data have also reported misleading results. For example, a review of 40 trials reported a risk ratio of 0.99 (0.88 to 1.11) for drop-out when paroxetine was compared with placebo.

It is a futile exercise to rank depression pills after how good they are based on flawed trial reports and—most importantly—when the patients prefer to be treated with a placebo.

All depression pills should be avoided. They have no relevant effect on depression; they double the risk of suicide, including in adults; and the patients prefer psychotherapy, which halves the risk of suicide and has relevant effects on depression.

***

Mad in America hosts blogs by a diverse group of writers. These posts are designed to serve as a public forum for a discussion—broadly speaking—of psychiatry and its treatments. The opinions expressed are the writers’ own.

Leave a Comment