Having already read Peter Gøtzsche’s Dødelig medicin og organiseret kriminalitet: Hvordan medicinalindustrien har korrumperet sundhedsvæsenet. Art People, 2013, this book did not bring so much new. However, it did present things better than Gøtzsche did. To be fair, he focused mostly on proving that the farma industry are organized criminals. I agree, but the science is more interesting than reading about 100 different cases of farma companies cheating and getting fines.
If you’re a nerd, you might think: these files are electronic;
they’re PDFs, a type o f file specifically designed to make sharing
electronic documents convenient. Any nerd will know that if
you want to find something in an electronic document, it’s easy:
you just use the ‘find’ command: type in, say, ‘peripheral
neuropathy’, and your computer will find the phrase straight
off. But no: unlike almost any other serious government docu
ment in the world, the PDFs from the FDA are a series of photo
graphs of pages of text, rather than the text itself. This means
you cannot search for a phrase. Instead, you have to go through
it, searching for that phrase, laboriously, by eye.
Easily solved by OCR software.
Sharing data of individual patients’ outcomes in clinical
trials, rather than just the final summary result, has several
significant advantages. First, it’s a safeguard against dubious
analytic practices. In the VIGOR trial on the painkiller Vioxx,
for example, a bizarre reporting decision was made.83 The aim
of the study was to compare Vioxx against an older, cheaper
painkiller, to see if it was any less likely to cause stomach prob
lems (this was the hope for Vioxx), and also if it caused more
heart attacks (this was the fear). But the date cut-off for mea
suring heart attacks was much earlier than that for measuring
stomach problems. This had the result of making the risks look
less significant, relative to the benefits, but it was not declared
clearly in the paper, resulting in a giant scandal when it was
eventually noticed. If the raw data on patients was shared,
games like these would be far easier to spot, and people might
be less likely to play them in the first place.
Occasionally – with vanishing rarity – researchers are able to
obtain raw data, and re-analyse studies that have already been
conducted and published. Daniel Coyne, Professor of Medicine
at Washington University, was lucky enough to get the data on a
key trial for epoetin, a drug given to patients on kidney dialysis,
after a four-year-long fight.84 The original academic publication
on this study, ten years earlier, had switched the primary
outcomes described in the protocol (we will see later how this
exaggerates the benefits of treatments), and changed the main
statistical analysis strategy (again, a huge source of bias). Coyne
was able to analyse the study as the researchers had initially
stated they were planning to in their protocol; and when he did,
he found that they had dramatically overstated the benefits of
the drug. It was a peculiar outcome, as he himself acknowl
edges: ‘As strange as it seems, I am now the sole author of the
publication on the predefined primary and secondary results of
the largest outcomes trial of epoetin in dialysis patients, and I
didn’t even participate in the trial.’ There is room, in my view,
for a small army o f people doing the very same thing, re-
analysing all the trials that were incorrectly analysed, in ways
that deviated misleadingly from their original protocols.
This is the kind of second-order scientist that was described in the paper:
Nosek, Brian A., and Yoav Bar-Anan. “Scientific utopia: I. Opening scientific communication.” Psychological Inquiry 23.3 (2012): 217-243.
This paper is extremely interesting by the way. Read it. Yes, seriously!