Friday, March 14, 2014

The pitfalls of pre-registration: The Morrison et al CBT paper

There has been quite a fuss about the recently published (online prepublication) Lancet paper on unmedicated cognitive therapy for schizophrenia patients (see below for a reading list of posts about the study). This paper reports on a longitudinal (9 to 18 months) follow-up study of a treatment group and a control group and its basic claim is that the cognitive therapy treatment group fares better than the control group. Though I contributed to a letter to the editor on a study in the same domain that had tremendous reporting flaws, this research is well outside of my discipline. Still, the criticism on the Lancet paper sparkled my interest in it and even for an outsider, there are strange things going on with said manuscript. In this post I will focus on the peculiar timeline of things that shows one of the greatest pitfalls of scientific pre-registration. Though I am greatly in favor of preregistration for a lot of studies, I do think that the mass application of preregistration might lead to new problems. One evident problem is that merely refering to a preregistered study protocol does not imply that the protocol was sensible or that the researchers actually followed it.

The summary
It took quite some investigative academic journalism, I think, but the summary of the study's timeline is presented below. Plain grey timeline moments are based on assumptions following the different pieces of evidence that are available; those with a colored border are based on one of the four public sources.
Timeline - click to enlarge