The drama in programmatic online advertising

In times of adversity, online advertising routines fail. Drastically

The above example of an inappropriate display ad is just one in a range of similar advertising anomalies following the crash of Germanwings flight 4U9525 in March 2015. In the hours and days after such events, newspapers witness an increased online traffic to their websites and reports about the disasters in particular. Unfortunately, such reports are often accompanied by inappropriate advertising. Similarly, users of social media often encounter those ads at those times. Conversations on social media about such inappropriate ads often blame either the websites (i.e., news outlets) or the advertisers. But whose fault is it anyway? How should we understand this? And most important, how should different stakeholders within the online advertising business prepare themselves to avoid such mistakes in the future?

Happy Birthday, New Coke (°March 23, 1985)

Tuesday April 23rd of 1985 the Coca-Cola Company launched New Coke. This still remains one of the major cases in the marketing literature. It often ranks first in all-time lists of marketing fails (such as on marketwired, or in Matt Haig's book Brand Failures). Apart from the obvious failure (for the short version, see below; for the longer version, see Wikipedia), I think it is also a good demonstration of the intangible part of brand value, which I will explain below.

How credibility spoiled this mini-lecture on statistics

In a recent reply to one of our commentaries (Smits et al., 2015), Domenic Cicchetti gave an interesting mini-lecture on effect sizes, power, and sample sizes. You'll find it here. Though interesting in itself, it is a pity that the full scope of his contribution will largely remain unnoticed given a series of credibility issues with the text itself and his position as the "statistics editor" of Journal of Nervous and Mental Disease. Given that some of my recent research deals with the credibility of online word-of-mouth communication, and because I feel a bit misunderstood as the author of two earlier commentaries to this journal, I will elaborate these credibility issues.

A science preregistration revolution? Not yet.

Within a short span of only 24 hours or so, I received two opposing types of news concerning preregistered science. Its basic idea is that with good preregistration of researchers' intents, science would be more objective by reducing researcher degrees of freedom (e.g., HARKing: Hypothesising After Results are Known; p-hacking: using different analyses, data, variables until the hypothesized effect is found).

• Yesterday, May 19 2014, the journal Social Psychology published an issue stacked with preregistered replication studies. Not all of these replications were (fully) successful, including our own attempt to replicate the idea that a single exposure to music and an object would transfer your attitude about the music to that object (i.e., Gorn’s 1982 concept of single exposure musical conditioning). If you want to read more about this issue and the preregistration idea in (social) psychology, I would suggest Chris Chambers’ blog in the Guardian. For a more critical voice, John Bohannon expressed the idea (or fear) that preregistration relates to academic bullying.

Though I participated in the preregistration replication effort for Social Psychology I am still in doubt about the exact status of preregistration. I believe it is a good and even necessary tool for honest replication attempts. I am not sure whether we should use it for all of our research, as long as we are sure to refrain from HARKing our exploratory analyses. In any case, I think we should move on in honest reporting. Preregistration will only foster scientific integrity if we link it with specific reporting guidelines. Sure, a researcher can also do exploratory analyses that were not preregistered. But we should not conceal them with persuasive wording suggesting that all was planned (cf. HARKing, p-hacking)

"Preregistration will only foster scientific integrity if we link it with specific reporting guidelines"

Sadly, this is not an opinion everybody shares…

• The Lancet finally gave me an editorial decision for the Letter I submitted, addressing precisely the lack of good preregistration and the persuasive framing used by authors to cover up their HARKing. See an earlier blogpost for the (astonishing) details of this. Lancet decided not to publish the submitted commentary. Of course I can understand that all kinds of editorial choices have to be made, but I did not receive a true explanation for the rejection. I sure hope they do acknowledge the seriousness of the issue I tried to address.  

Above, I already talked about HARKing and p-hacking as a serious threat to science integrity. For the non-informed: compare them to a sports game where the rules change. Claiming a soccer victory not because you scored more goals but because you had higher ball possession percentages increases your degrees of freedom to a non-acceptable extent. Your degrees of freedom are limited meaning that when one team wins the game, we understand that this could be due to different facts but at least their is something common to all wins, namely scoring more goals. In (empirical, quantitative) science, an implicit set of (bad) rules has long been dominating "the game". Problem is that science does not pretend to be a mere game. So we should have good rules on interpretation of findings. The problem with ill-executed preregistration of research is that researchers can start pretending to follow the rule book while, in fact, they don't.

That is a serious threat to the preregistration idea – as demonstrated in the Morrison article that I discussed in that commentary that was rejected by Lancet. Such ill-suited practice conceals the researcher degrees of freedom and concealing is worse than not caring about it, I believe.

Submitted, but rejected commentary

References for the commentary:

1 Morrison AP, Turkington D, Pyle M, Spencer H, Brabban A, Dunn G, Christodoulides T, Dudley R, Chapman N, Callcott P, Grace T, Lumley V, Drage L, Tully S, Irving K, Cummings A, Byrne R, Davies LM, Hutton P. Cognitive therapy for people with schizofrenia spectrum disorders not taking antipsychotic drugs: a single-blind randomised controlled trial. Lancet 2014 

2 http://www.controlled-trials.com/ISRCTN29607432/morrison

3 Morrison AP, Wardle M, Hutton P, Davies L, Dunn G, Brabban A, Byrne R, Drage L, Spencer H, Turkington D. Assessing Cognitive Therapy Instead Of Neuroleptics: Rationale, study design and sample characteristics of the ACTION trial. Psychosis 2013; 5(1): 82-92.

4 Schulz KF, Altman DG, Moher D. Protocols, probity, and publication. Lancet 2009; 373: 1524.

5 Glasziou P, Altman DG, Bossuyt P, Boutron I, Clarke M, Julious S, Michie S, Moher D, Wager E. reducing waste from incomplete or unusable reports of biomedical research. Lancet 2014; 383: 267-76.