How credibility spoiled this mini-lecture on statistics

In a recent reply to one of our commentaries (Smits et al., 2015), Domenic Cicchetti gave an interesting mini-lecture on effect sizes, power, and sample sizes. You'll find it here. Though interesting in itself, it is a pity that the full scope of his contribution will largely remain unnoticed given a series of credibility issues with the text itself and his position as the "statistics editor" of Journal of Nervous and Mental Disease. Given that some of my recent research deals with the credibility of online word-of-mouth communication, and because I feel a bit misunderstood as the author of two earlier commentaries to this journal, I will elaborate these credibility issues.

Don't get all psychotic on this paper: Had I (or we) Been A Reviewer (HIBAR)

Academic publishing sometimes displays a strange parallel with advertising. There's the celebrity endorsement by reknown authors, a credibility inference evoked by the academic journal and the assumed thorough peer review, and the sales talk in the paper's abstract.

Original article with some marketing claims added

As a researcher, we should of course know to read beyond that abstract, but a recent Twitter discussion pointed me towards a paper that has a very blatant discrepancy between its sales talk and the article's actual core. The paper appeared to be hard to understand due to a substandard use of statistics and equally bad reporting of the study's methods. This resulted in a "letter-to-the editor" written by Daniel Lakens (@lakens), Stuart Ritchie (@StuartJRitchie) and Keith Laws (@Keith_Laws). The story behind that Letter-to-the-Editor is interesting in its own sake? Be sure to read about it on Daniel's blog.

Below, you'll find a summary of our most important issues with the original paper and a list of other issues that were not included in the letter to the editor. I guess that for many students (irrespective of their field of interest) it would be a good excercise to go through the paper and find at least some of these errors themselves. Indeed, as I already said in the Twitter discussion, the paper would not pass as a Master's thesis (at least, I hope so). This, of course is troublesome with regard to the paper itself, but also concerning the review process that should be able to reject such a write-up.

Shame on the authors, reviewers and editor for sloppy reporting of data. http://t.co/9n2JXJLrV4 HT @Keith_Laws Wouldn't pass as a MA thesis
— Tim Smits (@TimSmitsTim) January 14, 2014

Food plating preferences and academic peer review

I recently wrote a letter to the editor of Acta Paediatrica, commenting on an earlier published article. That original article investigated an interesting intervention parents can easily do at home to increase their children's liking of food platings, as such potentially increasing their appetite. That original article, by Zampollo, Griffin, Wansink and Shimizu, can be read here.

There are, however, a few issues with the statistics and the interpretation of these statistics, as I explained in the commentary, which can be found here.

What you hear about research

Academia is often criticized for being a too slow innovation force, with research being either too fundamental and abstract or specific but outdated by the time it gets published. This is certainly the case in more applied disciplines like marketing communication and advertising research. While I will not contest this observation, I will try to make clear in this blogpost that the slowness of academia should not be replaced by mediocre research from non-academic research agencies. Of course, everybody is free to conduct research but non-academic research is rapidly gaining impact in society and this could be troublesome if it is treated just as if it was a piece of research that went through the complete scientific publication process.