Beyond Binary: Why Null Hypothesis Significance Testing Should No Longer Be the Default for Statistical Analysis and Reporting

Null hypothesis significance testing (NHST) is the default approach to statistical analysis and reporting in marketing and, more broadly, in science. Despite its default role, however, NHST has long been criticized by both statisticians and applied researchers, including those within marketing.

The most prominent criticisms relate to NHST’s dichotomization categorization of results as “statistically significant” versus “statistically nonsignificant.” This binary treatment of results, using p-values or otherwise, loses information, can be misleading, and prevents meta-analyses which is what science is really all about!

In a new article published in The Journal of Marketing, my colleagues Blakeley B. McShane, John G. Lynch, Jr., Robert Meyer, and I propose a fundamental shift in statistical analysis and reporting in marketing and beyond. In fact, we propose abandoning NHST as the default approach altogether as statistical (non)significance should never be used as a basis to draw conclusions, or as a filter to select what data to prioritize when making decisions.