Frank Zenker

Scholars, as opposed to scientists, do normally not conduct empirical studies. But their own work regularly cites—and thus fully relies on—published results that others provide. In the social and behavioral sciences, the main strategy to generate such empirical results has remained null-hypothesis significance testing (NHST). Recent attempts, however, broadly failed to replicate allegedly well-established result, giving rise to an ongoing “replication crisis.” The crisis entails that NHST retains too many errors; otherwise many more such effects should have replicated successfully. This makes trusting and citing even results that top-tier journals publish a true difficulty. On this background, we advocate the research program strategy (RPS)—employing both Frequentist NHST-elements and elements of Bayesian hypothesis-testing (BHT)—as superior to NHST. Data-simulation shows that RPS’s six steps (leading from a discovery against a random model to a statistically verified hypothesis) retain far fewer errors, and thus deserve far greater trust, than NHST- or BHT-results. We also show how to estimate the expectable error proportion among published results. A background in inferential statistics is helpful, but not necessary to profit from this talk. It is based, among others, on these publications:

Witte, E.H., and Zenker, F. (2017b). From discovery to justification. Outline of an ideal research program in empirical psychology. Frontiers in Psychology, 8, 1847 (ISSN 1664-1078).

Krefeld-Schwalb, A; Witte, E.H., and Zenker, F. (2018). Hypothesis-testing demands trustworthy data—a simulation approach to statistical inference advocating the research program strategy. Frontiers in Psychology, 9, 460 (ISSN 1664-1078).

Witte, E.H., and Zenker, F. (2018). Data replication matters, replicated hypothesis-corroboration counts. (Commentary on “Making Replication Mainstream” by Rolf A. Zwaan, Alexander Etz, Richard E., Lucas, and M. Brent Donnellan). Behavioral and Brain Sciences, 41, e120 (ISSN: 0140-525X).

Frank Zenker, Lund University, Sweden