NHST problems and controversies
Channel: Mikko Rönkkö
Date: 2019-07-10

The video discusses the limitations and common misunderstandings about p-values. The video highlights that p-values, while commonly used, are often misinterpreted and do not provide the answers researchers are seeking. Specifically, a p-value doesn't directly indicate the probability of the null hypothesis being true, nor does it measure the probability of the alternative hypothesis being true. Additionally, it is emphasized that p-values cannot determine the logical validity of a hypothesis, as they are based on probabilistic statements rather than absolute truths. The video also points out that a small p-value does not necessarily imply an important or meaningful effect, emphasizing the need to interpret other statistical measures for a comprehensive understanding of the data.

Addressing the misuse of p-values in publication decisions, the video discusses how this practice can lead to biased knowledge due to the preference for studies supporting the hypothesis. To combat this, some journals are de-emphasizing or banning p-values, and there's a shift towards registered reports where study designs are reviewed instead of relying on p-values. The video suggests exploring Bayesian analysis as an alternative, which incorporates prior information into the analysis, offering a different perspective compared to p-values. However, Bayesian analysis is not widely used, partly due to a lack of familiarity among researchers and reviewers. The video concludes by acknowledging the necessity of understanding p-values due to their prevalence in current research, while also recognizing the potential benefits of alternative statistical methods like Bayesian analysis.

Link to slides: https://osf.io/5c9fp