Why do we read many articles with bad statistics? : what does the new American Statistical Association's statement on p-values mean?

Many researchers are confused by the new statement on p-values recently released by the American Statistical Association (ASA) [1]. Researchers commonly use p-values to test the "null hypothesis", i.e., no differences between two groups or no correlation between a pair of characteristics. The smaller the p-value is, the less likely the observed value would occur by chance. Generally, a p-value of 0.05 or less is regarded as statistically significant, and researchers believe that such findings constitute an express ticket for publication. However, this is not necessarily true, as the ASA1) statement notes. Many statisticians have pointed out the problem of the "fallacy of the transposed conditional", which is to assume that P(A|B) = P(B|A) [2]. This expression states that the probability of A being true given B is the same as the probability of B being true given A; however, this is not the same thing. Statisticians are increasingly concerned that the p-value is being misapplied. They hope that the ASA1) statement will play a role in resolving the reproducibility and replicability (R&R) crisis. In the ASA1) statement, a p-value is informally defined as follows [1]: "A p-value is the probability under a specified statistical model that a statistical summary of the data (for example, the sample mean difference between two compared groups) would be equal to or more extreme than its observed value. " The statement actually describes what we "do" and "do not" with p-values. Table 1 shows the six principles for using p-values. The p-value is an indication of how incompatible a dataset is with the null hypothesis. A p-value does not measure the probability that the research hypothesis is true, given the definition of p-value stated above. Many researchers and decision-makers for business or policy are usually interested only in whether a p-value passes a specific threshold. Ultimately, this can lead to incorrect conclusions and poor business or policy decisions. For the proper inference, full reporting and transparency are always needed. We should not perform "data dredging" [3]. The p-value is not the effect size. It can be low, even if one has a very small effect with large sample sizes and small error. Recall the above definition of p-value. A p-value of 0.05 does not mean that there is a 95% chance that a given hypothesis is correct [4]. We should recognize that a p-value without context or other evidence (e.g., confidence intervals) provides only limited information. It does not provide a good measure of evidence concerning a hypothesis. Table 1 Six Principles for Using p-values The importance of this statement is that professional statisticians have voiced their concern over statistical problems that appear in the literature of other areas. This is not an effort to correct misapplication of the p-value. For example, in 2015 the journal Basic and Applied Social Psychology formally announced that they oppose publishing papers containing p-values. The journal editor explained that this was because p-values were too often used to support lower-quality research, with findings that could not be reproduced [5]. Franklin Dexter [6], the Statistics Editor for Anesthesia & Analgesia, has already written that a small p-value itself does not necessarily indicate an important finding and that the p-value should be accompanied by confidence intervals to quantify the clinical importance of the estimated difference. In an article on the statistical methods used in anesthesia articles, Avram et al. [7] wrote that most errors in statistical analysis are related to the misuse of elementary hypothesis tests. We all have read and written too many papers with bad statistics showing that p-values overstate the evidence against the null hypothesis. Thus, we are both victims and offenders.