In Defense of P Values

Academics love to hate P values. Recently, dozens of commentaries in prestigious academic journals and well-thought-out position papers in academic blogs have criticized the use or misuse of P values in biomedical research, and in science more broadly. In fact, one journal (Basic and Applied Social Psychology) recently banned the use of hypothesis testing, stating that there is “no inferential statistical procedure that has elicited widespread agreement”. Although this aversion is justified to a limited extent, it is doubtful whether this hostility should be specifically targeted at P values, rather than a more diffuse sense of negativity toward absolutist thinking about hypothesis testing, a general fear of statistics and mathematics, and uncertainties associated with conducting and publishing research. If so, the solution is not as simple as doing away with P values, but rather teaching researchers to more effectively communicate and evaluate empirical results. This requires that people are able to effectively communicate and critically evaluate the various decisions made in the analytical pipeline so that they can identify the validity of the hypothesis being tested. That said, if the scientific community collectively decides to move away from P values, then logically these values must be replaced with some other construct, such as confidence intervals, Bayesian credible intervals, or perhaps another yet-tobe-discovered statistic. The catch is that simply replacing P values with another statistic does not address the negativity directed at P values. P values are statistical tools that guide hypothesis testing and scientific decision making. However, P values are only one tool in a much larger statistical toolbox that includes many other implements: correlations, regression coefficients, odds ratios, relative risk ratios, standard errors, effect sizes, and so on. To conduct an analysis and make any conclusions about a research question, we must simultaneously use several procedures. It is important to stress that nurse anesthetists should not base clinical decisions too heavily on P values and overlook other equally important statistics, but we must also be cautious not to act too rashly by prematurely discarding potentially useful information. There are positive and negative aspects of most tools, and whether the positives outweigh the negatives often lies in the expertise of the user, not in the tool itself. One likely reason that we emphasize P values over other statistics is the extreme simplicity, and nearuniversal acceptance, of the “rule” that a hypothesis test must have a P less than . 05 to be deemed statistically significant. This arbitrary yet dogmatically accepted rule fools us into believing that we can compare disparate statistics by reducing everything to P values. In doing so, we obscure the difference between statistical significance and clinical relevance. P values are influenced by a plethora of scientific decisions that are made in the process of conducting a study, such as who or what should be measured, how people should be assessed, and how the data should be analyzed. Some of these decisions are made intentionally after much deliberation, whereas others are made reflexively, or because that is the way that it is always done. Many clinicians, healthcare providers, and even research scientists do not appreciate the subtleties of P values and are unaware of just how many factors influence, and potentially invalidate, them. P values have become the scapegoat for a wide variety of problems in science. P values are generally overemphasized, often incorrectly applied, and in some cases even abused. However, alternative methods of hypothesis testing will likely fall victim to the same criticisms currently leveled at P values if more fundamental changes are not made in the research process. Increasing the general level of statistical literacy and enhancing training in statistical methods provide a potential avenue for identifying, correcting, and preventing erroneous conclusions from entering the academic literature and for improving the general quality of patient care.