Should the Term “Statistical Significance” and P Values Go Away in Pharmacy Publications?:

We all have read published papers that have included the term ‘‘statistical significance’’ or ‘‘statistically significant’’ to describe the difference between 2 numbers, but should we be reading these terms in any published paper? Why do authors have to tell us that something is statistically significant? Do they think we are incapable of determining whether the difference between 2 values is important if sufficient information is provided? It is common for authors to provide statistical values, such as P values or confidence intervals, in a sentence that also states that the difference was statistically different. I am sorry, but that is redundant. If you provide the evidence that the 2 numbers are different by including the statistical value, then you do not need to state that there is a statistical difference between the 2 numbers. We need to remember that statistics is a mathematical tool; it is intended to help us determine whether the outcome of an experiment is the result of differences associated with the treatments or merely the result of chance. It would be better for the authors to state the results from the experiment and let the statistical values support that statement. The other practice that is deceptive is to refer to a finding as highly statistically different because the P value is small (eg, P , .001 or P 5 .00001). The only piece of information that a P value provides is whether there is a mathematical and probability difference between the values. The more zeros or the smaller the number, the higher the probability that the difference is not by chance, so we can feel more confident that the difference might be real. What statistics does not tell us is whether there is any clinical difference between the 2 values. It is possible to show that there is a statistical difference between 2 treatments, but that difference may be small enough that it has no meaning in clinical practice. As we continue along the evidence-based medicine path and its application to clinical practice, we need to move away from dependence on P values and move toward the use of confidence intervals. A P value lets us know that there is a difference between the 2 values, but it provides no information on the size of the effect or the amount of variation in the data. The confidence interval provides more information. The mean data point is the difference between the 2 numbers, and the 95% confidence interval is the interval that should contain the true value for the study population. The wider the confidence interval, the higher the variability of the data and more uncertainty we have about the true difference between the values. As the size of the study population increases, the width of the confidence interval will decrease and the more certainty we have that the true value for this population is contained within the confidence interval. Therefore, the confidence interval provides us with a feel for the reliability of that conclusion and can let us decide whether the value is statistically and clinical meaningful. When possible, we should be using confidence intervals alone or in combination with P values or other statistical values. We should also avoid using terms such as highly statistically significant and let the data speak for themselves. In the end, the real question is whether there a real difference between the 2 treatments and whether that difference is meaningful for our patients.