Statistically Speaking Statistical Power

Statistical power often is thought of solely in the context of determining sample size for a planned study, but power is in fact more relevant and useful beyond this context, and thus it’s useful to understand it from a statistician’s perspective. It’s important to remember that power is a theoretical property of a statistical test in a given study situation, in much the same way that sensitivity and specificity are properties of medical tests, as will be discussed herein. When we speak of “power calculations,” we usually are either using our knowledge of a specific study situation to estimate how strong its power will be, or we have a target power level in mind for a situation and are estimating how large the sample size we will need to achieve our goal. There are 4 main components that affect the power of a study: sample size, significance threshold, population variance of the effect, and effect size. Knowing these 4 values will allow a researcher to calculate the theoretical power of a given study design. Of these 4 factors, sample size is the component most under a researcher’s control; the latter 3 factors typically are dictated by the situation. Thus, researchers usually use power calculations to work backwards: for a given significance threshold, population variance, and effect size, they calculate what sample size is necessary to design a study with 80% power. These 4 factors will be discussed in more detail herein. Conceptually, the power of a statistical test is similar to the power of a telescope: it is the ability to separate a true pattern from its background. The smaller the object to be detected, or the more distracting the environment is, the more magnification power is needed to detect a true signal. A statistically high-powered, placebo-controlled randomized study, for instance, is more likely to spot even subtle differences between the treatment and control groups than a low-powered study would, even when there is high person-to-person variability (“noise”) in the results. Thus, designing a study to have high power reduces the chance of overlooking true findings (ie, reducing false negatives, or Type II errors), and it also increases the chances that a significant finding in fact represents a true effect (a true positive). Terminology: Sensitivity: The “true positive” rate for a test; the proportion of “positives” that the test correctly flags as being positive. Specificity: The “true negative” rate for a test; the proportion of “negatives” that the test correctly excludes as being negative. Type I error rate: The “false-positive” rate for a statistical test; the proportion of “negatives” that will be incorrectly flagged as being positive. Also known as a, or the significance threshold. Type II error rate: The “false-negative” rate for a statistical test; the proportion of “positives” that will be incorrectly excluded as being negative. Positive predictive power: Of the cases flagged by the test as being positives, the proportion that are truly “positives. ” Negative predictive power: Of the cases excluded by the test as being negatives, the proportion that truly are “negatives. ” Effect size: A general term for the quantitative magnitude or strength of the effect being studied. Specific examples include the difference in outcome measure between 2 groups; the correlation between 2 measures; the change in an outcome measure over time. Power: The prestudy probability that a statistical test will correctly return a result of “statistically significant” for true effects that are no smaller than a certain size. PM R 8 (2016) 907-912 www.pmrjournal. org