An Interactive FORTRAN IV program for calculating aspects of power in correlational research

In designing correlational studies, the issue of statistical power is extremely important. Power is the probability that a correlation, if it exists, will be significantly different from zero in a given experiment. The importance of determining power in experimental planning was highlighted by Schmidt, Hunter, and Urry (1976), who reported that statistical power in many correlational investigations was much lower than that required for meaningful research. Tversky and Kahneman (1971) pointed out that researchers assume greater power than small samples actually provide. Schmidt et al. (1976) reported a general agreement that sample sizes from 30 to 50 were adequate for criterion-related validity studies. Moreover, the median sample size for all validity studies (both published and unpublished) was estimated to be between 40 and 70 (cf. Guion, 1965; Lent, Aurbach, & Levin, 1971). Schmidt et al. (1976) noted that the common intuitive overestimation of power with small samples has major implications for the valid interpretation of correlational results. Because the power of a design is a complex function of sample size, alpha level, population variance, and actual effect size, few researchers bother to compute power. Thus, many correlational studies, because of small sample sizes, may lack sufficient statistical powerto reach valid conclusions about the relationships that mayor may not exist between variables. Schmidt et al. (1976) outlined a procedure for estimating the level of statistical power in correlational designs. This approach takes into account the possibility that the variables may be collected from a limited sample (i.e., a restriction of range), and that the instruments employed in the investigation may provide unreliable measures of the constructs under investigation. When this approach was applied to studies in which sample sizes were in the •'typical" range, statistical power was found to be disturbingly low. Researchers often were misled by the results of their experiments. Other reasons why investigators have neglected the issue of power are the inaccessibility of the necessary formulas and appropriate statistical tables, and the level of statistical sophistication required to calculate power (Cohen, 1977). The program listed here provides researchers and practitioners with estimates of statistical power, without their having to track down tables and formulas from the statistical literature, and without their having to perform complex calculations or table interpolations. The program is written in the same general manner as two previous programs, one to compute power for designs to be analyzed by analysis of variance (Dunlap, 1981), and one to determine power for frequency data where the usual method of analysis is chi-square (Dunlap, 1982). The Program. The program is designed to operate interactively in planning or assessing correlational research. It first requests the alpha level (two-tailed). Two options are employed only if the user is familiar with the issues and complexities of corrections for range restriction and unreliability, and has worked through some general treatment such as that of Schmidt et al. (1976). The program asks whether the user would like to correct for restriction of range. Ifthe reply is "yes," the user is asked to enter the population standard deviation (SO), followed by the SO of the sample. Generally, the sample SO will be smaller than the full population SO, if the sample has been preselected on some basis that relates to the predictor, or if there is self-selection or other sample mortality related to the predictor or criterion. Usually, if the predictor is a standardized instrument, then population norms will provide the population SD, and the sample SO can be computed directly from the data. Next, the program asks whether the user wishes to correct for unreliability. If the response is "yes," the user is asked to enter reliability estimates for the criterion and the predictor (a reliability of one may be entered where no correction is required). At this point, the user is given the choice of computing either power, sample size, or the detectable correlation. In order to calculate anyone of these three aspects of power, values must be entered for the other two. For example, to compute sample size, the program first requests a value for the population correlation. The program then responds with the correction for range restriction and the correction for less than perfect reliability , if these items were selected earlier. Next, the program asks the user to enter the desired power, and responds with the sample size that would produce power closest to that requested. The other options are executed in a similar interactive manner, allowing the user to review the various aspects of power under whatever variety of conditions are entered. Several algorithms were tested for both efficiency and accuracy in computing the power associated with the test of significance for a correlation coefficient, using Monte Carlo simulation as a guidepost. Both the procedure used by Cohen (1977) and that used by Schmidt et al. (1976) were found to be reasonably accurate; the latter was chosen because it is computationally simpler. Power is calculated by first solving: