A Simulation Study of the Power Efficiency of Certain Nonparametric Statistical Tests for Normal Alternatives.

A Monte Carlo simulation technique was used to investigate the power efficiency of three nonparametric two-sample tests. The power of the sign test, the Kolmogorov-Smirnov test, and the Mann-Whitney U test was compared with the power of their t-test equivalent— the paired t-test in the case of the sign test, and the t-test for independent samples for the Kolmogorov-Smirnov test and the Mann-Whitney test. The simulation process permitted the investigation of a wide range of parameters. Each test was investigated for one-tailed signifi­ cance levels of . 05 and . 01; equal samples of size m = n = 6(2)20, 30, AO, 50; and location-shift alternatives 0 = 0.0(0.2)1.0, 2.0, 3.0, where © m *̂2 V*i. Restrictions on computer time prevented the analysis from O encompassing a wider range of parameters. The analysis was performed on an IBM 360/65 computer with a simulation process based on a Monte Carlo procedure of generating random normal deviates. Random samples of equal size were generated from normal distributions with equal variances of one; the first sample being drawn from a distribution with y = 0 and the second sample from a distribution with y 0. Two thousand separate samples were tested for each set of parameters for samples 6 to 20 and 1,000 repetitions for samples 30 to 50. Power was obtained by establishing a decision rule and determining the number of rejections in the total number of test samples. The findings were divided into two categories— probability of a Type I error (0 ■ 0.0) and power efficiency. The results obtained from simulating the probability of a Type I error Indicate that, In general, each nonparametrlc and parametric test was operating under similar test conditions, and, therefore, valid find­ ings were produced in the study. However, for the Kolmogorov-Smirnov te s t, which is based upon the establishment of cumulative frequency dis­ tributions, it was necessary to increase the number of class intervals in the cumulative distributions to 2(n + m) before valid results were obtained. The power efficiency of the sign test decreased from approximately 80 percent for the smaller parameter values of n and G to approximately 60 percent as the parameters increased. Over the same range of parameter values, the relative efficiency of the K-S test increased from approxi­ mately 50 to 70-75 percent, and all of the power efficiency values for the U-test fluctuated, primarily, between 90 and 100 percent. A slight increase in power efficiency was noted for both the sign test and the Kolmogorov-Smirnov test as the significance level decreased. Sampling error prevented any patterns from emerging as parameters changed for the U-test. It was anticipated that the K-S test would outperform the sign test for all parameter values. This proved not to be true for the smaller parameters. The power of the K-S test relies upon the assumption of continuous distributions and if this assumption is violated by creating too few classes then performance suffers. Therefore, the researcher is advised to use at least 2(n + m) class intervals in the test procedure. The power of the U-test was found to be very close to that of the t-test. The U-test is recommended over the t-test in all cases for test­ ing the hypothesis of equal means, except those in which the underlying distributions can be safely assumed to be normal. The Kolmogorov-Smirnov test is preferred to the sign test when large samples or large locationshift alternatives are encountered. However, when small samples or alternatives are involved the evidence of this study favors the sign test, especially when the ease of computation is considered.