An Investigation of the Power of Stout's Test of Essential Unidimensionality.

The power of Stout's procedure to detect deviations from essential unidimensionality in twodimensional data was investigated for minor, moderate, and large deviations from unidimensionality. The criteria used in the categorization of deviations from unidimensionality were based on Shepard, Camilli, and Williams's categorization of area measures of item bias. The power of Stout's procedure was directly related to the deviation from unidimensionality based on deviation areas. Deviation areas were inversely related to the correlation between the dominant ability and the reference composite. When the sample size increased, the power of Stout's procedure also increased. The power for 40-item tests was higher than for 20-item tests. When the proportion of items loaded on the minor dimension was 20%, the power was the highest. Although the power for the 20% condition was higher than the 100% condition, the correlations 13.601 for the 20% condition were also extremely high. For the 10% and 20% conditions, even when the correlations Rol were near 1.00, the rejection rates were high. In general, Stout's procedure had sufficient power to reject the null hypothesis of essential unidimensionality if 10% to 20% of the items were dimensionally distinct from the rest of the items. This is because only 10% to 20% of the items are being selected into the subtest (AT1) used in testing essential unidimensionality. When AT1 is dimensionally distinct from the rest of the items, Stout's null hypothesis of essential unidimensionality will be rejected. The results of this study indicate that for minor deviation from unidimensionality, the rejection rates of Stout's procedure were not near the nominal levelof 5%. For moder and large deviation from unidimensionality, Stout's procedure had power to reject the null hypothesis of essential unidimensionality, especially if the sample size was 1,500 and the test length was 40. Further studies are recommended. AN INVESTIGATION OF THE POWER OF STOUT'S TEST OF ESSENTIAL UNIDIMENSIONALITY Introduction Stout's procedure and the concept of essential unidimensionality have been described in detail (Nandakumar, 1991; Stout, 1987, 1990). Although the power of Stout's procedure has been studied (Nandakumar, 1991; Stout, 1987), all conditions manipulated.were not conducted with known minor, moderate, and large deviations from unidimensionality. Also, the effect of the proportion of items loaded on the minor dimension and the effects of test lengths have not been systematically studied. Purpose The purpose of this study was to investigate the power of Stout's procedure to detect deviations from essential unidimensionality in two-dimensional data. The specific questions were as follows: 1. How do minor, moderate, and large deviations from unidimensionality affect the power of Stout's procedure for testing essential unidimensionality? 2. How does the proportion of items loaded on the minor dimension affect the power of Stout's procedure for testing essential unidimensionality? 3. How does test length affect the power of Stout's procedure for testing essential unidimensionality? 4. How does sample size affect the power of Stout's procedure for testing essential unidimensionality? Design of the Study Test Length and Sample Size In the present study, the test length and sample size each have two levels. The test lengths studied were 20 (a short test) and 40 (an averat,e-length test). Small and large sample sizes of 700 and 1,500 were studied. Hattie (1984) stated that sample sizes smaller than 300 tended to be unstable for latent trait procedures. In addition, large sample sizes (>5,000) might result in inappropriate rejection rates. 2 Item-Parameters The item test parameters of the two-dimensional model with one major and one minor dimension were used in generating item response data. The first dimension was the major dimension that the test was purported to measure and the second dimension was the minor dimension. The influence of the second dimension on each item was relatively weak compared to that of the first dimension. The means and variances of a, and a2 reflected the degree to which their respective traits influenced item scores. An item with a large a, and a small a2 was much more heavily influenced by 0, than by 02 and vice-versa (Nandakumar, 1991). Both 01 and 02 were normal with mean zero and variance equal to one. The correlation between the abilities was set at zero. A preliminary investigation using Nandakumar's 4 (1991) to control the weight of the major dimension relative to the weigh> of the minor dimension was carried out. The item test parameters a, and a2 were computed by varying p, a and 4 in the following expressions: a, N((1-4)p, (1-4)'/2a) a2 N(41.1, 41/2a) a, + a2 1\1(p, a) (1) With p = 1.07 and a2 = 0.16, Stout's test of essential unidimensionality had little power. Even when the seight of the minor dimension was the same as the major dimension (4 = 0.5), the rejection rates were less than 20%. With a2 = 0.64 however, there was substantial power even at l, = 0.3. Using Nandakumar's 4 requires a large a2 because increasing 4 with a constant p and a2 led to a decrease in (5,, = (14)'/20 and 11.1 = (1-4)p. Unless a,12 is relatively large, the mininum of the variance of a, and a2 will also be small ((3) and lead to little power (Nandakumar, 1991). To avoid using a large a2 and hence a large range, in this study a small a2 was studied and the effect of the reduction in a. , and pa, (due to 4) was controlled by holding both aa, and pa, constant. In this study, the values of a, were fixed across conditions; that is, only one set of a, was used across deviation areas for each test length. For the 40-item tests, the al parameters used in this study were the discrimination parameters of a 40-item ACT math test reported by Drasgow (1987). The mean and