bluuebunny/crossref_metadata_2025 huggingface dataset:
The Milwaukee Voucher Experiment
Authors: Witte John F.
Published: March 2008

This article provides a summary of the results of the first five years of the Milwaukee Parental Choice Program, which was the first program in the United States to allow students to attend private schools with public vouchers. I begin with a brief discussion of the theoretical and research issues. Following a description of the initial program and subsequent changes, I outline who participated in the program—including characteristics of students and families and schools. I then describe the results in terms of the effects on families, student outcomes, and schools. I conclude with a discussion of the implications for this type of program and more open-ended voucher programs. For those holding extreme positions on this controversial issue, there will be both ammunition and frustration, for the results contain both positive and negative elements. The quality of both the public and private schools and therefore student outcomes vary within and between schools, and that variance is more extreme than in middle-class or wealthy communities. Some schools are excellent, and families fight to get in them and stay in them. Some are so bad that they fail and, if they are private, cease to exist—often in mid-year. The general results of the voucher program follow that pattern: Some results are clearly positive, some can be interpreted either way, and others are negative.
Merit or Family Background? Problems in Research Policy Initiatives in Japan
Authors: LeTendre Gerald K. , Rohlen Thomas P. , Zeng Kangmin
Published: March 2008

In reviewing educational trends in Japan during the post-World War II era, the authors found that current national data sets do not contain questions about family background or rank of school attended that are detailed enough to permit a significant analysis of the impact of family background on educational attainment. Given Japan’s stature in the current community of nations, the continued government aversion to collecting family income or educational background data, along with precise data on rank of school attended and/or types of cram school, appears negligent. The authors discuss both cultural and political reasons that may create a reluctance to collect such data.
Cluster Analysis Applied to the Validation of Course Objectives
Authors: Kettenring J. R. , Rogers W. H. , Smith M. E. , Warner J. L.
Published: March 2008

This paper describes the use of cluster analysis to aid in empirically validating the course objectives of an industrial training curriculum. For instance, clusters of basic training needs were found and compared against existing course contents. Two major findings emerged from the data analysis: the discovery of an important job activity missing from the curriculum and the identification of sizeable groups of workers with distinctly different training needs. Considerable attention is also devoted to the statistical aspects of clustering which were important in obtaining these results.
An Expected Utility Model for “Optimal” Selection
Authors: Petersen Nancy S.
Published: March 2008

The Threshold Utility Model for culture-fair selection introduced by Gross and Su (1975) and by Petersen (1974) is defined here for both quota-free and for restricted selection. In addition, a full Bayesian solution based on posterior predictive distributions is provided. By providing a mechanism for coherently combining probabilities and utilities, this model makes it possible to determine an optimal selection strategy in all situations in which a threshold utility function is applicable. This paper studies the mathematical properties of the solution in detail in the case of quota-free selection. In the more complex restricted selection case, the study lacks some detail but is more complete than that given by Gross and Su and should be adequate for most practical purposes. Several data analyses highlight the strengths and weaknesses of the model.
Assignment to Treatment Group on the Basis of a Covariate
Authors: Rubin Donald B.
Published: March 2008

When assignment to treatment group is made solely on the basis of the value of a covariate, X, effort should be concentrated on estimating the conditional expectations of the dependent variable Y given X in the treatment and control groups. One then averages the difference between these conditional expectations over the distribution of X in the relevant population. There is no need for concern about “other” sources of bias, e.g., unreliability of X, unmeasured background variables. If the conditional expectations are parallel and linear, the proper regression adjustment is the simple covariance adjustment. However, since the quality of the resulting estimates may be sensitive to the adequacy of the underlying model, it is wise to search for nonparallelism and nonlinearity in these conditional expectations. Blocking on the values of X is also appropriate, although the quality of the resulting estimates may be sensitive to the coarseness of the blocking employed. In order for these techniques to be useful in practice, there must be either substantial overlap in the distribution of X in the treatment groups or strong prior information.
Tests for Equality of Correlated Proportions in a Polychotomous Response Design
Authors: Katz Barry M.
Published: March 2008

This paper surveys available techniques and introduces an explicit statement of a new statistic to test for equality of correlated proportions in a polychotomous response design. The statistic does not require use of general linear models methodology, but it is an explicit, special case of methodology implicit in Bhapkar (1966) , Grizzle, Starmer, and Koch (1969) and Koch and Reinfurt (1971) . The large-sample distribution for the test statistic is derived as well as computational formulas for calculating the statistic. Associated post hoc procedures are also developed. A conservative, modified form of the statistic is constructed, and an empirical investigation of the small sample behavior of both statistics is carried out via Monte Carlo sampling. A set of guidelines for the potential user of the techniques is provided.
Utilizing Innovative Methodology in Education: An Evaluation Procedure for Laboratory Development Amenable to Subsequent Lea Implementation Decisions
Authors: Bible Thomas D.
Published: March 2008

A method for evaluating the effectiveness of innovative school programs as a function of well defined school objectives is developed. The same procedures are amenable to program development as well as implementation stages. LEA’s can use the same framework as educational researchers developing a program when evaluating the effectiveness of new school programs on the basis of program components. The evaluation method will enable LEA program implementors to sequentially adopt interdependent or multicomponent educational innovations should local needs or finance dictate that strategy. A numerical example of the evaluation methodology applied to a compensatory reading program is reported.
Nonorthogonal Analysis of Variance Programs: An Evaluation
Authors: Hosking James D. , Hamer Robert M.
Published: March 2008

Recently, at least four methods have been suggested for the analysis of nonorthogonal designs. However, even researchers who have decided which method is appropriate often do not know which computer programs are capable of performing the desired analysis. In addition, available programs differ widely in their accuracy and ease of use. We have evaluated some widely used ANOVA computer programs with respect to their capabilities for performing analyses using all four methods, their accuracy in performing these analyses, and other characteristics which influence their utility.
Unifactor Latent Trait Models Applied to Multifactor Tests: Results and Implications
Authors: Reckase Mark D.
Published: March 2008

In order to extend the use of latent trait models across the full spectrum of mental testing, the applicability of the models to multivariate data must be determined. Since all of the commonly used models assume a unidimensional test, the applicability of the procedures to obviously multidimensional tests, such as achievement tests, is questionable. This paper presents the results of the application of latent trait analyses to a series of tests that vary in factorial complexity. The purpose is to determine what characteristics are estimated by the models for these tests, while at the same time determining the relationship of latent trait parameters to traditional item analysis and factor analysis indices.
A Comment on Correctly Calculating Higher Order Semipartial Correlations
Authors: Bush Andrew J. , Rakow Ernest A. , Gallimore Debra N.
Published: March 2008

A calculational strategy presented in educational statistics texts for computing second and higher order semi-partial correlations is critically examined and shown to be incorrect.
Continuity Correction for Fisher's Exact Probability Test
Authors: Overall John E.
Published: March 2008

Fisher's exact probability test is severely conservative when interpreted with reference to conventional alpha levels due to the discontinuity of the sampling distribution for 2 × 2 tables. An adjustment of the cell frequencies is proposed that results in a correction for continuity with appropriate alpha protection and increased power.
Evaluation of Three Sample Size Formulae for 2 × 2 Contingency Tables
Authors: Hornick Chris W. , Overall John E.
Published: March 2008

Estimates of sample size requirements for 2 × 2 tables, derived from three approximate formulae, were evaluated by comparison wlth exact calculations. Although different, the estimates derived from all three formulae were judged adequate for most practical purposes, with the most accurate results being obtained from Cohen's formula. The strategy for exact determination of the minimum N required to provide specified power for tests of significance of differences in proportions in two treatment groups is described.
Distribution Theory for Glass's Estimator of Effect size and Related Estimators
Authors: Hedges Larry V.
Published: March 2008

Glass's estimator of effect size, the sample mean difference divided by the sample standard deviation, is studied in the context of an explicit statistical model. The exact distribution of Glass's estimator is obtained and the estimator is shown to have a small sample bias. The minimum variance unbiased estimator is obtained and shown to have uniformly smaller variance than Glass's (biased) estimator. Measurement error is shown to attenuate estimates of effect size and a correction is given. The effects of measurement invalidity are discussed. Expressions for weights that yield the most precise weighted estimate of effect size are also derived.
Comparison of Procedures for Detecting test-Item Bias with both Internal and External Ability Criteria
Authors: Shepard Lorrie , Camilli Gregory , Averill Marilyn
Published: March 2008

Test bias is conceptualized as differential validity. Statistical techniques for detecting biased items work by identifying items that may be measuring different things for different groups; they identify deviant or anomalous items in the context of other items. The conceptual basis and technical soundness were reviewed for the following item bias methods: transformed item difficulties, item discriminations, one- and three-parameter item characteristic curve methods, and chi-square methods. Sixteen bias indices representing these approaches were computed for black-white and Chicano-white comparisons on both the verbal and nonverbal Lorge-Thorndike Intelligence Tests. In addition, bias indices were recomputed for the Lorge-Thorndike tests using an external criterion.

Convergent validity among bias methods was examined in correlation matrices, by factor analysis of the method correlations, and by ratios of agreements in the items found to be “most biased” by each method. Although evidence of convergent validity was found, there will still be important practical differences in the items identified as biased by different methods. The signed full chi-square procedure may be an acceptable substitute for the theoretically preferred but more costly three-parameter signed indices. The external criterion results also reflect on the validity of the methods; arguments were advanced, however, as to why internal bias methods should not be thought of as proxies for a predictive validity model of unbiasedness.
Correlation Coefficients for Comparing Expressed and Inventoried Interests
Authors: Guthrie Donald
Published: March 2008

This paper presents simple correlation measures between expressed and inventoried interests. Tables are given for significance testing, and asymptotic standard errors are derived.
Methodological Issues in the Search for a Factor Model: Exploration Through Confirmation
Authors: Kroonenberg Pieter M. , Lewis Charles
Published: March 2008

The usefulness of some procedures suggested by Jöreskog for performing exploratory factor analysis is investigated through an in-depth analysis of some of the Holzinger-Swineford test data. In this case the procedures, which use both exploratory and confirmatory techniques, work not unreasonably, although they manage only just, and sometimes not quite, to steer away from misleading results.
Contingency Table Models for Assessing Item Bias
Authors: Mellenbergh Gideon J.
Published: March 2008

Two strategies for assessing item bias are discussed: Methods based on Item x Group interaction and methods that compare the probabilities of a correct response for different groups conditional on ability level. In latent trait models, correct response probabilties are compared conditional on latent ability; in Scheuneman’s (1979) method these probabilities are compared conditional on the observed test score. Scheuneman’s method is modified to fit the general theory of loglinear and logit models for contingency tables. A distinction is made between uniform and nonuniform item bias, and a method to assess item bias and distinguish between uniform and nonuniform bias is described. Data reported by Scheuneman are used to demonstrate the method in detail; the differences obtained with Scheuneman’s method are discussed. In addition, the method is applied to two tests administered by van der Flier (1980) in Kenya and Tanzania, and the results are compared to those obtained with Scheuneman’s approach.
Fitting Categorical Models to Effect Sizes from a Series of Experiments
Authors: Hedges Larry V.
Published: March 2008

One method of combining the results of a series of two-group experiments involves the estimation of the effect size (population value of the standarized mean difference) for each experiment. When each experiment has the same effect size, a pooled estimate of effect size provides a summary of the results of the series of experiments. However, when effect sizes are not homogeneous, a pooled estimate can be misleading. A statistical test is provided for testing whether a series of experiments have the same effect size. A general strategy is provided for fitting models to the results of a series of experiments when the experiments do not share the same effect size and the collection of experiments is divided into a priori classes. The overall fit statistic H T is partitioned into a between-class fit statistic H B and a within-class fit statistic H w. The statistics H B and H w permit the assessment of differences between effect sizes for different classes and the assessment of the homogeneity of effect size within classes.
Coherent Analysis-of-Variance Hypothesis-Testing Strategies: A General Model
Authors: Betz M. Austin , Levin Joel R.
Published: March 2008

Logically consistent (“coherent”) hypothesis-testing strategies for factorial analysis-of-variance (ANOVA) designs are proposed in the context of a hierarchical model. It is shown that all of the hypotheses associated with the “traditional” factorial ANOVA model (i.e., main effects and interactions) are conceptually independent and occupy the lowest levels of the hierarchy. A research example is presented to illustrate the kind of conclusions that legitimately follow from testing the traditional hypotheses on the one hand, versus a variety of alternative hypotheses on the other.
Parametric Alternatives to the Analysis of Variance
Authors: Clinch Jennifer J. , Keselman H. J.
Published: March 2008

The ANOVA, Welch, and Brown and Forsyth tests for mean equality were compared using Monte Carlo methods. The tests’ rates of Type I error and power were examined when populations were non-normal, variances were heterogeneous, and group sizes were unequal. The ANOVA F test was most affected by the assumption violations. The test proposed by Brown and Forsyth appeared, on the average, to be the “best” test statistic for testing an omnibus hypothesis of mean equality.
Multivariate Generalized Beta Distributions with Applications to Utility Assessment
Authors: Libby David L. , Novick Melvin R.
Published: March 2008

Two multivariate probability distributions, namely a generalized beta and a generalized F, that appear to be useful in utility modeling are derived. They reduce to the standard beta and F distributions, respectively, in special cases. Reproduction of distributional form is demonstrated for marginal and conditional distributions. Formulas for the moments of these distributions are given. The usefulness of these distributions in utility modeling derives from the fact that they generally do not demand increasing risk aversion as do most standard forms. An example of the use of the bivariate generalized beta distribution in utility modeling is presented. This distribution compares favorably in an example given here to both a normal model and an unstructured model.
An Empirical Study of Related Correlation Coefficients
Authors: Boyer John E. , Palachek Albert D. , Schucany William R.
Published: March 2008

Tests for comparing the strength of association between a variable, X1, and each of two potential predictor variables, X2 and X3, are proposed and examined in a simulation study. The variances of X2 and X3 and the correlation between X2 and X3 are nuisance parameters. A simple modification of a test proposed by Williams (1959) is found to have reasonable properties for a wide range of parameter values and both normal and nonnormal distributions.
Effectiveness of Analytic Smoothing in Equipercentile Equating
Authors: Kolen Michael J.
Published: March 2008

An analytic procedure for smoothing in equipercentile equating using cubic smoothing splines is described and illustrated. The effectiveness of the procedure is judged by comparing the results from smoothed equipercentile equating with those from other equating methods using multiple cross-validations for a variety of sample sizes. Data on randomly equivalent groups of approximately 3,000 examinees per form from four forms of each of the four tests of the ACT Assessment Program (AAP) were used in this evaluation. Relative to the other equating procedures studied, smoothed equipercentile equating was found to be most adequate for the AAP, especially for the most dissimilar form pairs.
Parametric ANCOVA and the Rank Transform ANCOVA When the Data are Conditionally Non-Normal and Heteroscedastic
Authors: Olejnik Stephen F. , Algina James
Published: March 2008

Parametric analysis of covariance was compared to analysis of covariance with data transformed using ranks. Using a computer simulation approach, the two strategies were compared in terms of the proportion of Type I errors made and statistical power when the conditional distribution of errors was normal and homoscedastic, normal and heteroscedastic, non-normal and homoscedastic, and non-normal and heteroscedastic. The results indicated that parametric ANCOVA was robust to violations of either normality or homoscedasticity. However, when both assumptions were violated, the observed α levels underestimated the nominal α level when sample sizes were small and α = .05. Rank ANCOVA led to a slightly liberal test of the hypothesis when the covariate was non-normal, the sample size was small, and the errors were heteroscedastic. Practical significant power differences favoring the rank ANCOVA procedures were observed with moderate sample sizes and a variety of conditional distributions.
Bayesian Analysis of a 2 × 2 Contingency Table with Both Completely and Partially Cross-Classified Data
Authors: Smith Philip J. , Choi Sung C , Gunel Erdogan
Published: March 2008

A frequently used experimental design is one in which the experimental units are measured twice (e.g., under different test conditions). When the response variable is dichotomous, the equality of the two proportions is usually assessed by a test due to McNemar (1947) . However, in addition to obtaining this complete data where two responses are available for each unit, incomplete data may be available also: In this case observations are available on the first response alone for some units and additional observations are available on the second response alone for other units. In this paper Bayesian methods are presented for estimating and testing hypotheses regarding the two success probabilities in light of both the complete and incomplete data. A method by which the prior distribution may be assessed is sketched and a numerical example to illustrate the method is presented.
A Strategy to Teach the Concept and Application of Power of Statistical Tests
Authors: Kraemer Helena Chmura
Published: March 2008

A method is suggested to incorporate instruction in the concept and the application of power considerations into an applied statistics course. This is based on designing one table usable for a wide range of the type of statistical tests usually introduced in such a course.
Teaching How to Derive Directly Interpretable Coding Schemes for Multiple Regression Analysis
Authors: Serlin Ronald C. , Levin Joel R.
Published: March 2008

Multiple linear regression is a versatile model for encompassing analysis of variance, analysis of covariance, and aptitude-by-treatment interaction designs. The question of how to teach the coding of levels of a qualitative variable is addressed in this paper. Although a variety of coding schemes will produce invariant omnibus statistical results for a given set of data, one’s interpretation of treatment effects and treatment differences depends on the particular code values chosen. A general procedure is presented that allows the user to generate, on an a priori basis, code values that result in directly interpretable estimates of interest.
Type I Error Probabilities and Power of the Rank and Parametric Ancova Procedures
Authors: Seaman Samuel L. , Algina James , Olejnik Stephen F.
Published: March 2008

Empirical type I error rates and the power of the parametric and rank transform ANCOVA were compared for situations involving conditional distributions that differed between groups in skew and/or scale. For the conditions investigated in the study, the parametric ANCOVA was typically the procedure of choice both as a test of equality of conditional means and as a test of equality of conditional distributions. For those conditions in which rank ANCOVA was the procedure of choice, the power advantages were usually quite small.
Error Probabilities in Educational and Psychological Research
Authors: Westermann Rainer , Hager Willi
Published: March 2008

The well-known problem of cumulating error probabilities is reconsidered from a general epistemological perspective, namely, the concepts of severity (Popper) and of fairness of tests. Applying these concepts to hypothesis-testing research leads to a reevaluation of the relative importance of the probabilities of Type 1 and Type 2 errors connected with those statistical hypotheses that have been derived from the substantive ones. It is shown that not only Type 1 but also Type 2 errors can cumulate. This cumulation is discussed for various basic types of empirical situations in which substantive hypotheses are examined by means of statistical ones. A new adjustment strategy based on the Dunn-Bonferroni inequality for planned tests is proposed and applied to some empirical examples.
Reliability of Composite Measurements Based on them Highest ofn Equivalent Components
Authors: Huynh Huynh
Published: March 2008

Under the assumptions of classical measurement theory and the condition of normality, a formula is derived for the reliability of composite scores based on m highest of n equivalent components. The formula represents an extension of the Spearman-Brown formula to the case of truncated data. The results of a simulation study indicate that errors encountered in the use of the said formula for projection purposes are confined largely to the second decimal.
Simultaneous Estimation of Regression Functions for Marine Corps Technical Training Specialties
Authors: Dunbar Stephen B. , Mayekawa Shin-Ichi , Novick Melvin R.
Published: March 2008

This paper considers the application of Bayesian techniques for simultaneous estimation to the specification of regression weights for selection tests used in various technical training courses in the Marine Corps. Results of a method for m-group regression developed by Molenaar and Lewis (1979) suggest that common weights for training courses belonging to certain general categories are justified in many cases. However, such commonality of regression weights does not appear to hold for all courses in these categories—weights for some training courses remain distinct even after the application of the simultaneous estimation procedure. Thus, a hypothesis of complete generalization of the predictor-criterion relationships across training courses in a given category would only be retained for a carefully selected subset of courses and not for all groups included in the analysis.
Determining the Most Significant Parametric Function for a Given Linear Hypothesis
Authors: Lutz J. Gary , Cundari Leigh A.
Published: March 2008

After a hypothesis about some linear statistical model has been tested and rejected (e.g., in an ANOVA), many researchers employ the Scheffe procedure to locate the source(s) of the rejection. This procedure guarantees that there is at least one linear combination of the model parameters (consistent with the hypothesis) that is significantly different from its hypothesized value. This most significant parametric function is not always easy to find, however, because it may not manifest itself in simple functions (such as pairwise contrasts between groups) or in “obvious” functions (such as those suggested by the graph of an interaction). A general solution to this problem is presented along with a practical example of its application.
Fisher’s Tanh−1 Transformation of the Correlation Coefficient and a Test for Complete Independence in a Multivariate Normal Population
Authors: Reddon John R.
Published: March 2008

Computer sampling from a multivariate normal spherical population was used to evaluate Type I error rates for a test of P = I based on Fisher’s tanh−1 variance stabilizing transformation of the correlation coefficient. The range of variates considered was 5 to 25 and Type I error rates were estimated for several sample sizes with 2,500 independent replications. Except for small samples the test was well behaved. After the test converges to an acceptable Type I error rate it is preferable to Box’s test of P = I.
The Rasch Model and Multistage Testing
Authors: Glas C. A. W.
Published: March 2008

This paper concerns the problem of estimating the item parameters of latent trait models in a multistage testing design. It is shown that using the Rasch model and conditional maximum likelihood estimates does not lead to solvable estimation equations. It is also shown that marginal maximum likelihood estimation, which assumes a sample of subjects from a population with a specified distribution of ability, will lead to solvable estimation equations, both in the Rasch model and in the Birnbaum model.
Approximation for Bayesian Ability Estimation
Authors: Tsutakawa Robert K. , Soltys Michael J.
Published: March 2008

An approximation is proposed for the posterior mean and standard deviation of the ability parameter in an item response model. The procedure assumes that approximations to the posterior mean and covariance matrix of item parameters are available. It is based on the posterior mean of a Taylor series approximation to the posterior mean conditional on the item parameters. The method is illustrated for the two-parameter logistic model using data from an ACT math test with 39 items. A numerical comparison with the empirical Bayes method using n = 400 examinees shows that the point estimates are very similar but the standard deviations under empirical Bayes are about 2% smaller than those under Bayes. Moreover, when the sample size is decreased to n = 100, the standard deviation under Bayes is shown to increase by 14% in some cases.
Bayes and Empirical Bayes Shrinkage Estimation of Regression Coefficients: A Cross-Validation Study
Authors: Nebebe Fassii , Stroud T. W. F.
Published: March 2008

A Bayesian and an empirical Bayes approach to shrinkage estimation of regression coefficients and the uses of these in prediction are investigated. The methods, along with least squares and least absolute deviations, are applied to data sets of different sizes and cross-validated with observations not in the data sets. The fully Bayes and empirical Bayes methods are seen to perform consistently better in predicting the response variable than either of least squares or least absolute deviations.
The Internal Correlation: Its Applications in Statistics and Psychometrics
Authors: Joe George W. , Mendoza Jorge L.
Published: March 2008

The internal correlation, a measure of dependency in a set of variables, is discussed and generalized. This coefficient is an upper bound to the product moment correlations, multiple correlations, and canonical correlations that can be defined in a set of variables. Applications of the internal correlation coefficient and its generalizations are given for a number of data-analytic situations. Where appropriate, we discuss tests of significance. We illustrate the internal correlation and expand the concept to a series of additional indices: local internal, up-internal, and down-internal correlations. Uses of these indices are illustrated in several areas: multicollinearity, ridge regression, factor analysis, principal components analysis, and test reliability.
Type I Error Rates and Power Estimates for Selected Two-Sample Tests of Scale
Authors: Algina James , Olejnik Stephen , Ocanto Romer
Published: March 2008

Estimated Type I error rates and power are reported for a modified Fligner-Killeen test, O’Brien’s test, O’Brien’s test using Welch’s modified ANOVA, the Brown-Forsythe test, and two tests developed by Tiku. Normal and nonnormal distributions and a two-group design were investigated. O’Brien’s test and the Brown-Forsythe test had estimated Type I error rates near the nominal level for all conditions investigated. To maximize power when the sample sizes are equal, O’Brien’s test should be used with platykurtic distributions and the Brown-Forsythe test with leptokurtic distributions. Either test can be used when the kurtosis is zero. When the sample sizes are unequal, O’Brien’s test should be used with platykurtic distributions and the Brown-Forsythe with symmetric-leptokurtic distributions. With other distributions, the tests have similar power.
The Average Response Method of Scaling
Authors: Beaton Albert E. , Johnson Eugene G.
Published: March 2008

The average response method (ARM) of scaling nonbinary data was developed to scale the data from the assessments of writing conducted by the National Assessment of Educational Progress (NAEP). The ARM applies linear models and multiple imputations technologies to characterize the predictive distribution of the person-level average of ratings over a pool of exercises when each person has responded to only a few of the exercises. The derivations of “plausible values” from the individual-level distributions of potential scale scores are given. Conditions are provided for the unbiasedness of estimates based on the plausible values, and the potential magnitude of the bias when the conditions are not met is indicated. Also discussed is how the plausible values allow for an accounting of the uncertainties due to the sampling of individuals and to the incomplete information on each sampled individual. The technique is illustrated using data from the assessment of writing.
Confidence Envelopes for Item Response Theory
Authors: Thissen David , Wainer Howard
Published: March 2008

Confidence envelopes for the one-, two-, and three-parameter logistic item response models are illustrated. In addition, we describe m-line plots, which show the genesis of the envelope as well as the density of lines in the confidence region. These too are illustrated for the one-, two-, and three- parameter logistic models.
Public/Catholic Differences in the High School and Beyond Data: A Multigroup Structural Equation Modeling Approach to Testing Mean Differences
Authors: Marsh Herbert W. , Grayson David
Published: March 2008

Previous research based on the large, nationally representative High School and Beyond (HSB) study has compared senior year achievement test scores for public and Catholic high school students after controlling for background variables and sophomore year test scores. These analyses, however, were based on traditional applications of multiple regression with its implausible assumptions that variables are measured without error and that residuals are uncorrelated. The present study demonstrates tests for mean differences on latent constructs using the LISREL approach to multigroup structural equation modeling (SEM) for this substantively important issue. Public/Catholic differences, even after controlling for background and sophomore outcomes, favored Catholic high school students on senior year outcomes (achievement, educational aspirations, and academic course selection) and subsequent college attendance. These public/Catholic differences were similar for students differing in race, SES (social economic status), and initial ability. Public/Catholic differences in achievement, educational aspirations, and college attendance were, however, apparently mediated by the academic orientation of course selection. The flexibility and advantages—but also the limitations—of this multigroup SEM approach are discussed.
Improved Significance Probabilities of the Wilcoxon Test
Authors: Hodges J. L. , Ramsey Philip H. , Wechsler Sergio
Published: March 2008

An Edgeworth approximation for accurate significance probabilities for the Wilcoxon two-sample test is substantially simplified. A method is developed that allows quick and easy calculations of very accurate probabilities. Exact formulas are given for most of the remaining cases, and tables are presented comparing the accuracy of the new simplification relative to the most likely alternatives.
Computation and Statistical Inference for Decision Consistency Indexes Based on the Rasch Model
Authors: Huynh Huynh
Published: March 2008

Procedures based on latent-trait models are described for computation and asymptotic statistical inference for two decision consistency indexes often used in mastery or competency testing. Details are given for the Rasch model. Simulations were conducted for some typical mastery testing situations with tests of 30 to 60 items and for raw score frequency distributions with small to marked levels of negative skewness. The results appear to indicate that, for these situations, the asymptotic normal theory may be safely used with samples of 100 subjects.
A Note on Levine’s Formula for Equating Unequally Reliable Tests Using Data From the Common Item Nonequivalent Groups Design
Authors: Hanson Bradley A.
Published: March 2008

Levine’s formula for equating unequally reliable tests using data collected in the common item nonequivalent groups equating design is an estimate of a linear function relating true scores on two test forms to be equated. Because a function relating true scores is applied to the observed score, it is not clear how the resulting converted observed score is in any sense comparable to the observed score it is being equated to. This article demonstrates that Levine’s formula can be interpreted as a method of moments estimate of an equating function that results in first order equity of the equated test score under a classical congeneric model.
The Isthmus of Acceptance: A Graphical Tool for Function-Based Item Analysis and Test Construction
Authors: Wainer Howard
Published: March 2008

Old fashioned measures of item discrimination, such as various item-test score correlations, are sensitive to many other aspects of the test data. Estimates of discrimination parameters in the most general of the binary IRT models are quite unstable. In this article, the isthmus of acceptance is proposed as a graphical tool for item analysis. It ameliorates some of the parameter stability problems by utilizing a functional representation of the item’s performance. As currently proposed, this tool is tuned to match current standards, when they are appropriately used and correctly applied. Recommendations are made as to fruitful avenues of prospective use; one use is illustrated.
Statistical Models for Behavioral Observations
Authors: Rogosa David , Ghandour Ghassan
Published: March 2008

Statistical models for the Behavior Stream (renewal processes) are used to explore properties of behavioral observations. For continuous observation, statistical and psychometric properties are obtained for four types of behavioral measures: (i) empirical rates of behavior; (ii) empirical proportions, or relative frequencies, of a type of behavior; (iii) empirical prevalence (proportion of time the behavior occurs); and (iv) empirical event duration. Also, time-sampling alternatives to continuous observation are evaluated. Our formulation includes representations for three sources of unreliability: (a) finite observation time, (b) recorder errors, and (c) heterogeneity (instability) over occasions of observation. Traditional psychometric methods carried over from the analysis of responses to test items (including generalizability theory) are shown not to be applicable to behavioral observations. Our results provide a guide for design and a framework for statistical analysis in behavioral observation research.
A Correction for the ε̃ Approximate Test in Repeated Measures Designs With Two or More Independent Groups
Authors: Lecoutre Bruno
Published: March 2008

The routine ε̃ approximate test procedure in repeated measures designs when the condition of circularity is not fulfilled uses an erroneous formula in the case of two or more groups. This may lead to a substantial underestimation of the deviation from circularity when the total number of subjects is small.
Chapter 2: Sampling and Weighting in the National Assessment
Authors: Rust Keith F. , Johnson Eugene G.
Published: March 2008

This chapter describes procedures for obtaining the National Assessment of Educational Progress (NAEP) student samples used in the national and state assessments and for deriving survey weights for use in the analysis of the survey data. Following the description of general procedures, more detailed discussion is included about several issues that relate to the procedures used. In some cases, these involve procedures that NAEP is actively reviewing and investigating, with a view toward implementing improvements in the future. In other cases, the procedures, although well established in NAEP, involve technical aspects with interesting features not fully described in the available technical reports.
Meta-Analysis
Authors: Hedges Larry V.
Published: March 2008

The use of statistical methods to combine the results of independent empirical research studies (meta-analysis) has a long history. Meta-analytic work can be divided into two traditions: tests of the statistical significance of combined results and methods for combining estimates across studies. The principal classes of combined significance tests are reviewed, and the limitations of these tests are discussed. Fixed effects approaches treat the effect magnitude parameters to be estimated as a consequence of a model involving fixed but unknown constants. Random effects approaches treat effect magnitude parameters as if they were sampled from a universe of effects and attempt to estimate the mean and variance of the hyperpopulation of effects. Mixed models incorporate both fixed and random effects. Finally, areas of current research are summarized, including methods for handling missing data, models for publication selection, models to handle studies that are not independent, and distribution-free models for random effects.
Summarizing Monte Carlo Results in Methodological Research: The One- and Two-Factor Fixed Effects ANOVA Cases
Authors: Harwell Michael R. , Rubinstein Elaine N. , Hayes William S. , Olds Corley C.
Published: March 2008

Meta-analytic methods were used to integrate the findings of a sample of Monte Carlo studies of the robustness of the F test in the one- and two-factor fixed effects ANOVA models. Monte Carlo results for the Welch (1947) and Kruskal-Wallis (Kruskal & Wallis, 1952) tests were also analyzed. The meta-analytic results provided strong support for the robustness of the Type I error rate of the F test when certain assumptions were violated. The F test also showed excellent power properties. However, the Type I error rate of the F test was sensitive to unequal variances, even when sample sizes were equal. The error rate of the Welch test was insensitive to unequal variances when the population distribution was normal, but nonnormal distributions tended to inflate its error rate and to depress its power. Meta-analytic and exact statistical theory results were used to summarize the effects of assumption violations for the tests.
Using Results From Replicated Studies to Estimate Linear Models
Authors: Becker Betsy Jane
Published: March 2008

This article outlines analyses for the results of a series of studies examining intercorrelations among a set of p + 1 variables. A test of whether a common population correlation matrix underlies the set of empirical results is given. Methods are presented for estimating either a pooled or average correlation matrix, depending on whether the studies appear to arise from a single population. A random effects model provides the basis for estimation and testing when the series of correlation matrices may not share a common population matrix. Finally, I show how a pooled correlation matrix (or average matrix) can be used to estimate the standardized coefficients of a regression model for variables measured in the series of studies. Data from a synthesis of relationships among mathematical, verbal, and spatial ability measures illustrate the procedures.
Thin Versus Thick Matching in the Mantel-Haenszel Procedure for Detecting DIF
Authors: Donoghue John R. , Allen Nancy L.
Published: March 2008

This Monte Carlo study examined strategies for forming the matching variable for the Mantel-Haenszel (MH) differential item functioning (DIF) procedure; thin matching on total test score was compared to forms of thick matching, pooling levels of the matching variable. Data were generated using a three-parameter logistic (3PL) item response theory (IRT) model with common guessing parameter. Number of subjects and test length were manipulated, as were the difficulty, discrimination, and presence/absence of DIF in the studied item. Outcome measures were the transformed log-odds &Deltacirc; MH, its standard error, and the MH chi-square statistic. For short tests (5 or 10 items), thin matching yielded very poor results, with a tendency to falsely identify items as possessing DIF against the reference group. The best methods of thick matching yielded outcome measure values closer to the expected value for non-DIF items, as well as a larger value than thin matching when the studied item possessed DIF. Intermediate length tests yielded similar results for thin matching and the best methods of thick matching. The method of thick matching that performed best depended on the measure used to detect DIF. Both difficulty and discrimination of the studied item were found to have a strong effect on the value of &Deltacirc; MH.
A Bayesian Approach to Estimation of GPAs of University of Iowa Freshmen Under Order Restrictions
Authors: Albert James H.
Published: March 2008

The problem of interest is to estimate a two-way table of means of grade point averages (GPAs) of University of Iowa freshmen grouped by levels of ACT scores and high school percentiles. Suppose one believes a priori that the table means satisfy a specific partial order. The use of two different classes of prior distributions is considered in modeling this prior information. The use of simulation and, in particular, the Gibbs sampler are outlined to summarize various posterior distributions of interest. The posterior distribution is used to predict the grade point averages of future Iowa freshmen.
Permutation Tests for Common Locations Among Samples With Unequal Variances
Authors: Mielke Paul W. , Berry Kenneth J.
Published: March 2008

In completely randomized experimental designs where population variances are equal under the null hypothesis, it is not uncommon to have multiplicative treatment effects that produce unequal variances under the alternative hypothesis. Permutation procedures are presented to test for (a) median location and scale shifts, (b) scale shifts only, and (c) mean location shifts only. Corresponding multivariate extensions are provided. Location-shift power comparisons between the parametric Bartlett-Nanda-Pillai trace test and three alternative multivariate permutation tests for five bivariate distributions are included.
Efficient Analysis of Mixed Hierarchical and Cross-Classified Random Structures Using a Multilevel Model
Authors: Rasbash Jon , Goldstein Harvey
Published: March 2008

An efficient and straightforward procedure is described for specifying and estimating parameters of general mixed models which contain both hierarchical and crossed random factors. This is done using a model formulated for purely hierarchically structured data and generalizes the results of Raudenbush (1993) . The exposition is for the continuous response linear model with natural extensions to generalized linear, nonlinear, and multivariate models.
Linking to a Large-Scale Assessment: An Empirical Evaluation
Authors: Bloxom Bruce , Pashley Peter J. , Nicewander W. Alan , Yan Duanli
Published: March 2008

This article develops and evaluates a linkage of a routinely administered measure, the Armed Services Vocational Aptitude Battery (ASVAB), to a large-scale assessment, the mathematics proficiency scale of the National Assessment of Educational Progress (NAEP). Analyses of both simulated and real data indicated that the statistical accuracy of the proficiency distribution estimated by projection (i.e., by regression on the ASVAB sub-scales) was very close to the accuracy obtained by the use of the NAEP methodology. However, for both methods, proficiency may have been systematically underestimated because of motivational factors in the administration of the NAEP measures to examinees in this study.