bluuebunny/crossref_metadata_2025 huggingface dataset:
Using Longitudinal Data to Model Teachers' Ratings of Classroom Behavior as a Dynamic Process
Authors: Plewis Ian
Published: March 2008

Simple Markov models are fitted to a small sample of longitudinal categorical data of teachers' ratings of children's classroom behavior. Although the data consist only of observations at 5 occasions, it was possible, after dividing the data into two groups, to fit plausible models in continuous time. Measurement error and alternative longitudinal designs are discussed, and some possible educational implications are noted.
Two Multinomial Random Sociometric Voting Models
Authors: Shapiro Stewart B. , Sobel Milton
Published: March 2008

Two multinomial random sociometric voting models are presented to study the critical number of votes that can occur under peer voting schemes. Assumed in Model I are equal probabilities of votes for each candidate other than self. The given conditions are that a fixed number of ballots are cast (denoted by N) and there are exactly b-N abstentions (N ≤ b). Model I incorporates the restriction of not voting for self; Model II is presented without this restriction. Abstentions are possible under both models. Empirical evidence for the convergence of the two models is presented so that Model II can be substituted for Model I in groups where b ≥ 8 regardless of the value of N (ballots cast). This substitution greatly simplifies calculation of the critical values (r) of sociometric frequencies attributed in classroom groups which vary in size. Appendices present tables for Models I and II for various values of r when b and N are less than or equal to 8. A table is also given for Model II up to N=b=50.
On the Use of a Cumulative Distribution as a Utility Function in Educational or Employment Selection
Authors: Chen James J. , Novick Melvin R.
Published: March 2008

Formal decision theory can make important contributions to educational or employment decision-making, provided one can quantify the utilities of different possible outcomes such as test scores and grade-point averages. Because utility is usually a monotonic increasing function of performance score, a cumulative probability function may be convenient for describing one’s utilities. Moreover, calculations of expected utility of a decision are greatly simplified when the utility and the probability function have the same functional form, e.g., both are normal. A least-squares procedure, developed by Lindley and Novick for fitting a utility function, is applied to truncated normal and extended beta distribution functions. The truncated normal and beta distributions avoid the symmetry and infinite range restrictions of the normal distribution and can provide fits in some cases in which the normal functional forms cannot provide a reasonable fit.
Assessing Efficiency of Decisions in Mastery Testing
Authors: Huynh Huynh
Published: March 2008

Two indices for assessing the efficiency of decisions in mastery testing are proposed. The indices are generalizations of the raw agreement index and the kappa index. Both express the reduction in the proportion of average loss (or the gain in utility) resulting from the use of test scores to make decisions. Empirical data are presented which show little discrepancy between estimates based on the beta-binomial and compound binomial models for one index.
Bayesian Estimation in the Rasch Model
Authors: Swaminathan Hariharan , Gifford Janice A.
Published: March 2008

Bayesian estimation procedures based on a hierarchical model for estimating parameters in the Rasch model are described. Through simulation studies it is shown that the Bayesian procedure is superior to the maximum likelihood procedure in that the estimates are (a) more accurate, at least in small samples; and (b) meaningful in that parameters corresponding to perfect item and ability responses can be estimated.
Detection of Aberrant Response Patterns and their Effect on Dimensionality
Authors: Tatsuoka Kikumi K. , Tatsuoka Maurice M.
Published: March 2008

Two indices were developed for measuring the degree of conformity or consistency of an individual examinee’s response pattern on a set of items. The first, called the norm conformity index (NCI), measures the proximity of the pattern to a baseline pattern in which all 0’s precede all l’s when the items are arranged in some prescribed order. The second, called the individual consistency index (ICI), measures the extent to which an individual’s response pattern remains invariant when he or she responds to several waves of parallel items. We show how these two indices, used in conjunction with test scores, can spot aberrant response patterns of students requiring detailed error diagnosis and remedial treatment.
Optimal Randomized Response Models and Methods for Hypothesis Testing
Authors: Carr John W. , Marascuilo Leonard A. , Busk Patricia
Published: March 2008

This paper reviews several randomized response models for gathering self-report data when persons are asked sensitive questions, and introduces a new general model for reducing the standard error by asking questions conditional upon earlier answers. Hypothesis testing and confidence-interval procedures are demonstrated for data collected by the randomized response technique.
Irt Versus Conventional Equating Methods: A Comparative Study of Scale Stability
Authors: Petersen Nancy S. , Cook Linda L. , Stocking Martha L.
Published: March 2008

Scale drift for the verbal and mathematical portions of the Scholastic Aptitude Test (SAT) was investigated using linear, equipercentile and item response theory (IRT) equating methods. The linear methods investigated were the Tucker, Levine Equally Reliable and Levine Unequally Reliable models. Three IRT calibration designs were employed. These designs are referred to as (1) concurrent, (2) fixed b’s method, and (3) characteristic curve transformation method.

The results of the various equating methods were compared both graphically and analytically. These results indicated that for reasonably parallel tests, linear equating methods perform adequately. However, when tests differ somewhat in content and length, methods based on the three-parameter logistic IRT model lead to greater stability of equating results. Of the conventional equating methods investigated, the Levine Equally Reliable model appears to be the most robust for the type of equating situation used in this study. The IRT method that provided the most stable equating results overall was the concurrent calibration method.
An Item Response Model for the Estimation of Demographic Effects
Authors: Reiser Mark
Published: March 2008

Item sampling procedures employed in many assessment studies are designed so that each respondent answers a small number of items in each of a large number of skill areas. If the item population contains several items designed to measure an underlying variable such as computation skill, it may be desirable to fit an item response model to the data. In studies that employ multiple matrix sampling there are not enough answers from each individual to employ such a model. To circumvent this problem, a model is formulated on the assumption that individual level variability appears as independent error within the cells of the cross classification of manifest demographic variables such as sex and race. This model is successfully fit to a scale of items, involving addition of fractions, from the National Assessment Study.
Planned and Post Hoc Comparisons in Tests of Concordance and Discordance forG Groups of Judges
Authors: Serlin Ronald C. , Marascuilo Leonard A.
Published: March 2008

Statistical procedures for making comparisons on the ranking of a set ofK elements, wines, people, and so forth, have been worked out for the case in which the ranking is done by a single group ofn people. In this special situation, Friedman’s test can be called upon to assess the significance of differences in mean ranking, and Kendall’s coefficient of concordance can be used to measure the level of agreement among the individuals doing the ranking.

Unfortunately, the problem of comparingG groups of subjects to determine whether or not they produce the same mean ranking has, until recently, remained unanswered. A procedure was developed for testing two-group concordance by Schucany and Frawley. Hollander and Sethuraman have questioned this procedure and have provided an alternative model for investigating the same problem. It is easy to show that the Schucany and Frawley model tests for positive rank correlation in the mean ranks as an alternative, whereas the Hollander and Sethuraman model tests identity of mean ranks across theK compared elements as the null. In this article, these two models are examined, the criticism is evaluated, and both models are extended to theG group situation. The extension to planned and post hoc comparisons is illustrated using data based on a study described by Marascuilo and Dagenais.
Covariance Structures Under Polynomial Constraints: Applications to Correlation and Alpha-Type Structural Models
Authors: Bentler P. M. , Lee Sik-Yum
Published: March 2008

This paper provides methods for the estimation of covariance structure models under polynomial constraints. Estimation is based on maximum likelihood principles under constraints, and the test statistics, parameter estimates, and standard errors are based on a statistical theory that takes into account the constraints. The approach is illustrated by obtaining statistics for the squared multiple correlation, for predictors in a standardized metric, and in the analysis of longitudinal data via old and new models having constraints that cannot be obtained by standard methods.
Test Scores as Social Statistics: Comparing Distributions
Authors: Spencer Bruce D.
Published: March 2008

If test scores are considered ordinal but not cardinal attributes, the average test score may not be a meaningful way to summarize the scores of a group of individuals. Similarly, correlation coefficients may not form meaningful summary measures of association between test scores. Using the theory of stochastic ordering and quadrant dependence, new techniques that do not depend on any particular test scale are developed for reporting, interpreting, and comparing distributions of test scores. The use of a relevant test scale is considered also, as well as measures of distance between distributions.
Maximum Likelihood Analysis for a Generalized Regession-Discontinuity Design
Authors: Visser Ronald A. , De Leeuw Jan
Published: March 2008

The regression-discontinuity design (RDD) offers the possibility of making inferences about causal effects from observations on selected groups. The quasi-experimental groups are formed by dividing the scores of a premeasurement in two halves. The treatment effect is inferred from the differences between the regression of a postmeasurement on the premeasurement for the two groups. We discuss a generalized form of this design: (a) Apart from parallel shift of the regression lines, differences in variance and covariance are considered; (b) pretest and posttest may be multivariate; and (c) more than two groups may be involved in the design. Data from such a design are considered to have a truncated bivariate distribution. For the RDD, maximum likelihood parameter estimation procedures and tests of hypotheses are presented.
Accounting for Statistical Artifacts in Item Bias Research
Authors: Shepard Lorrie , Camilli Gregory , Williams David M.
Published: March 2008

Theoretically preferred IRT bias detection procedures were applied to both a mathematics achievement and vocabulary test. The data were from black and white seniors on the High School and Beyond data files. To account for statistical artifacts, each analysis was repeated on randomly equivalent samples of blacks and whites ( n’s = 1,500). Furthermore, to establish a baseline for judging bias indices that might be attributable only to sampling fluctuations, bias analyses were conducted comparing randomly selected groups of whites. To assess the effect of mean group differences on the appearance of bias, pseudo-ethnic groups were created, that is, samples of whites were selected to simulate the average black-white difference.

The validity and sensitivity of the IRT bias indices was supported by several findings. A relatively large number of items (10 of 29) on the math test were found to be consistently biased; they were replicated in parallel analyses. The bias indices were substantially smaller in white-white analyses. Furthermore, the indices (with the possible exception of χ2) did not find bias in the pseudo-ethnic comparison. The pattern of between-study correlations showed high consistency for parallel ethnic analyses where bias was plausibly present. Also, the indices met the discriminant validity test—the correlations were low between conditions where bias should not be present. For the math test, where a substantial number of items appeared biased, the results were interpretable. Verbal math problems were systematically more difficult for blacks.

Overall, the sums-of-squares statistics (weighted by the inverse of the variance errors) were judged to be the best indices for quantifying ICC differences between groups. Not only were these statistics the most consistent in detecting bias in the ethnic comparisons, but they also intercorrelated the least in situations of no bias.
An Improved Internal Consistency Reliability Estimate
Authors: Cliff Norman
Published: March 2008

Kuder-Richardson Formula 20 was derived on the assumption that the average covariance between items on different forms is the same as the average covariance on the same form. This formula is considered an underestimate of the parallel form reliability because the average covariance of items with identical difficulty is expected to be higher than the average covariance of items of different difficulty.

The proposed coefficient is derived by assuming instead that the average Goodman-Kruskal gamma between items of identical difficulty would be the same as the average for items of different difficulty. Given the average gamma for the present items, this permits an estimate of the covariance between items of identical difficulty. This leads directly to an estimate of the correlation between scores on the present test and those on a form that had an identical distribution of difficulty. Some examples are given.
The Biases and Mean Squared Errors of Estimators of Multinormal Squared Multiple Correlation
Authors: Lucke Joseph F. , Embretson Whitely Susan
Published: March 2008

The sample squared multiple correlation coefficient, R2, is known to have certain unsatisfactory properties as an estimator of the population squared multiple correlation. Hence, numerous adjusted estimators based on functions of 1 – R2 have been proposed. We examine the biases and mean squared errors of five adjusted estimators as well as R2. General results are given for estimators linear in 1 – R2, and four such estimators are examined in detail. In addition, a quadratic estimator and the minimum variance unbiased estimator are examined. Comparisons among these estimators are made in terms of absolute bias and mean squared error.
On Criterion-Referenced Grading Models
Authors: Klauer Karl Josef
Published: March 2008

This article presents two criterion-referenced grading models developed in Germany, and shows how they can be applied in the classroom setting. The models are a binomial grading model ( Lindner, 1980 ) and an arcsine grading model ( Klauer, 1982 ). In both cases, one can assign grades on the basis of a criterion-referenced test with a systematic control of the possible errors of misclassification.
Estimation of Two-Parameter Logistic Item Response Curves
Authors: Tsutakawa Robert K.
Published: March 2008

Under the assumption that ability parameters are sampled from a normal distribution, the EM algorithm is used to derive maximum likelihood estimates for item parameters of the two-parameter logistic item response curves. The observed information matrix is then used to approximate the covariance matrix of these estimates. Responses to a questionnaire on general arthritis knowledge are used to illustrate the procedure and simulated data are used to compare the estimated and actual item parameters. The resulting estimates are found to be very close to those obtained from LOGIST. A computational note is included to facilitate the extensive numerical work required to implement the procedure.
Measuring the Rate of Return to Interrupted Schooling
Authors: Marcus Richard D.
Published: March 2008

The most widely followed technique to estimate the rate of return to a year of schooling was provided by Mincer (1974) . This paper extends Mincer's semilog wage regression method to include those who interrupted their schooling with years of work. Schooling and the duration of the interruption interact to create nonlinearities in the rate of return to schooling. The proposed method is then applied to both Vietnam era G.I. students and civilian interrupters. It is found that interrupters earn substantially the same rate of return as the rate of return to uninterrupted schooling at the same level of schooling. G.I. students earned slightly higher rates of return to their interrupted schooling, but their accumulated work experience was not valued highly in the labor market.
Tolerance Intervals for True Scores
Authors: Jarjoura David
Published: March 2008

Issues regarding tolerance and confidence intervals are discussed within the context of educational measurement and conceptual distinctions are drawn between these two types of intervals. Points are raised about the advantages of tolerance intervals when the focus is on a particular observed score rather than a particular examinee. Because tolerance intervals depend on strong true score models, a practical implication of the study is that true score tolerance intervals are fairly insensitive to differences in assumptions among the five models studied.
Generalizability Theory: The Software Problem
Authors: Bell John F.
Published: March 2008

This paper outlines the problems associated with the estimation of variance components in generalizability analyses using analysis of variance software, and discusses the most useful software currently available for this specialist application: the MIVQUE method of the SAS procedure VARCOMP.
A Probabilistic Model for Diagnosing Misconceptions By The Pattern Classification Approach
Authors: Tatsuoka Kikumi K.
Published: March 2008

This paper introduces a probabilistic approach to the classification and diagnosis of erroneous rules of operations that result from misconceptions (“bugs”) in a procedural domain of arithmetic. The model is different from the usual deterministic strategies common in the field of artificial intelligence because variability of response errors is explicitly treated through item response theory. As a concrete example, we analyze a dataset that reflects the use of erroneous rules of operation in problems of signed-number subtraction. The same approach, however, is applicable to the classification of several different groups of response patterns caused by a variety of different underlying misconceptions, different backgrounds of knowledge, or treatment.
Anova of Residuals and Ancova: Correcting an Illusion by Using Model Comparisons and Graphs
Authors: Maxwell Scott E. , Delaney Harold D. , Manheimer Jerry M.
Published: March 2008

Analysis of covariance is often conceptualized as an analysis of variance of a single set of residual scores that are obtained by regressing the dependent variable on the covariate. Although this conceptualization of an equivalence between the two procedures may be intuitively appealing, it is mathematically incorrect. If residuals are obtained from the pooled within-groups regression coefficient ( bw), an analysis of variance on the residuals results in an inflated α-level. If the regression coefficient for the total sample combined into one group ( bT) is used, ANOVA on the residuals yields an inappropriately conservative test. In either case, analysis of variance of residuals fails to provide a correct test, because the significance test in analysis of covariance requires consideration of both bw and bT, unlike analysis of residuals. It is recommended that the significance test of treatment effects in analysis of covariance be conceptualized, not as an analysis of residuals, but as a comparison of models whose parameters are estimated by the principle of least squares. Focusing on model comparisons and their associated graphs can be used effectively here as in other cases to teach simply and correctly the logic of the statistical test.
Statistical Consulting in the University: A Faculty Member’s Perspective
Authors: Stegman Charles E.
Published: March 2008

Statistical consultants share many common roles and problems. This article addresses some of the issues faced by the educational statistician working as a faculty member in a university. The university milieu, with its emphasis on research, teaching, and service presents some unique opportunities as well as problems for the statistical consultant. The roles, functions, and responsibilities of the educational statistician are emphasized.
Educating and Training Undergraduate Applied Statisticians
Authors: Peres Clovis A. , Morettin Pedro A. , Narula Subhash C.
Published: March 2008

A course on Applied Statistics, offered since 1978 at the Instituto de Matemática e Estatística, Universidade de São Paulo, Brasil, is designed to educate statisticians at the bachelor’s level for jobs in government statistical offices, industry, and business. Because most of the statistical work at these places is carried out by bachelor’s level individuals, such a course is almost mandatory for developing countries and may be useful for other countries. Our objective is to share our experience with the course.
On “State Education Statistics”
Authors: Wainer Howard , Holland Paul W. , Swinton Spencer , Wang Min Hwei
Published: March 2008

In January 1984 and again in January 1985, then Secretary of Education Bell released the table “State Education Statistics.” These tables contained a variety of education indicators, among them average SAT or ACT scores for each state. In this paper we examine these scores to see if they can be used for state-by-state comparisons to aid in the evaluation of those educational policies that vary across states. We conclude that statistical adjustment to remove the bias introduced by inappropriate aggregation and self-selection of examinees is not sufficient to insure the validity of the kinds of inferences that are desired.
Discussion of “on State Education Statistics”: A Difficulty with Regression Analyses of Regional Test Score Averages
Authors: Rosenbaum Paul R. , Rubin Donald B.
Published: March 2008

The Department of Education’s table “State Education Statistics” reports mean test scores by state and mean resource inputs by state. The means are calculated from quite different groups of students, a process we call inconsistent aggregation. We investigate the bias in regression coefficients caused by inconsistent aggregation, first using theoretical calculations, and then by artificially aggregating data from the High School and Beyond sample.
A Comparison of the Minimax and Rasch Approaches to Set Simultaneous Passing Scores for Subtests
Authors: Huynh Huynh , Casteel Jim
Published: March 2008

Two approaches are described for the simultaneous determination of passing scores for subtests when the passing score for the total test is known. The minimax approach seeks subtest passing scores in such a way that there is maximum agreement between pass-fail classifications based on each subtest and pass-fail decisions based on the entire test. The Rasch procedure seeks subtest passing scores that occupy approximately the same place on the ability scale with the total test passing score. In the context of a basic skills assessment program, the two approaches yield essentially the same set of passing scores.
A Comparison of the Power of the Paired Samples Rank Transform Statistic to that of Wilcoxon’s Signed Ranks Statistic
Authors: Blair R. Clifford , Higgins James J.
Published: March 2008

This study was concerned with the effects of reliability of observations, sample size, magnitudes of treatment effects, and the shape of the sampled population on the relative power of the paired samples rank transform statistic and Wilcoxon’s signed ranks statistic. It was found that factors favoring the Wilcoxon statistic were high reliability of observations, moderate to large sample sizes, and small treatment effects. Factors favoring the rank transform statistic were low reliability of observations, small sample size, and moderate to large treatment effects. It was also noted that the Wilcoxon statistic appeared to maintain the power advantage under normal theory assumptions.
Bayesian Full Rank Marginalization for Two-Way Contingency Tables
Authors: Leonard Tom , Novick Melvin R.
Published: March 2008

A general approach is proposed for modeling the structure of an r × s contingency table and for drawing marginal inferences about all parameters (e.g., interaction effects) in the model. The main approach is relevant whenever rs − r − s + 1≥5. The approach may also be used to check the adequacy of Rasch’s multiplicative Poisson model. In general, the posterior estimates of the cell probabilities compromise between the cell frequencies and the fitted values obtained under the reduced model in the spirit of Leonard (1975) . It is also possible to compute reasonable approximations to the full posterior densities of many parameters of interest, following Leonard (1982) and Tierney and Kadane (1984) . All prior parameters are evaluated with the assistance of the data via a hierarchical Bayes procedure, thus reducing the subjectivity involved in the analysis. An r × s cross classification of 5,648 Marine Corps clerical students by school and test grade is analyzed in detail, and the posterior densities of the 96 possible interactions are used to suggest a simplified structure partitioning and collapsing the table into a meaningful 3 × 2 table.
Random Coefficient Models for Multilevel Analysis
Authors: De Leeuw Jan , Kreft Ita
Published: March 2008

We propose a possible statistical model for both contextual analysis and slopes as outcomes analysis. These techniques have been used in multilevel analysis for quite some time, but a precise specification of the regression models has not been given before. We formalize them by proposing a random coefficient regression model, and we investigate its statistical properties in some detail. Various estimation methods are reviewed and applied to Dutch school-career example.
Robustness of Irt Parameter Estimation to Violations of The Unidimensionality Assumption
Authors: Harrison David A.
Published: March 2008

Multidimensional item response data were created from a hierarchical factor model under a variety of conditions. The strength of a second-order general factor, the number of first-order common factors, the distribution of items loading on those common factors, and the number of items in simulated tests were systematically manipulated. The computer program LOGIST effectively recovered both item parameters and trait parameters implied by the general factor in nearly all of the experimental conditions. Implications of these findings for computerized adaptive testing, investigations of item bias, and other applications of item response theory models are discussed.
Comment on “Statistical Power with Group Mean as the Unit of Analysis”
Authors: Blair R. Clifford , Higgins James J.
Published: March 2008

Barcikowski (1981) has provided tables for use in situations where unit (e.g., class or school) means are to be used as the unit of analysis. Apparently unrecognized by Barcikowski, and perhaps by the research community at large, is the fact that under the conditions specified for use of these tables (i.e., knowledge of the intraclass correlation), one would not generally wish to carry out analyses based on group means since a superior analytic strategy is available in this situation. This paper explicates this methodology.
A Comparison of Efficiencies of Longitudinal, Mixed Longitudinal, and Cross-Sectional Designs
Authors: Berger Martijn P. F.
Published: March 2008

The choice among a longitudinal, mixed longitudinal, or cross-sectional design is often called for in educational and psychological research. The problem of choosing the most efficient design to estimate polynomial parameters for time-structured data is considered, and the comparison of the efficiencies shows that the assumed degree of the polynomial is crucial for the selection of the most efficient design. When the degree is much smaller than the number of time points and the correlations between adjacent time points are not too large, cross-sectional and mixed longitudinal designs are more efficient than a longitudinal design.
Simultaneous Confidence Intervals for the Linear Functions of Expected Mean Squares Used in Generalizability Theory
Authors: Bell John F.
Published: March 2008

This paper demonstrates a method, derived by Khuri (1981) , of constructing simultaneous confidence intervals for functions of expected values of mean squares obtained when analyzing a balanced design by a random effects linear model. The method may be applied to obtain confidence intervals for the variance components and other linear functions of the expected mean squares used in generalizability theory, with probability of simultaneous coverage guaranteed to be greater than or equal to the specified confidence coefficient. The Khuri intervals are compared with the approximate intervals obtained by using Satterthwaite’s (1941 , 1946) method in conjunction with Bonferroni’s inequality.
Minority Contributions to the Sat Score Turnaround: an Example of Simpson’s Paradox
Authors: Wainer Howard
Published: March 2008

Since 1980 the decline in SAT scores has stopped and the scores have started to creep back up. The scores for white Americans have increased 8 points during this period and 15 points for non-whites. It was thus surprising to discover that the overall mean increased only 7 points. This is not an arithmetic error but rather an example of a well-known statistical phenomenon called Simpson’s Paradox. In this note we explain the paradox and describe a method that will avoid it in the future.
Comparing Medians: A Monte Carlo Study
Authors: Wilcox Rand R. , Charlin Ventura L.
Published: March 2008

Several procedures have been proposed for testing H0:θ x = θ y. where θ x and θ y are the medians corresponding to two independent treatment groups. Unfortunately, most of these procedures have been found to be inadequate. For a variety of distribution shapes, this paper compares the performance of three large-sample tests of H0 for sample sizes ranging from 11 to 25. Although two of these procedures are simple extensions of existing results, they have not been previously considered in terms of testing θ x = θ y. The method that gave the best results is based on a normal approximation of the distribution of the sample median where the variance is estimated using results reported by Maritz and Jarrett (1978). The Fligner-Rust procedure is found to be inadequate in situations not considered in previous investigations, but it performs well when the sample sizes are equal and reasonably large.
Selecting an Equating Method: Linear or Equipercentile?
Authors: Budescu David V.
Published: March 2008

A new procedure for comparing results of linear and equipercentile equating methods is presented and illustrated. The proposed procedure requires (a) approximating the empirical score distributions of the two forms by means of the first terms of an infinite series, and (b) contrasting the results obtained when only the first two moments are used (i.e., linear equating) with more complete representations. The procedure is demonstrated by calculating the Kolmogorov-Smirnov and Cramer-Von Mises measures of discrepancy for a large number of unimodal distributions approximated by the Edgeworth and Gram-Charlier expansions. The results indicate that both statistics can be accurately predicted on the basis of the skewness and kurtosis of the two distributions. The most attractive feature of the proposed method is that it allows one to calculate the expected degree of efficiency in equating that can be achieved by linear equating, based on the moments of the two relevant distributions.
As Others See Us: A Case Study in Path Analysis
Authors: Freedman D. A.
Published: March 2008

In 1967, Blau and Duncan proposed a path model for education and stratification. This is one of the most influential applications of statistical modeling technique to social data. There is recent use of the same technique in Hope’s (1984) comparative study of Scotland and the United States, As Others See Us: Schooling and Social Mobility in Scotland and the United States. A review of path analysis is offered here, with Hope’s model used as an example, the object being to suggest the limits of the method in analyzing complex phenomena.
Examining Correlates of Diversity
Authors: Raudenbush Stephen W. , Bryk Anthony S.
Published: March 2008

Statistical methods are presented for studying “correlates of diversity”: characteristics of educational organizations which predict dispersion on the dependent variable. The conceptual framework for these methods distinguishes between variance heterogeneity that arises from educational program effects and heterogeneity that merely reflects heterogeneity of variance of inputs. The estimation theory is empirical Bayes, requiring probabilistic models both for the data and for the random dispersion parameters from each of many groups. Two strategies are considered, one based on exact distribution theory and the second based on an asymptotic normal approximation. The accuracy of the approximation is evaluated analytically and its use illustrated by an analysis of mathematics achievement data from a random sample of U.S. high schools.
A Heteroscedastic ANOVA Procedure With Specified Power
Authors: Wilcox Rand R.
Published: March 2008

When testing the equality of the means of J independent normal distributions, two-stage procedures have the advantage of providing exact control over both Type I errors and power even when the variances are unequal. Single-stage procedures have been proposed for handling unequal variances, but recent results (which are briefly reviewed in this paper) have shown that in certain practical situations these approximate solutions give unsatisfactory control over both Type I errors and power. A practical problem with two-stage procedures is the assumption that an equal number of observations is sampled from each treatment group in the first stage. Of course, for various reasons, a researcher might conduct a study resulting in unequal sample sizes. This paper describes a simple yet accurate method for dealing with this problem.
Corrections for Attenuation and Range Restriction on the Predictor
Authors: Mendoza Jorge L. , Mumford Michael
Published: March 2008

Researchers often need to obtain an estimate of the relationship between a predictor and a criterion under conditions where the magnitude of the correlation coefficient has been distorted by an error component in the predictor and criterion variables (attenuation) and restriction in the range of the predictor. The present paper delineates the factors that influence the joint impact of attenuation and range restriction on the magnitude of the correlation coefficient, and establishes a theoretical framework for the generation of appropriate correction formulas. A set of equations for making joint attenuation and range restriction corrections when there is attenuation and restriction on the predictor is presented. Finally, it is shown that the concept of predictor reliability changes in selection situations because of the correlation between error and true scores.
Estimating Ability With the Wrong Model
Authors: Wainer Howard , Thissen David
Published: March 2008

No model is ever a perfect reflection of the data it is to summarize. There are always errors of fit. This is as true with modern item response theory (IRT) as with all other models. It is important to know to what extent the accuracy of measurement made with these models is influenced by misfit and what can be done to minimize the inaccuracy. First, a detailed general model was fit to data to provide the framework for a realistic simulation structure. Then three of the most commonly used IRT models were fit in this simulation. A variety of robust estimators of ability were used and the accuracy and efficiency of each estimator was determined. With short tests, a simple model coupled with a robust estimator seemed to be the methodology of choice for describing the data. As test length increased, so too did the benefits of utilizing a more complex parameterization. An unexpected finding was that coupling robust estimators with a Bayesian prior yielded substantial shrinkage. Future work on ability estimation, especially for practical applications of adaptive testing, is required to “unshrink” ability estimates.
Meaningless Statistics
Authors: Marcus-Roberts Helen M. , Roberts Fred S.
Published: March 2008

We discuss the controversy over the limitations that scales of measurement impose on the statistical procedures we may apply, using the measurement-theoretic concept of meaningfulness in our analysis of these limitations.
Small Sample Asymptotics
Authors: Fligner Michael A.
Published: March 2008

An approach for modifying the results of asymptotic theory to improve the performance of statistical procedures in small to moderate sample sizes is described in the context of hypothesis testing. The method is illustrated by a series of examples.
Educational Applications of Hierarchical Linear Models: A Review
Authors: Raudenbush Stephen W.
Published: March 2008

The search for appropriate statistical methods for hierarchical, multilevel data has been a prominent theme in educational statistics over the past 15 years. As a result of this search, an important class of models, termed hierarchical linear models by this review, has emerged. In the paradigmatic application of such models, observations within each group (e.g., classroom or school) vary as a function of group-level or “microparameters.” However, these microparameters vary randomly across the population of groups as a function of “macroparameters.” Research interest has focused on estimation of both micro- and macroparameters. This paper reviews estimation theory and application of such models. Also, the logic of these methods is extended beyond the paradigmatic case to include research domains as diverse as panel studies, meta-analysis, and classical test theory. Microparameters to be estimated may be as diverse as means, proportions, variances, linear regression coefficients, and logit linear regression coefficients. Estimation theory is reviewed from Bayes and empirical Bayes viewpoints and the examples considered involve data sets with two levels of hierarchy.
Msultiplicative Models and MTMM Matrices
Authors: Cudeck Robert
Published: March 2008

The rationale for multiplicative models in the context of MTMM covariance matrices, as developed by Swain and Browne, is described and illustrated with several sets of empirical data. Comparisons with restricted factor analysis models are made, and certain criticisms of this approach are noted. An extension of the multiplicative model to three-facet data that includes occasions, methods, and traits is also presented.
Estimating Change in Dispersion
Authors: Raudenbush Stephen W.
Published: March 2008

Statistical methods are presented for studying changes in the dispersion of a dependent measure over time, for estimating the effects of treatments on the rate of dispersion change, and for combining such estimated effects from a series of related studies. Available methods for comparing both uncorrected and correlated dispersion estimates are first reviewed, including exact likelihood methods, approximate likelihood methods based on log transformed variances, and robust methods. Use of the log transformation provides reasonably efficient estimation, extends simply to the case of correlated dispersion estimates, and yields scale invariant estimates of dispersion change. However, because standard error estimates are not robust to violations of the normality assumption, a non-normality parameter is introduced which facilitates sensitivity analysis. Relative efficiency is evaluated for estimates of dispersion change based on posttest-only comparisons, “pre-post” change comparisons, and comparisons of residual variances after covariance adjustments. These ideas are illustrated by a reanalysis of data from two experiments assessing the effect of teacher expectancy on pupil IQ.
Scale Shrinkage and the Estimation of Latent Distribution Parameters
Authors: Camilli Gregory
Published: March 2008

In this paper the phenomenon of scale shrinkage is examined. Specifically, emphasis is placed on the pattern of decreasing variances in IRT scale scores from fall to spring within a grade. It is concluded that certain situations exist in which scale shrinkage is predictable with unidimensional tests. It depends, to a large degree, on the match between item difficulties and the level of examinee ability. As the mismatch increases, so do distortions of scale because of systematic estimation errors (bias), measurement errors, and unobtainable ability estimates. These problems exist for all observed or estimated scores; however, it is shown in this paper that questions concerning the population distributions of true ability can potentially be addressed with empirical Bayes techniques.
Models for Pretest-Posttest Data: Repeated Measures ANOVA Revisited
Authors: Jennings Earl
Published: March 2008

Eight models that can be used to analyze data from a pretest-posttest design are discussed. Artificial data are used to demonstrate that comparisons among some of the models generate results identical to those obtained by using repeated measures ANOVA. Such results are shown to be potentially misleading. Repeated measures ANOVA is not recommended with pretest-posttest data.
Standardized Discriminant Coefficients: Which Variance Estimate Is Appropriate?
Authors: Mueller Ralph O , Cozad James B.
Published: March 2008

The authors discuss standardization procedures in discriminant analysis and compare three leading statistical software packages in terms of their calculations of unstandardized and standardized discriminant coefficients. They describe estimation procedures in each and point out inconsistencies that could lead to the misinterpretation of data. They present arguments that favor using within-group—rather than total—variance estimates to calculate standardized discriminant coefficients.
Estimating the Standardized Mean Difference in Intervention Studies
Authors: Fowler Robert L.
Published: March 2008

Methods for approximating confidence intervals for the population standardized mean difference, δ , were evaluated analytically in small samples. A procedure based on the large sample approximation of the distribution of the sample standardized mean difference d was quite accurate over most of the range .25 ≤ δ ≤1.5 and 6 ≤ N ≤ 40 examined. Use of Hedges’s adjustment to d yielded confidence intervals that were slightly conservative, whereas the unadjusted d produced intervals that were somewhat liberal. An empirically determined simple linear adjustment to d demonstrated the most consistent precision when used to construct confidence intervals for δ , having a maximum error of less than 2% of the nominal level of confidence over the effect size range of most interest to behavioral scientists.
Determining the Most Significant Parametric Function for a Given Linear Multivariate Hypothesis
Authors: Lutz J. Gary , Cundari Leigh A.
Published: March 2008

After a hypothesis about some linear multivariate statistical model has been tested and rejected (e.g., in a MANOVA), many researchers employ simultaneous test procedures to locate the source(s) of the rejection. If the global test was conducted using Roy’s largest root criterion, then this procedure guarantees at least one linear combination of the model parameters relative to some linear combination of the dependent variables that is significantly different from its hypothesized value. This most significant parametric function is not always easy to find, however, because it may not manifest itself in simple or “obvious” functions. A general solution to this problem is presented along with a practical example of its application.
Eelworms, Bullet Holes, and Geraldine Ferraro: Some Problems With Statistical Adjustment and Some Solutions
Authors: Wainer Howard
Published: March 2008

There is no safety in numbers. When data are gathered from a sample in which the selection criteria are unknown, many problems can befall the unwary investigator. In this paper we explore some of these problems and discuss some solutions. Our principle example is drawn from data from students who choose to take the College Board’s Scholastic Aptitude Test (SAT). We explore methods of covariance adjustment as well as more explicitly model-based adjustment methods. Among the latter we discuss Heckman’s Selection Model, Rubin’s Mixture Model, and Tukey’s Simplified Selection Model.
Adjusting for Unequal Variances When Comparing Means in One-Way and Two-Way Fixed Effects ANOVA Models
Authors: Wilcox Rand R.
Published: March 2008

Numerous papers have shown that the conventional F test is not robust to unequal variances in the one-way fixed effects ANOVA model, and several methods have been proposed for dealing with this problem. Here I describe and compare two methods for handling unequal variances in the two-way fixed effects ANOVA model. One is based on an improved Wilcox (1988) method for the one-way model, which forms the basis for considering this method in the two-way ANOVA model. The other is an extension of James’s (1951) second order method.
Algorithms for Computerized Test Construction Using Classical Item Parameters
Authors: Adema Jos J. , van der Linden Wim J.
Published: March 2008

Recently, linear programming models for test construction were developed. These models were based on the information function from item response theory. In this paper another approach is followed. Two 0-1 linear programming models for the construction of tests using classical item and test parameters are given. These models are useful, for instance, when classical test theory has to serve as an interface between an IRT-based item banking system and a test constructor not familiar with the underlying theory.
Considerations and Techniques for the Analysis of NAEP Data
Authors: Johnson Eugene G.
Published: March 2008

The special characteristics of the National Assessment data that affect the validity of conventional techniques of statistical analysis are considered. In contrast to the assumptions underlying standard methods of statistical analysis, the NAEP samples are obtained by means of a stratified multistage probability sampling design in which clusters of students are selected and in which certain subpopulations are sampled at a higher rate. The resulting samples have different statistical characteristics from those of a simple random sample. Analytic techniques that take the structure of the data into account are discussed and their properties explored. A second feature of the National Assessment data arises from the fact that multiple matrix sampling techniques are used for the assignment of subsets of the pool of exercises to subsamples of students. National Assessment accounts for the effect of the missing data on its scaling models through the use of multiple imputation procedures, creating vectors of “plausible values” for the scale scores. The use of plausible values and their effect on analysis are illustrated. The analyses are illustrated with results from the NAEP assessment of mathematics.