bluuebunny/crossref_metadata_2025_split huggingface dataset:
ITEM PERFORMANCE ACROSS NATIVE LANGUAGE GROUPS ON THE TEST OF ENGLISH AS A FOREIGN LANGUAGE
Authors: Alderman Donald L. , Holland Paul W. 
Published: August 2014

ABSTRACTThe Test of English as a Foreign Language (TOEFL) was examined for instances in which the item performance of examinees with comparable scores differed according to their native languages. A chi‐square procedure, sensitive to deviations of less than ten percent from the expected frequencies of correct item responses across several language groups, revealed significant differences on seven‐eighths of the TOEFL items. Reviewers familiar with particular languages could attribute the relative advantage or disadvantage of those language groups on a specific item to linguistic similarities or dissimilarities with the English language. Reviewers could not, however, identify which items would exhibit differential performance across groups based upon inspection of a test form and answer key alone. These findings suggest that examinees' performance on given items in a test of proficiency in a second language will vary according to linguistic contrasts with their native language and that statistical procedures will be necessary for identifying items with exaggerated or unexplained differences across language groups.
COGNITIVE STYLES IN EDUCATIONAL PRACTICE*
Authors: Messick Samuel 
Published: August 2014

ABSTRACTThis paper examines characteristic features of cognitive styles and the distinctive ways in which they differ from one another and from intellective abilities. Educational implications of cognitive styles are discussed in terms of six main rubrics: Improving instructional methods, Enriching teacher behavior and conceptions, Enhancing student learning and thinking strategies, Expanding guidance and vocational decision making, Broadening educational goals and outcomes, and Tuning the stylistic demands of educational environments. Reasons why cognitive styles should have educational impact are addressed as well as reasons why such educational benefits are difficult to realize.
REDESCENDING M‐TYPE ESTIMATORS OF LATENT ABILITY
Authors: Jones Douglas H. 
Published: August 2014

ABSTRACTNew ability estimators have been proposed by Wainer and Wright (1980) and Mislevy and Bock (1981), that are resistant against guessing and careless behaviors exhibited by some examinees. This paper presents another class of ability estimators that are resistant to departures from the underlying assumptions concerning guessing and carelessness. In addition to computing the asymptotic relative efficiency of such estimators, this paper evaluates estimators by comparing their influence curves (Huber, 1981).
SMALL SAMPLE PROPERTIES OF ASYMPTOTIC FORMULAS FOR EXPECTED SAMPLE SIZE SAVINGS IN CURTAILED BINOMIAL TESTS
Authors: Herrmann Nira , Szatrowski Ted H. 
Published: August 2014

ABSTRACTAsymptotic formulas for expected sample size savings in curtailed binomial tests are derived using a central limit theorem of Feller (1943). Exact results for the expected sample size savings and the forms of the one‐ and two‐sided curtailed tests are reviewed. The small sample properties of the asymptotic approximation are discussed and comparisons with the exact values and the recent results of Eisenberg and Ghosh (1981) are presented. The asymptotic results derived here compare very favorably to the exact results.
DEVELOPING A COMMON METRIC IN ITEM RESPONSE THEORY
Authors: Stocking Martha L. , Lord Frederic M. 
Published: August 2014

ABSTRACTA common problem arises when independent estimates of item parameters from two separate data sets must be expressed in the same metric. This problem is frequently confronted in studies of horizontal and vertical equating and in studies of item bias. This paper discusses a number of methods for transforming one metric to another metric and presents a new method. Data are given comparing this new method with a current method and recommendations are made.
INTEGRATING COGNITIVE AND PSYCHOMETRIC MODELS TO MEASURE DOCUMENT LITERACY
Authors: Sheehan Kathleen , Mislevy Robert J. 
Published: August 2014

ABSTRACTThe Survey of Young Adult Literacy conducted in 1985 by the National Assessment of Educational Progress included sixty‐three items that elicited skills in acquiring and using information from written documents. These items were analyzed in two distinct ways: (1) with an item response theory (IRT) model, which characterized items' difficulties and respondents' proficiencies as revealed simply by tendencies toward correct response, and (2) a qualitative cognitive model, which characterized items in terms of the processing tasks they required. This paper demonstrates how a generalization of Fischer and Scheiblechner's Linear Logistic Test Model can be used to integrate information from the cognitive analysis into the IRT analysis.Subject Terms: Bayesian estimation; cognitive processing models; Item Response Theory; Linear Logistic Test Model; literacy assessment;National Assessment of Educational Progress
COMPUTERIZED MASTERY TESTING WITH NONEQUIVALENT TESTLETS*
Authors: Sheehan Kathleen , Lewis Charles 
Published: August 2014

ABSTRACTA practical procedure for determining the effect of testlet nonequivalence on the operating characteristics of a testlet‐based computerized mastery test (CMT) is introduced. The procedure involves estimating the CMT decision rule twice, once with testlets treated as equivalent and once with testlets treated as nonequivalent. In the equivalent testlet mode, the likelihood functions estimated for specific number right scores are assumed to be constant across testlets and a single set of cutscores is estimated for all testlets. In the nonequivalent testlet mode, the likelihood functions estimated for specific number‐right scores are allowed to vary from one testlet to another and a different set of cutscores is estimated for each permutation of testlet presentation order. Small differences between the estimated operating characteristics of the equivalent testlet decision rule and the nonequivalent testlet decision rule indicate that the assumption of equivalent testlets was warranted. This procedure is illustrated with data from the Architect Registration Examination, a professional certification examination administered by the National Council of Architectural Registration Boards (NCARB).
COMPARISON OF EXPECTED WITH ACTUAL FIELD OF GRADUATE STUDY: AN ANALYSIS OF GRE SURVEY DATA
Authors: Grandy Jerilee 
Published: August 2014

ABSTRACTThis study analyzed data from a survey conducted by Nettles in the fall of 1987. The sample consisted of 2,136 examinees who took the GRE during 1986/87. The purpose of this project was to determine how well the intended field‐of‐study item in the GRE background questionnaire can be relied upon as an indicator of what examinees will actually study in graduate school at the beginning of the next academic year.Major findings were as follows:
Fifty‐six percent of all examinees in the sample became enrolled in graduate or professional school in the fall of 1987. The percentages varied somewhat across intended fields of study. Of those planning to study education, for example, 71% became enrolled in graduate school; of those planning to study computer science, only 49% became enrolled.
Enrollment rates did not differ significantly between Black, Hispanic, and predominantly White examinees, nor did it differ between male and female examinees.
Subsequent enrollment status was correlated very slightly with college grades but not with test scores.
Of those examinees who became enrolled in graduate or professional school and who specified a definite intended field of study, 72% were enrolled in exactly the same field of study they had indicated on the background questionnaire. Another 10% were enrolled in a specialty field within the same department. In total, 82% could be said to have enrolled in the same general field of study.
Stepwise regressions suggested that examinees with low GRE scores, particularly quantitative scores, were somewhat more likely to change field than examinees with high scores. With other variables held constant, females were slightly more likely to change field than were males. The strengths of the predictions, however, were extremely small, with multiple correlations of only 0.2.
The study concluded that test scores, grades, and known demographic variables in the GRE files are not strongly enough associated with changes in field of study to be useful as predictors of enrollment or of change in field of study.
The study concluded that the intended‐field‐of‐study item in the background questionnaire is a useful and reasonably valid indicator of actual field of graduate study.
AN ADAPTIVE ALGEBRA TEST: A TESTLET‐BASED, HIERARCHICALLY‐STRUCTURED TEST WITH VALIDITY‐BASED SCORING§
Authors: Wainer Howard , Lewis Charles , Kaplan Bruce , Braswell James 
Published: August 2014

ABSTRACTEarlier (Wainer & Lewis, 1990) we reported the initial development of a testlet‐based algebra test. In this account we provide the details of this excursion into the use of hierarchical testlets and validity‐based scoring. A pretest of two 15 item hierarchical testlets was carried out in which examinees' performance on a four item subset of each testlet was used to predict performance on the entire testlet. Four models for constructing hierarchies were considered. These presentation hierarchies were compared with one another and with an optimally chosen set of four linearly administered items. The comparison was carried out using both the root mean square error and the conditional posterior variance as the criterion. It was found on cross validation that although an adaptive test is everywhere superior to a fixed format test, this superiority is crucially dependent upon the quality of the items. When items vary considerably in quality a fixed format test, which uses the best items, can do almost as well as an adaptive test of equal length.
USING BAYESIAN DECISION THEORY TO DESIGN A COMPUTERIZED MASTERY TEST*
Authors: Lewis Charles , Sheehan Kathleen 
Published: August 2014

ABSTRACTMastery testing is used in educational and certification contexts to decide, on the basis of test performance, whether or not an individual has attained a specified level of knowledge, or mastery, of a given subject. A theoretical framework for mastery testing, based on Item Response Theory and Bayesian decision theory, is described in this paper. In this framework, the idea of sequential testing is developed, with the goal of providing shorter tests for individuals who have clearly mastered (or clearly not mastered) a given subject, and providing longer tests for those individuals for whom the mastery decision is not as clear‐cut. In a simulated application of the approach to a professional certification examination, it is shown that average test lengths can be reduced by half without sacrificing classification accuracy.
PERFORMANCE MODELING THAT INTEGRATES LATENT TRAIT AND CLASS THEORY1
Authors: Gitomer ew H. , Yamamoto Kentaro 
Published: August 2014

ABSTRACTThe qualitative characterization of individual performance that is central to modern psychological theory is not adequately modeled by traditional psychometric theory that assumes, among other things, unidimensionality. In the present study, data are presented that are more adequately modeled by HYBRID, a model that incorporates both latent trait and latent class components. The latent classes were defined by a cognitive analysis of the understanding that individuals have for a circumscribed domain. In addition to providing a better statistical fit, the analysis also improves the amount of diagnostic information available for a given individual.
THE CONVERGENT VALIDITY OF EXPERT SYSTEM SCORES FOR COMPLEX CONSTRUCTED‐RESPONSE QUANTITATIVE ITEMS
Authors: Bennett Randy Elliot , Sebrechts Marc M. , Rock Donald A. 
Published: August 2014

ABSTRACTThis study investigated the convergent validity of expert‐system scores for four mathematical constructed‐response item formats. A five‐factor model was posed comprised of four constructed‐response format factors and a GRE General Test quantitative factor. Confirmatory factor analysis was used to test the fit of this model and to compare it with several alternatives. The five‐factor model fit well, although a solution comprised of two highly correlated dimensions—GRE‐quantitative and constructed‐response—represented the data almost as well. These results extend the meaning of the expert system's constructed‐response scores by relating them to a well‐established quantitative measure and by indicating that they signify the same underlying proficiency across item formats.
ON THE RELIABILITY OF TESTLET‐BASED TESTS
Authors: Sireci Stephen G. , Thissen David , Wainer Howard 
Published: August 2014

ABSTRACTIf a test is constructed of testlets one must take into account the within‐testlet structure in the calculation of test statistics. Failing to do so may yield serious biases in the estimation of such statistics as reliability. In this paper we demonstrate how to calculate the reliability of a testlet‐based test. We show that traditional reliabilities calculated on two reading comprehension tests constructed of four testlets are substantial overestimates.
PAIRWISE COMPARISON PROCEDURES FOR ONE‐WAY ANALYSIS OF VARIANCE DESIGNS
Authors: Zwick Rebecca 
Published: August 2014

ABSTRACTResearch in the behavioral and health sciences frequently involves the application of one‐factor analysis of variance models. The goal may be to compare several independent groups of subjects on a quantitative dependent variable or to compare measurements made on a single group of subjects on different occasions or under different conditions. In analyzing data of this kind, it is usually of interest to determine which pairs of population means are likely to differ. In this paper, the selection of pairwise multiple comparison procedures for one‐way analysis of variance designs is considered, following a discussion of Type I error and power issues as they apply to the testing of multiple hypotheses. Procedures are included which are appropriate when normality or variance homogeneity assumptions are violated. The focus is on procedures that are easy to understand and apply. Single‐step procedures are emphasized because of their simplicity and because they allow for the construction of confidence intervals.
PERFORMANCE AND PERSISTENCE: A VALIDITY STUDY OF THE SAT® FOR STUDENTS WITH DISABILITIES
Authors: Ragosta Marjorie , Braun Henry , Kaplan Bruce 
Published: August 2014

This study was designed to test the validity of the Scholastic Aptitude Test (SAT) in predicting overall performance and persistence in college of students with disabilities, especially those participating in special test administrations. An earlier validity study (Braun, Ragosta, and Kaplan 1986) used first‐year grade‐point averages in college to study the validity issue. The current study returned to the schools that had originally provided data and obtained information on overall grade‐point averages and graduation status.The study answered a number of questions about the usefulness of the SAT as a predictor of performance and persistence and concluded that the SAT does an adequate job. Overall grade‐point averages of disabled and nondisabled students were rather accurately predicted, although there was some slight overprediction for students with learning disabilities whether they took the test during regular or special test administrations. Persistence in college was also quite well predicted except for hearing‐impaired students from special test administrations.
THE CULTURAL CONTEXT OF CONSTRUCT VALIDITY AND THE ETHICS OF GENERALIZABILITY*
Authors: Laosa Luis M. 
Published: August 2014

ABSTRACTThis article considers the mutual relevance of four themes: cultural diversity, construct validity, population generalizability, and professional ethics. Questions regarding construct validity are considered in the context of both measurement and intervention. The implications of the existence and the absence of empirical evidence on which to base validity generalizations are considered in reference to decision making in the provision of services. Ethical dilemmas that arise from these considerations, which face the applied professional in a culturally diverse society, are discussed within an emerging framework for resolving them. Previous research is reviewed to illustrate validity generalization limits. Also included are an assessment of research needs and a discussion of future directions for research and evaluation.
FITTING NEW MEASUREMENT MODELS TO GRE GENERAL TEST CONSTRUCTED‐RESPONSE ITEM DATA
Authors: Bennett Randy Elliot , Sebrechts Marc M. , Yamamoto Kentaro 
Published: August 2014

ABSTRACTThis exploratory study applied two new cognitively sensitive measurement models to constructed‐response quantitative data. The models, intended to produce qualitative characterizations of examinee performance, were fitted to algebra word problem solutions produced by examinees taking the GRE General Test. Two types of response data were modeled—error diagnoses and partial‐credit scores—both produced by an expert system. Error diagnoses, analyzed using Yamamoto's (1989a) Hybrid model, detected a class of examinees who tended to miss important pieces of the problem solution but made relatively few errors of other types. Group members were of low quantitative proficiency overall, though considerable variability was evident. Comparisons with matched examinees whose response patterns were better captured by the unidimensional IRT model suggested subtle differences in error frequency rather than sharp qualitative distinctions. In contrast with the error data, partial‐credit scores modeled using Rock's (Rock & Pollack, 1987) HOST procedure did not fit well, in part owing to limitations of the task theory being tested. Implications for the development of refined task and error theories, improvements to expert‐system scoring procedures, and response modeling are discussed.
SMALL‐SAMPLE EQUATING WITH LOG‐LINEAR SMOOTHING
Authors: Livingston Samuel A. 
Published: August 2014

ABSTRACTThis study investigated the extent to which log‐linear smoothing could improve the accuracy of common‐item equating by the chained equipercentile method in small samples of examinees. Examinee response data from a 100‐item test were used to create two overlapping forms of 58 items each, with 24 items in common. The criterion equating was a direct equipercentile equating of the two forms in the full population of 93,283 examinees. Anchor equatings were performed in samples of 25, 50, 100, and 200 examinees, with fifty pairs of samples at each size level. Four equatings were performed with each pair of samples: one based on unsmoothed distributions and three based on varying degrees of smoothing. Smoothing reduced, by at least half, the sample size required for a given degree of accuracy. Smoothing that preserved only two moments of the marginal distributions resulted in equatings that failed to capture the curvilinearity in the population equating.
UNDERSTANDING GRAPHS AND TABLES
Authors: Wainer Howard 
Published: August 2014

ABSTRACTQuantitative phenomena can be displayed effectively in a variety of ways, but to do so requires an understanding of both the structure of the phenomena and the limitations of candidate display formats. This paper:• Recounts three historic instances of the vital role data displays played in important discoveries,• Provides three levels of information which form the basis of a theory of display to help us better measure both display quality and human graphicacy, and• Describes three steps to improve the quality of tabular presentation.
COMPARABILITY OF COMPUTER AND PAPER‐AND‐PENCIL SCORES FOR TWO CLEP® GENERAL EXAMINATIONS
Authors: Mazzeo John , Druesne Barry , Raffeld Paul C. , Checketts Keith T. , Muhlstein Alan 
Published: August 2014

ABSTRACTThis report describes two studies that investigated the comparability of scores from paper‐and‐pencil and computer‐administered versions of the College‐Level Examination Program (CLEP) General Examinations in Mathematics and English Composition. The first study used a prototype computer‐administered version of each examination. Based on the results of the first study and feedback from the study participants, several modifications were made to these prototype versions. A second study was then conducted using the modified computer versions. Both studies used a single‐group counterbalanced equating design. Data for the Mathematics Examination were collected at Southwest Texas State University, and data for the English Composition Examination were collected at Utah State University. The results of Study 1 suggest that, despite efforts to design computer versions of the CLEP Mathematics and English Composition General Examinations that were administratively similar to the paper‐and‐pencil examinations (i.e., allowed item review and answer changing and were comparably timed), mode‐of‐administration effects (i.e., changes in average scores as a function of the mode of test delivery) were found. The results of Study 2 suggest that the modifications made to the computer versions eliminated the mode‐of‐administration effects for the English Composition Examination but not for the Mathematics Examination. The results of both studies underscore the need to determine empirically (rather than to just assume) the equivalence of computer and paper versions of an examination.
THE ROLE OF THE MENTOR TEACHER: A TWO‐PHASE STUDY OF TEACHER MENTORING PROGRAMS
Authors: Wilder Gita Z. 
Published: August 2014

ABSTRACTIncreasing attention to mentoring in the teaching profession in recent years has been accompanied by a proliferation of programs, literature and expertise related to experienced teachers helping new teachers. To assess the extent to which there is a common set of functions performed by teacher mentors, a two‐phase study of mentoring was undertaken. In the first phase, a telephone survey of the 50 states and of more than 100 districts was conducted to gather data about the existence and nature of mentoring programs. The survey revealed that about 60 percent of the states have enacted legislation to create formal programs of mentoring, although many of the programs had not yet been implemented at the time of the survey. In addition, mentoring programs existed in districts without state programs. In the second phase of the study, visits were made to eight mentoring programs, selected to represent a range of sponsorship and program structures. Interviews were conducted with program, district and school administrators, and with mentors and their teacher‐proteges. The results suggest that, although mentoring programs differ with respect to the ways in which the work of mentors is structured, there are functions that are common to the role of mentor across programs. The common functions and the implications of these for selecting and training mentors are discussed.