bluuebunny/crossref_metadata_2025_split huggingface dataset:
LONG‐TERM PREDICTIVE AND CONSTRUCT VALIDITY OF TWO TRADITIONAL PREDICTORS OF LAW SCHOOL PERFORMANCE
Authors: Powers Donald E. 
Published: August 2014

The relative contribution of two traditional, widely used preadmission measures, undergraduate grade point average (UGPA) and the Law School Admission Test (LSAT), to predicting academic performance in each of the three years of law school was investigated in a heterogeneous sample of 23 law schools. The contribution of each measure was uneven across years, with the LSAT making its greatest contribution in early years and UGPA in later years. Differences between law school years with respect to the nature of instructional techniques and curricular emphases were considered in order to gain a better understanding of the construct validity of each measure as a predictor of performance in law school. The results were consistent with the test sponsor's interpretation of the LSAT as a measure of the ability to confront new situations and new problems such as those encountered in the first year of law school. They were also consistent with an interpretation of UGPA as a measure not only of academic competence but also of other qualities (such as persistence and motivation) that are needed over the long haul for the study of law.The author wishes to acknowledge the Law School Admission Council for its support of the project on which this paper is based. Thanks go to Franklin R. Evans for helpful suggestions in conducting the research; to Thomas F. Donlon, Kenneth M. Wilson, and three anonymous reviewers for thoughtful reviews of this paper; to Miriam Godshalk for helpful editorial suggestions; and to the admissions staff of the participating schools for providing the data. Special thanks go to Frieda Hardy and Marie Ormond for managing the data collection activities; to Christopher Constantini, Katherine Kornhauser, and Norma Norris for processing and analyzing the data; to Henry Braun for statistical consultation; and to Lorraine Simon for her careful attention to numerous administrative tasks and to the final preparation of this manuscript.
MULTIVARIATE VARIANCE COMPONENT ANALYSIS: AN APPLICATION IN TEST DEVELOPMENT
Authors: Longford Nicholas T. 
Published: August 2014

ABSTRACTWe present a multilevel variance component analysis of data from the pilot year of pretesting of an educational test. The multilevel nature of the data is induced by the clustering of students within colleges, and by having multivariate observations (scores) on students. The presence of a multidimensional trait underlying the scores is formulated as a hypothesis about the full rank of a variance matrix.
THE VALIDITY OF THE GMAT FOR THE PREDICTION OF SUCCESS IN DOCTORAL STUDY IN BUSINESS AND MANAGEMENT
Authors: Zwick Rebecca 
Published: August 2014

ABSTRACTThe focus of this validity study was the examination of the degree to which GMAT scores and undergraduate grade‐point average (UGPA) could predict first‐year average (FYA) and final grade‐point average in doctoral programs in business. It was found that the three preadmissions measures were associated with graduate school grades, but to a lesser degree than in M.B.A programs. In general, the prediction achieved using UGPA alone as a predictor tended to be more accurate than that obtained using GMAT verbal (GMATV) and GMAT quantitative (GMATQ) scores together. Using all three predictors was more effective than using UGPA alone. The most likely explanation for the lower levels of prediction is that doctoral programs tend to be more selective than M.B.A. programs. Within‐school means on GMATV, GMATQ, UGPA, and FYA were higher than those found in M.B.A. validity studies; within‐school standard deviations on FYA tended to be smaller. Among these very select, academically competent students, highly accurate prediction of success in doctoral programs may not be possible.
CURRENT STATE SCIENCE ASSESSMENTS: IS NOTHING BETTER THAN SOMETHING?
Authors: Gong Brian , Lahart Colleen , Courtney Rosalea 
Published: August 2014

ABSTRACTThis study reports a survey of current state science assessments. The study examined the extent to which the assessments supported standards of student performance consistent with three goals of science education reform: 1) focus on or promote development of depth of knowledge rather than breadth; 2) focus on central concepts and skills rather than on lower‐level facts; 3) make connections to technical, personal, and social applications.Information about each state's science assessment practices was collected, including whether a state science assessment was mandated, the publisher of the assessment, and use of test scores. A sample of state science test instruments were subjected to content and cognitive analysis.The analysis indicated that for a large majority of states, science assessments were not a driving force in directing the curriculum or instructional practices towards the standards of science education reform. Almost 50% of the states did not have a state science assessment. The state science assessments currently used were not consistent with the science education reform goals examined in this study. The implications of these findings were discussed for assessment and reform of science education at the state level.
AN ALTERNATIVE THREE‐PARAMETER LOGISTIC ITEM RESPONSE MODEL
Authors: Pashley Peter J. 
Published: August 2014

ABSTRACTBirnbaum's three‐parameter logistic function has become a common basis for item response theory modeling, especially within situations where significant guessing behavior is evident. This model is formed through a linear transformation of the two‐parameter logistic function in order to facilitate a lower asymptote. This paper discusses an alternative three‐parameter logistic model in which the asymptote parameter is a linear component within the logit of the function. This alternative is derived from a more general four‐parameter model based on a transformed hyperbola.
DIFFERENCES AMONG RACIAL AND ETHNIC GROUPS IN MEAN SCORES ON THE GRE AND SAT: LONGITUDINAL COMPARISONS
Authors: Pennock‐Román María 
Published: August 2014

ABSTRACTIf one compares racial/ethnic group differences on verbal and quantitative tests from published data on the Scholastic Aptitude Test (SAT) and Graduate Record Examinations (GRE) General Test, some group differences appear to be relatively larger (in standard deviation units) on the GRE General Test than on the SAT. The objective of this study was to undertake a retrospective, longitudinal analysis of a group of individuals who had taken both tests to see if group differences remain the same or change over time.After merging records on the GRE General Test with SAT records for the same individuals, the realized sample had 69,494 cases.In this sample, the pattern of results from longitudinal evidence was very different from that found in cross‐sectional comparisons of published GRE and SAT data for the various groups. The changes in the relative size of mean differences were in the direction of closing the gap between non‐Hispanic (NH) White and minority groups, especially for Asian American and Puerto Rican examinees.The regressions of GRE General Test scores on SAT scores (i.e., growth) for various racial/ethnic groups were dissimilar from that of the NH White group, but the differences in growth patterns did not widen the distance in means between NH White test takers and minority groups.There was also evidence that racial/ethnic groups may vary in the degree of self‐selection that occurs for entry into the graduate‐school‐bound pool of GRE test takers. This pattern may explain the relatively larger disparity in group means in the GRE population than in the SAT population that is evident in the cross‐sectional comparisons. However, this explanation needs to be verified with more complete data on pathways to graduate school that include non‐SAT test takers and students who apply to professional schools.
NOTE TAKING AND LISTENING COMPREHENSION ON THE TEST OF ENGLISH AS A FOREIGN LANGUAGE
Authors: Hale Gordon A. , Courtney Rosalea 
Published: August 2014

ABSTRACTThis study examined the effects of taking notes in the portion of the TOEFL listening comprehension section that contains short monologues, or “mini‐talks.” These effects were assessed in experimental testing sessions with students in intensive English language programs and with undergraduate and graduate international students. A multiple‐choice questionnaire surveyed the students' reactions to the opportunity to take notes and their previous note‐taking experiences.Allowing students to take notes had little effect on their performance, and urging students to take notes significantly impaired their performance. These effects were observed even for students who reported being in the habit of taking many notes or reported having had classroom instruction in note taking. Apparently, then, little benefit is gained by taking notes in the context of the present TOEFL mini‐talks, perhaps because they are designed to assess listening comprehension with minimal demand placed on memory. Responses to the questionnaire aid in understanding the results and provide useful general information about the students' note‐taking experiences and habits.
A STUDY OF THE EFFECTS OF CONTEXTUALIZATION AND FAMILIARIZATION ON RESPONSES TO THE TOEFL VOCABULARY TEST ITEMS
Authors: Henning Grant 
Published: August 2014

ABSTRACTIn an effort to evaluate criticisms of the current TOEFL® vocabulary item format and to ascertain the comparative effectiveness of alternative vocabulary test item formats, the present study investigated the functioning of eight different multiple‐choice formats that differed with regard to the length and inference‐generating quality of the stem, with regard to the nature of the task (matching versus supply), and with regard to the degree of passage embeddedness of item stems or response options. In all, 1,040 vocabulary test items (80 familiarization items and 960 experimental items) were developed and administered in counterbalanced clusters to a total participating sample of 190 adult English‐as‐a‐second‐language students (99 persons with a controlled prior 75‐minute familiarization activity and 91 with no familiarization activity) at two language institutes. These two levels of familiarization were examined to ascertain effects of familiarization on performance with the eight item types. Also, self‐reports of levels of prior exposure to particular item types were correlated with actual performance with those item types.Results indicated that, for the present sample of persons and items, the current TOEFL vocabulary item format performed comparatively well in terms of appropriateness of difficulty, mean internal consistency reliability, and criterion‐related validity. Such performance came in spite of earlier suggestions that current item stems may be too long, lack inference‐generating information about the meaning of the targeted vocabulary, and place undue reliance on matching rather than supply format.Among alternative item formats considered, only items embedded in reading passages appeared to outperform current TOEFL vocabulary format in terms of estimates of both reliability and criterion‐related validity. Use of an item format that differed from current TOEFL vocabulary format only in the uniform presence of inference‐generating information in the item stem and in reduction of mean length of stem also slightly outperformed current TOEFL vocabulary item format in terms of the particular criterion validity indicated by correlation with vocabulary total score; however, these correlational differences did not quite attain statistical significance. In general, multiple‐choice items using stems with underscored target vocabulary in context performed better than did either multiple‐choice single‐word‐or‐phrase matching tasks or multiple‐choice supply‐type items.Participation in the familiarization activity did not relate significantly or differentially to performance with any item type. However, self‐reports of familiarity with particular item types suggested that some types were more familiar than others, and the three most familiar item types, including current TOEFL vocabulary format, exhibited a significant positive correlation between self‐report of familiarity with the item type and successful performance with the item type.
A PROCEDURE FOR ESTIMATING THE CONDITIONAL STANDARD ERRORS OF MEASUREMENT FOR GRE GENERAL AND SUBJECT TESTS
Authors: Cowell William R. 
Published: August 2014

ABSTRACTA series of computer programs were written for computing the conditional standard errors of measurement (CSEM) for both rights‐scored and formula‐scored tests based on a method suggested in Lord (1984), commonly known as Lord's Method IV or the compound binomial method. These programs estimate the conditional standard errors of measurement for both raw and scaled scores, average results for two or more forms, and compute form‐to‐form difference statistics for pairs of forms.Conditional standard errors of measurement, averages, and differences have been computed for the verbal, quantitative, and analytical raw and converted scores for eight forms of the GRE General Test and for two forms each of 15 GRE Subject Tests.The Standards for Educational and Psychological Testing (Committee of AERA, APA, & NCME to Develop Standards, 1985) recommends that test publishers provide estimates of the standard error measurement at a number of widely spaced score levels. The CSEM data produced in this study have been made available to three programs which use GRE scores, along with other criteria, for awarding fellowships. These data also have been made available for use in GRE program publications and in correspondence.
NEGATIVE COEFFICIENTS IN THE GRE VALIDITY STUDY SERVICE
Authors: Longford Nicholas T. 
Published: August 2014

ABSTRACTOperational procedures for the Graduate Record Examinations Validity Study Service (GREVSS) are reviewed, with the emphasis on the problem of frequent occurrence of negative coefficients in the fitted within‐department regressions obtained by the empirical Bayes method of Braun and Jones (1985). Several alterations of the operational procedures are proposed that would reduce the frequency of negative coefficients, and, if desired, completely eliminate them. It is argued, however, that there are no a priori reasons for assuming thatallthe coefficients are nonnegative. Reports of the fitted within‐department regressions should be based on a single model, that would be found by model exploration. The estimation procedures could be improved by employing more flexible software for modelling between‐department variation.
COHORT DIFFERENCES ASSOCIATED WITH TRENDS IN SAT® SCORE AVERAGES
Authors: Morgan Rick 
Published: August 2014

ABSTRACTThe potential impact of cohort changes with regard to ethnic group, gender, class rank, and first language learned on yearly average Scholastic Aptitude Test (SAT) and Test of Standard Written English (TSWE) scores was studied. Regression and cross‐classification analyses were conducted on data from 1985, and 1987 to 1990. Both analyses adjusted test score means based on common cohorts of students. The analyses indicated that approximately half of the decline in SAT verbal (SAT‐V) scores was associated with cohort change. Average SAT mathematics (SAT‐M) scores could possibly have risen by three points, rather than remaining constant, if the 1987 and 1990 cohorts were the same.
USING A TRAINABLE PATTERN‐DIRECTED COMPUTER PROGRAM TO SCORE NATURAL LANGUAGE ITEM RESPONSES
Authors: Kaplan Randy M. 
Published: August 2014

ABSTRACTIn this study we investigated a computer scoring procedure for constructed‐response items whose responses consist of natural language phrases or sentences. The costs for scoring such items either manually or by a traditional natural language computer processing approach is typically expensive.The purpose of this study was to investigate the feasibility of an alternate method to the traditional natural language processing approach. The two main goals of the study were to create a prototype scoring program based on an appropriate methodology and to carry out a preliminary comparison of the results of the scoring methodology to the scores of human raters.Our approach to scoring is based on the assumption that a set of responses can be described as a small language. From this language we can construct a grammar that describes the language. The grammar can then be used as a tool for recognizing sentences in the language.With respect to the task of scoring, this means creating a grammar from a sample of responses that have been scored correct or incorrect. This grammar is then used by a program to classify responses that were not used to create the grammar. The program automatically classifies a response as correct (or incorrect).The computer program was used to score sets of responses from three items. Two sets of data consisted of short responses (3 to 5 words). The third set of data consisted of longer responses (12 to 15 words). The scoring program rating agreed perfectly with those of human raters for the first two data sets and achieved a maximum agreement of 90% for the third data set.We conclude from these preliminary results that a trainable pattern‐directed computer scoring program may be a viable, cost‐effective approach to scoring short natural language responses. Longer responses present a number of problems that require further investigation. We suggest a number of ways that the prototype program can be extended to compensate for these more complex responses.
TOWARD A TEST THEORY FOR ASSESSING STUDENT UNDERSTANDING
Authors: Mislevy Robert J. , Yamamoto Kentaro , Anacker Steven 
Published: August 2014

ABSTRACTThe view of learning that underlies standard test theory is inconsistent with the view rapidly emerging from cognitive and educational psychology. Learners become more competent not simply by learning more facts and skills, but by reconfiguring their knowledge; by “chunking” information to reduce memory loads; and by developing strategies and models that help them discern when and how facts and skills are important. Neither classical test theory nor item response theory (IRT) is designed to inform educational decisions conceived from this perspective. This paper sketches the outlines of a test theory built around models of student understanding, as inspired by the substance and the psychology of the domain of interest. The ideas are illustrated with a simple numerical example based on Siegler's balance beam tasks. Directions in which the approach must be developed to be broadly useful in educational practice are discussed.
CONSTRUCTED RESPONSE AND DIFFERENTIAL ITEM FUNCTIONING: A PRAGMATIC APPROACH1
Authors: Dorans Neil J. , Schmitt Alicia P. 
Published: August 2014

ABSTRACTDifferential item functioning (DIF) assessment attempts to identify items or item types for which subpopulations of examinees exhibit performance differentials that are not consistent with the performance differentials typically seen for those subpopulations on collections of items that purport to measure a common construct. DIF assessment requires a rule for scoring items and a matching variable on which different subpopulations can be viewed as comparable for purposes of assessing their performance on items. Typically, DIF is operationally defined as a difference in item performance between subpopulations, e.g., Blacks and Whites, that exists after members of the different subpopulations have been matched on some total score. Constructed‐response items move beyond traditional multiple‐choice items, for which DIF methodology is well‐defined, towards item types involving selection or identification, reordering or rearrangement, substitution or correction, completion, construction, and performance or presentation. This paper defines DIF, describes two standard procedures for measuring DIF and indicates how DIF might be assessed for certain constructed‐response item types. The description of DIF assessment presented in this paper is applicable to computer‐delivered constructed‐response items as well as paper and pencil delivered items.
SEX DIFFERENCES IN THE RELATIONSHIP OF ADVANCED PLACEMENT ESSAY AND MULTIPLE‐CHOICE SCORES TO GRADES IN COLLEGE COURSES
Authors: Bridgeman Brent , Lewis Charles 
Published: August 2014

ABSTRACTEssay and multiple‐choice scores from Advanced Placement (AP) examinations in American History, European History, English Language and Composition, and Biology were matched with freshman grades in a sample of 32 colleges. Multiple‐choice scores from the American History and Biology examinations were superior to essays for predicting overall GPA, but essay scores were essentially equivalent to multiple‐choice scores for predicting grades in history, English, and biology. In history courses, males and females received comparable grades and had nearly equal scores on the AP essays, but the multiple‐choice scores of males were about half of a standard deviation higher than the scores of females. To the extent that the AP history examinations are intended to emulate performance in college history courses, placing greater weight on the essay component of the AP history examinations would reduce sex differences without compromising content or predictive validity.
DIFFERENTIAL SPEEDEDNESS AND ITEM OMIT PATTERNS ON THE SAT
Authors: Schmitt Alicia P. , Dorans Neil J. , Crone Carolyn R. , Maneckshana Behroz T. 
Published: August 2014

ABSTRACTTwo editions of the Verbal and Mathematical portions of the Scholastic Aptitude Test (SAT) were used to study differential speededness and differential omission and the relationships among differential item functioning (DIF), differential omission, and item difficulty for Asian‐Americans, Blacks, Hispanics, and females. Consistent and replicable evidence of differential speededness was found for Blacks and Hispanics. Use of an unspeeded criterion for matching in place of the traditional total score, which contains speeded items, does not affect the DIF analyses of the speeded items. A strong artifactual negative relationship between DIF and differential omission was found. The relationship between differential omission and difficulty was consistently positive on the Verbal sections for all comparison groups except the Asian‐American group, for whom it was consistently negative. On the Mathematical sections, this relationship was only consistently found for the female/male comparison, for whom it was negative. Finally, the relationship between difficulty and DIF was negative but smaller than previously observed.
THE PREDICTION OF GRE READING COMPREHENSION ITEM DIFFICULTY FOR EXPOSITORY PROSE PASSAGES FOR EACH OF THREE ITEM TYPES: MAIN IDEAS, INFERENCES AND EXPLICIT STATEMENTS
Authors: Freedle Roy , Kostin Irene 
Published: August 2014

ABSTRACTThis study examines the predictability of GRE reading item difficulty (equated delta) for three major reading item types: main idea, inference and explicit statement items. Each item type is analyzed separately, using 110 GRE reading passages and their associated 244 reading items; selective analyses of 285 SAT reading items are also presented. Stepwise regression analyses indicates that the percentage of GRE delta variance accounted for varied from 20% to 52% depending upon the item type.Details of item predictability were explored by evaluating several hypotheses. Results indicated that (1) multiple‐choice reading items are sensitive to variables similar to those reported in the experimental literature on comprehension, (2) many of these variables provide independent predictive information in regression analyses, and (3) substantial agreement between GRE and SAT reading predictability was found.