bluuebunny/crossref_metadata_2025_split huggingface dataset:
QUALIFYING ESSAY READERS FOR AN ONLINE SCORING NETWORK (OSN)
Authors: Powers Don , Kubota Mel , Bentley Jill , Farnum Marisa , Swartz Rich , Willard Ann 
Published: August 2014

ABSTRACTCurrently, personnel who evaluate examinee‐constructed responses for ETS‐administered testing programs must possess certain academic credentials. This study was designed to determine the extent to which current prerequisites for essay readers might be relaxed without sacrificing quality with regard to the accuracy with which essay scores are awarded. The specific interest was in a particular class of potential readers, namely, those who are not currently involved in postsecondary teaching and who therefore do not meet current standards for ETS essay readers. Secondary objectives of the study were to (1) provide information that would facilitate the determination of a passing score on the certification test that all ETS essay readers will be required to pass in the future and (2) evaluate, in a preliminary fashion, the effects of training on the performance of essay readers.To accomplish these objectives, both experienced and inexperienced readers were recruited for the study. All study participants evaluated a set of essays before any training and another set after they had undergone standard training for scoring essays.The results showed that training did affect the accuracy with which readers (especially those who were previously inexperienced) scored essays. Moreover, after training, a significant proportion of inexperienced readers exhibited a level of accuracy that was commensurate with that shown by experienced readers. A comparison of the performances of experienced and inexperienced readers suggested that a defensible passing score of about 90% accuracy could be set on the reader certification test when the standard of accuracy is agreement within one point of previously established essay scores.
INTERCULTURAL TRANSITIONS IN HUMAN DEVELOPMENT AND EDUCATION
Authors: Laosa Luis M. 
Published: August 2014

ABSTRACTIntercultural transitions are alterations that occur in individuals and in their environments as a consequence of contact between cultures. This paper focuses on the life events that children (and the adults in their lives) may face in bridging cultures, the options they are given, the choices they make‐or that adults make for them‐the circumstances and experiences that accompany these choices, and their consequences. The paper identifies and discusses issues in need of research concerning ways to foster scholastic success and socioemotional well‐being in populations that contemporary policies and practices may be putting at risk for school failure and for other forms of marginality. A better understanding of these issues can contribute to the improvement of intergroup relations, and it can be used to enhance the educational development and life chances of individuals from all ethnocultural and linguistic backgrounds.
UNIVARIATE AND BIVARIATE LOGLINEAR MODELS FOR DISCRETE TEST SCORE DISTRIBUTIONS
Authors: Holland Paul W. , Thayer Dorothy T. 
Published: August 2014

ABSTRACTThe well‐developed theory of exponential families of distributions is applied to the problem of fitting the univariate histograms and discrete bivariate frequency distributions that often arise in the analysis of test scores. These models are powerful tools for many forms of parametric data smoothing and are particularly well‐suited to problems where there is little or no theory to guide a choice of probability models, e.g. smoothing a distribution to eliminate roughness and zero frequencies in order to equate scores from different tests. Attention is given to efficient computation of the maximum likelihood estimates of the parameters using Newton's Method and to computationally efficient methods for obtaining the asymptotic standard errors of the fitted frequencies and proportions. We discuss tools that can be used to diagnose the quality of the fitted frequencies for both the univariate and the bivariate cases. Five examples, using real data, are used to illustrate the methods of this paper.
SCIENCE AND ENGINEERING GRADUATES: CAREER ADVANCEMENT AND CAREER CHANGE
Authors: Grandy Jerilee 
Published: August 2014

ABSTRACTThis project studied science and engineering (S/E) graduates who, after having been out of college some years, planned to return to graduate school either to advance their careers or to change careers. Two samples were studied.First, from the 1990–91 GRE files, there were more than 10,000 examinees who had bachelor's degrees in mathematics, natural sciences, computer sciences, or engineering (S/E) and had been out of college at least 5 years. All were either applying to graduate school or had just begun graduate work. About 40% of engineers planned to change fields, and more than half of science graduates planned to switch. This great a loss of talent from the science/engineering talent pool motivated the second part of the study.The second part of the study consisted of a questionnaire survey of a stratified sample drawn from the 1991–92 GRE files of science and engineering graduates who had been out of college at least five years. Women and minorities were oversampled so that separate analyses could be done for members of these groups. The sample was surveyed by mail to learn what factors might explain why some graduates were planning to continue in S/E while others were switching to different fields. Responses were obtained from 2,201 (82% of the original sample).People changing fields, on average, were older and had been out of college longer. More important than any other variable was that they placed a greater value on making a contribution to society and working directly with other people than on the satisfaction of technical challenge. There was no evidence that people switched fields because they were unemployed.Field switching was more frequent for females than males, for African Americans more than other ethnic groups, and for graduates in biological sciences more than graduates in other fields. Correlates of field switching were studied, and differences by gender and ethnic group were noted. There was very little gender difference in ratings of satisfaction with the most recent job in S/E. Gender was associated with many other variables, such as the importance of making a contribution to society. In a path analysis model using LISREL, the entire association between gender and continuation in S/E was accounted for by other variables.
EFFECTS OF COACHING ON SAT® I: REASONING SCORES
Authors: Powers Donald E. , Rock Donald A. 
Published: August 2014

ABSTRACTA College Board‐sponsored survey of a nationally representative sample of 1995–96 SAT® takers yielded a data base for more than 4,000 examinees, about 500 of whom had attended formal coaching programs outside their schools. Several alternative analytical methods were used to estimate the effects of coaching on SAT I: Reasoning scores. The various analyses produced somewhat different estimates. All of the estimates, however, suggested that the effects of coaching are far less than is claimed by major commercial test preparation companies. The revised SAT I does not appear to be any more coachable than its predecessor.
COMPARABILITY OF PAPER‐AND‐PENCIL AND COMPUTER ADAPTIVE TEST SCORES ON THE GRE® GENERAL TEST
Authors: Schaeffer Gary A. , Bridgeman Brent , Golub‐Smith Marna L. , Lewis Charles , Potenza Maria T. , Steffen Manfred 
Published: August 2014

ABSTRACTThis report summarizes a study conducted to assess the comparability of paper‐and‐pencil (P&P) and computer adaptive test (CAT) scores on the GRE® General Test. Volunteer examinees from around the country were randomly assigned to take the test in either CAT mode or P&P mode. Results indicated that for each measure (verbal, quantitative, analytical), mean scores on the CAT were higher than mean scores on P&P. It was hypothesized that CAT examinee test‐taking behavior with regard to the CAT scoring method that was in place at the time of the study (allowing student to receive a score by answering only 80% or more of the questions) may have affected CAT scores. Investigation of this hypothesis indicated that CAT examinees who did not complete their CATs obtained higher mean scores than would be predicted. However, mean scores for examinees who completed their CATs were similar to mean scores for P&P examinees. A new psychometrically‐defensible CAT scoring method, called proportional scoring, was developed in which it is to the examinees' advantage, in terms of maximizing their scores, to carefully consider and answer as many items as they can. The proportional scoring method has been implemented in the operational GRE CAT program, and it is believed that this scoring method will result in CAT scores that are comparable to P&P scores. GRE CAT scores will continue to be closely monitored.
MODELS OF THE RELATIONSHIP BETWEEN EDUCATION SPENDING AND THE SOCIAL DISTRIBUTION OF ACHIEVEMENT
Authors: Wenglinsky Harold 
Published: August 2014

ABSTRACTThis study relates various types of educational spending to mean mathematics achievement and its social distribution among students. A nationally representative database of 7,217 twelfth graders was developed by linking the 1992 National Assessment of Educational Progress in mathematics to the U.S. Common Core of Data and the School District Data Book. Through applying Hierarchical Linear Modeling to the database, the study found that spending on instruction and capital expenditures, while not related to mean achievement levels, were related to differences in achievement between SES groups; lower spending levels are associated with greater achievement gaps within schools.
THREE RESPONSE TYPES FOR BROADENING THE CONCEPTION OF MATHEMATICAL PROBLEM SOLVING IN COMPUTERIZED‐ADAPTIVE TESTS
Authors: Bennett Randy Elliot , Morley Mary , Quardt Dennis 
Published: August 2014

ABSTRACTWe describe three open‐ended response types that should broaden the conception of mathematical problem solving embodied in computerized‐adaptive admissions tests. The Mathematical Expressions response type presents single‐best answer problems that call for an algebraic formalism, the correct rendition of which may take an infinite number of surface forms. Generating Examples presents loosely structured problems which may have many good answers, taking the form of a value, letter pattern, expression, equation, or list. Graphical Modeling problems ask the examinee to represent a given situation by plotting points on a grid, where there may be a single‐best answer or multiple correct responses. Finally, these three basic types can be combined to form extended constructed‐response problems. For each of the three basic types, this paper gives example items, describes the examinee interface and approach to automatic scoring, and reports available findings.
AN INVESTIGATION OF THE SIMULTANEOUS MODERATION OF AVERAGE GENDER AND AFRICAN‐AMERICAN SCORE DIFFERENCES ON A TEST OF MATHEMATICAL REASONING
Authors: Stocking Martha L. , Jirele Thomas , Lewis Charles , Swanson Len 
Published: August 2014

ABSTRACTA pool of items from operational tests of mathematical reasoning was constructed to investigate the feasibility of using automated test assembly methods to simultaneously moderate possibly irrelevant differences between the performance of women and men and African‐American and White test takers. None of the artificial tests investigated exhibited substantial impact moderation, although the estimated mean scaled score differences for the relevant population indicated a modest move in the intended direction: the difference between scaled score means was reduced by about 20% for women and men and about 9% for African‐American and White test takers. Although many issues in the implementation of this methodology remain to be solved, the consideration of impact in automated test assembly along with the maintenance of the detailed test plan appears to be a potential method of moderating possibly irrelevant mean test score differences.
THE POWER OF THE K‐INDEX (OR PMIR) TO DETECT COPYING
Authors: Lewis Charles , Thayer Dorothy T. 
Published: August 2014

ABSTRACTAt Educational Testing Service, the K‐Index is used to assess unusual agreement between the incorrect responses of two test takers on a multiple‐choice test. Since the Fall of 1996, the value of the K‐Index used for this assessment has been adjusted, based on the Bonferroni inequality. The resulting value is referred to as the Probability of Matching Incorrect Responses (PMIR). If a test taker copies from someone (a Source) with no incorrect responses, the PMIR is useless (i.e., has zero power) for detecting unusual agreement. Similarly, if a Source has relatively few incorrect responses, the power of the PMIR will be low. We propose a framework within which to study the power of the PMIR. The power function derived within this framework exhibits quite complex behavior; however, we conclude that the power of the PMIR to detect substantial amounts of copying is quite low, even when the Source has a relatively large number of incorrect responses.
EXPLORATION OF AN AUTOMATED EDITING TASK AS A GRE® WRITING MEASURE1
Authors: Breland Hunter M. 
Published: August 2014

ABSTRACTTwo editing tasks were developed and programmed for the computer to explore the possibility that such tasks might be useful as measures of writing skill. An informal data collection was then conducted with 52 prospective graduate students. These students completed the editing tasks with no time limit, as well as a writing experience questionnaire. Scores obtained on the two editing tasks were correlated with variables developed from the questionnaire. The total score for the two editing tasks correlated .52 with student self‐assessments of their writing ability, .46 with grade‐point average (GPA) based on courses requiring at least some writing, and .30 with writing accomplishments. The correlation with GPA, however, was only .14. The reliability of the total editing score was estimated at .84.
VALIDITY OF THE SECONDARY LEVEL ENGLISH PROFICIENCY TEST AT TEMPLE UNIVERSITY‐JAPAN
Authors: Wilson Kenneth M. , Graves Kimberly 
Published: August 2014

ABSTRACTThe study reported herein assessed levels and patterns of concurrent correlations for Listening Comprehension (LC) and Reading Comprehension (RC) scores provided by the Secondary Level English Proficiency (SLEP) test with direct measures of ESL speaking proficiency (interview rating) and writing proficiency (essay rating), respectively, and (b) the internal consistency and dimensionality of the SLEP, by analyzing intercorrelations of scores on SLEP item‐type “parcels” (subsets of several items of each type included in the SLEP). Data for some 1,600 native‐Japanese speakers (recent secondary‐school graduates) applying for admission to Temple University‐Japan (TU‐J) were analyzed. The findings tended to confirm and extend previous research findings suggesting that the SLEP – which was originally developed for use with secondary‐school students–also permits valid inferences regarding ESL listening‐ and reading‐comprehension skills in postsecondary‐level samples.
THE ROLE OF THE UNIT OF ANALYSIS IN DIMENSIONALITY ASSESSMENT
Authors: Dorans Neil J. , Lawrence Ida M. 
Published: August 2014

ABSTRACTA variety of techniques exist for addressing the fundamental question: What does a test measure? The answer to that question often depends on how one asks the question of the data. At one extreme, we see correlations of a test with other editions of that test (e.g., parallel form reliability estimates). At the other extreme, we see exploratory factor analyses of item‐level data. In this paper, we argue that item‐level and test‐score dimensionality are not necessarily identical, define mathematical relationships among item‐level factors, test‐score factors and parcel‐level factors, and advocate that the choice of dimensionality technique should be based on the purpose of the dimensionality analysis. Finally, we illustrate how different approaches yield different answers with SAT‐Verbal data.
ITEMS BY DESIGN: THE IMPACT OF SYSTEMATIC FEATURE VARIATION ON ITEM STATISTICAL CHARACTERISTICS
Authors: Enright Mary K. , Morley Mary , Sheehan Kathleen M. 
Published: August 2014

ABSTRACTThis study investigated the impact of systematic item feature variation on item statistical characteristics and the degree to which such information could be used as collateral information to supplement examinee performance data and reduce pretest sample size. Two families of word problem variants for the quantitative section of the Graduate Record Examination (GRE®) General Test were generated by systematically manipulating item features. For rate problems, the item design features affected item difficulty (Adj. R2 = .90), item discrimination (Adj. R2 = .50), and guessing (Adj. R2 = .41). For probability problems the item design features affected difficulty (Adj. R2 = .61), but not discrimination or guessing. The results demonstrate the enormous potential of systematically creating item variants. However, questions of how best to manage variants in item pools and to implement statistical procedures that use collateral information must still be resolved.
GRAPHICAL MODELING: A NEW RESPONSE TYPE FOR MEASURING THE QUALITATIVE COMPONENT OF MATHEMATICAL REASONING
Authors: Bennett Randy Elliot , Morley Mary , Quardt Dennis , Rock Donald A. 
Published: August 2014

ABSTRACTThis study investigated the psychometric functioning of Graphical Modeling (GM) – a new computer‐delivered response type for assessing mathematical reasoning that asks candidates to respond to a problem situation by creating a graphical representation. GM problems can be like the single‐best‐answer items currently found on the General Test or they can be more loosely defined, allowing for multiple correct responses. Two GM tests differing from one another in the manipulation of specific item features were randomly spiraled among study participants. Analyses were performed relating to internal consistency reliability, relations with external criteria, features that contribute to item difficulty, adverse gender impact, and examinee perceptions. Results showed that GM scores were very reliable and moderately related to the General Test's quantitative section, suggesting that the introduction of GM items on the General Test might help broaden the GRE quantitative construct. In exploratory analyses of difficulty, one of three manipulated item features, problem structure, had a significant effect. Our impact analyses detected no significant gender differences independent of those associated with the GRE quantitative section. Finally, while more participants preferred regular multiple‐choice graphical reasoning questions to GM items, more also thought GM was the fairer indicator of their ability to undertake graduate study.
ASSESSING TIME TRENDS IN SEX DIFFERENCES IN SWIMMING AND RUNNING
Authors: Wainer Howard , Njue Catherine , Palmer Samuel J. 
Published: August 2014

ABSTRACTMen and women perform at different levels in athletics. In swimming and running races men are almost always faster. Over the past century the differences observed between men and women have been shrinking. This shrinkage seems to have been more dramatic at longer distances. In this article we examine the time trends in these differences by event through examination of world records and Olympic performances. We show that in running, using one sensible measure, women seem to be nearer men's performances in sprints than in distance races; in swimming the reverse is true. This measure provides better predictive inferences than the typical alternatives.
A SIMULATION STUDY OF THE EFFECT OF RATER DESIGNS ON ABILITY ESTIMATION
Authors: Hombo Catherine M. , Donoghue John R. , Thayer Dorothy T. 
Published: August 2014

ABSTRACTAs more assessment programs move towards the use of constructed response and performance assessment items, the use of human raters to score these items necessarily will increase. Different designs can be created to assign raters to score examinee responses, and in this study some of these designs are evaluated in terms of their impact on the accuracy of examinee ability estimation. As expected, the optimum design where every rater judges every performance by every examinee results in ability estimates with minimum bias and small standard error. As this design is rarely, if ever, practical in real assessment situations, the results for nested and spiral rater designs are of more interest to practitioners. The nested rater designs result in biased ability estimates for the examinees judged by the most extreme raters, but the spiral rater designs examined prove surprisingly robust to rater tendencies.
SUBSTITUTING SAT® II: SUBJECT TESTS FOR SAT I: REASONING TEST: IMPACT ON ADMITTED CLASS COMPOSITION AND QUALITY
Authors: Bridgeman Brent , Burton Nancy , Cline Frederick 
Published: August 2014

ABSTRACTUsing data from a sample of 10 colleges at which most students had taken both SAT® I: Reasoning Test and SAT II: Subject Tests, we simulated the effects of making selection decisions using SAT II scores in place of SAT I scores. Specifically, we treated the students in each college as forming the applicant pool for a more select college, and then selected the top two‐thirds (and top one‐third) of the students using high school grade point average (HSGPA) combined with either SAT I scores or the average of SAT II scores. Success rates, in terms of freshman grade point averages, were virtually identical for students selected by the different models. The percent of African American, Asian American, and White students selected varied only slightly across models. Appreciably more Mexican American and Other Latino students were selected with the model that used SAT II scores in place of SAT I scores because these students submitted Subject Test scores for the Spanish Test on which they had high scores.
PSYCHOMETRIC EVALUATION OF THE NEW GRE® WRITING ASSESSMENT
Authors: Schaeffer Gary A. , Briel Jacqueline B. , Fowles Mary E. , Boodoo Gwyneth , Braun Henry , Lewis Charles , Thayer Dorothy 
Published: August 2014

ABSTRACTThis study analyzed examinee responses to two essay prompts being considered for the Graduate Record Examinations (GRE®) Writing Assessment: “Present your Perspective on an Issue” and “Analyze an Argument.” Forty prompts (20 issue and 20 argument) were administered to over 2,300 students at 26 participating U.S. colleges and universities. Each student wrote two essays in response to either two issue prompts, two argument prompts, or one of each. Results show that the issue and argument writing tasks appear to assess relatively similar constructs, supporting the decision to include both types of prompts in the operational GRE Writing Assessment and to report a single, combined score. The results also support the random administration of prompts without equating adjustments, because within each task type, most of the prompts were comparable in difficulty and no important subgroup interactions with prompt classifications were detected. Finally, from a fairness perspective, the results show there were advantages to administering the issue prompt first and the argument prompt second. The GRE Program used the psychometric information provided by this study to make final design, delivery, and scoring decisions before introducing the operational assessment in the fall of 1999.