bluuebunny/crossref_metadata_2025_split huggingface dataset:
THE GRADUATE MANAGEMENT ADMISSION TEST PREDICTION BIAS STUDY
Authors: Braun Henry T. , Jones Douglas H. , Rubin Donald B. , Thayer Dorothy 
Published: August 2014

ABSTRACTThis document presents the results of an investigation into the prediction of first‐year average in graduate management school based on undergraduate grade point average and GMAT scores. It is found that the use of a single prediction equation for both black students and white students tends to produce over predictions of the former's graduate school performance. Substantial effort is devoted to developing a prediction system which takes into account ethnic status. This approach is made practicable by employing the empirical Bayes estimation paradigm, and the new predictions represent a substantial improvement over the ones based on a single equation for both races.
SCORING AND RESPONSE STYLE ON WRITTEN SIMULATION PROBLEMS
Authors: Alwood James E. , Scheuneman Janice Dowd 
Published: August 2014

ABSTRACTWritten Simulation Problems are a form of examination which offers an alternative to multiple choice testing for the evaluation of problem solving ability in fields such as medicine or other health professions. Although these problems have been used for assessment and evaluation for over a decade, relatively little attention has been paid to the psychometric properties of the scores derived from this type of examination. Of particular concern in this study are those scores derived for problem sections in which the examinee controls the number of options to be selected. Eight different procedures for scoring these sections were examined with respect to the impact of different types of error on the score, the susceptibility of the scoring procedure to response tendencies toward selecting many rather than few options, and the effects on the score of different examinees responding to different sections of the problem. Reliability of the scores was also determined. Results showed that the scores were generally adequate measures if the problems were balanced with regard to positive and negative options. With increasing imbalance, however, several of the scores became unsatisfactory, including some of the more commonly recommended. Suggestions for improved design and analysis of the problems were also made.
UNBIASED PREDICTIONS IN SPARSE DATA PROBLEMS
Authors: Braun Henry I. , Jones Douglas H. 
Published: August 2014

ABSTRACTThis paper presents a brief synopsis of some recent work of the authors (Braun and Jones, 1981) in connection with the use of regression methods to predict performance in graduate schools of management on the basis of undergraduate grade point average (UGPA) and the Graduate Management Admission Test (GMAT) score. The central problem is how to provide stable unbiased predictions for identifiable groups of candidates whose members may be sparsely distributed across the institutions under study. It is shown how empirical Bayes methods provide a workable solution even when classical procedures can not be implemented.
MAXIMUM LIKELIHOOD ESTIMATION OF THE JOINT COVARIANCE MATRIX FOR SECTIONS OF TESTS GIVEN TO DISTINCT SAMPLES WITH APPLICATION TO TEST EQUATING*
Authors: Thayer Dorothy T. 
Published: August 2014

ABSTRACTConsider an old test X consisting of s sections and two new tests Y and Z similar to X consisting of p and q sections respectively. All subjects are given test X plus two variable sections from either test Y or Z. Different pairing of variable sections are given to each subsample of subjects. We present a method to estimate the covariance matrix of the combined test (X1, …, Xs, Y1, …, Yp, Z1, …, Zq) and describe an application of these estimation techniques to linear, observed‐score, test‐equating.
DIFFERENTIAL TRENDS IN LAW GRADES OF MINORITY AND NONMINORITY LAW STUDENTS
Authors: Powers Donald E. 
Published: August 2014

ABSTRACTThree‐year trends in the law averages awarded to nonminority students and to two principal minority groups—Black students and Chicano students—were studied in two sets of law schools that enrolled more than 900 Black students and more than 300 Chicano students. In 18 of 21 law schools, Black students showed greater improvement than White students when third‐year grades were compared with first‐year grades. In 10 of 21 schools the improvement of Black students was significantly greater, statistically, than that of White students. Chicano students, who generally showed less prior educational disadvantagement than Black students, exhibited differential improvement in six of nine schools, but a statistically significant improvement in only one school.Various reasons for the differential improvements in grades are considered, including statistical artifacts (ceiling and floor effects, changes in criterion reliability, and unequal units of measurement in the grading of minority and nonminority students) and substantive factors such as course selection patterns, differences in the academic demands of each law school year, and support systems available to disadvantaged students. The implications of the findings are discussed for counseling students, for academic dismissal and probation policies, and for the study of effective institutional support services.
HOW ARE WE DOING?: A REVIEW OF SOCIAL INDICATORS III
Authors: Wainer Howard 
Published: August 2014

ABSTRACTThe aim of the international social indicators movement is to gather and disseminate data on a variety of variables that will track the social trends of our civilization, much as economic indicators track economic trends.Social Indicators IIIis the third in a series of compendia of statistical information selected and organized to depict important aspects of our current social situation and the underlying trends and developments. This paper reviews this volume.
UNDERGRADUATE DEBT AND PARTICIPATION IN GRADUATE EDUCATION: THE RELATIONSHIP BETWEEN EDUCATIONAL DEBT AND GRADUATE SCHOOL ASPIRATIONS, APPLICATIONS, AND ATTENDANCE AMONG STUDENTS WITH A PATTERN OF FULL‐TIME, CONTINUOUS POSTSECONDARY EDUCATION
Authors: Ekstrom Ruth , Goertz Margaret , Pollack Judith , Rock Donald 
Published: August 2014

ABSTRACTThis study explored the relationship between undergraduate educational loan indebtedness and graduate school aspirations, application, enrollment, and persistence, using data from the National Longitudinal Study of 1972 and from the 1980 and 1982 cohorts of High School and Beyond. The analysis was limited to full‐time students who did not interrupt their postsecondary education. The results show that aspirations and application for graduate education were slightly higher for indebted than non‐indebted college seniors in 1976 and 1984 and essentially equal for these two groups in 1986. Slightly more college graduates with educational debt than without enrolled in graduate or professional school in 1976 and 1984. Regression analyses showed little unique contribution of debt level to graduate aspirations, application, enrollment and persistence after controlling for factors such as background and differences in undergraduate educational experience.
FOUNDATIONS OF A NEW TEST THEORY
Authors: Mislevy Robert J. 
Published: August 2014

ABSTRACTIt is only a slight exaggeration to describe the test theory that dominates educational measurement today as the application of twentieth century statistics to nineteenth century psychology. Sophisticated estimation procedures, new techniques for missing‐data problems, and theoretical advances into latent‐variable modeling have appeared—all applied with psychological models that explain problem‐solving ability in terms of a single, continuous variable. This caricature suffices for many practical prediction and selection problems because it expresses patterns in data that are pertinent to the decisions that must be made. It falls short for placement and instruction problems based on students' internal representations of systems, problem‐solving strategies, or reconfigurations of knowledge as they learn. Such applications demand different caricatures of ability—more realistic ones that can express patterns suggested by recent developments in cognitive and educational psychology. The application of modern statistical methods with modern psychological models constitutes the foundation of a new test theory.
VALIDATING THE GRE ANALYTICAL ABILITY MEASURE AGAINST FACULTY RATINGS OF ANALYTICAL REASONING SKILLS
Authors: Enright Mary K. , Powers Donald E. 
Published: August 2014

ABSTRACTThe aim of this study was to develop a criterion of graduate school success as an alternative to first‐year average. More specifically, faculty rating scales of students' analytical abilities were developed as a potential criterion against which to validate both the current Graduate Record Examinations (GRE) analytical measure and future modifications of it.The rating instrument was based on previous research (Powers & Enright, 1986, 1987), which identified a number of independent dimensions underlying faculty perceptions of the importance of a wide variety of reasoning skills. The instrument included six separate scales for faculty to rate individual students with respect to their skills in analyzing arguments, drawing inferences, defining problems, reasoning inductively, and generating alternatives, as well as their overall analytical style.The rating scales were completed by faculty members in a sample of 24 graduate departments representing six disciplines. Three important results have implications for the use of faculty ratings as a criterion of success. First, faculty raters were not able to distinguish among students on the six individual scales, which exhibited very high intercorrelations. This suggests that the rating instrument could be simplified for future use.Secondly, although the ratings and first‐year grades were highly correlated, indicating that both criteria reflect success in graduate school, evidence that ratings and first‐year grades measure somewhat different aspects of success in graduate school was also found. Each of the three GRE General Test measures–verbal, quantitative, and analytical–was more highly correlated, on the average, with ratings than with first‐year averages. Undergraduate grades, on the other hand, correlated better with first‐year grades than with the ratings.Finally, results were mixed with respect to the validity of faculty ratings of students' analytical abilities. When the three GRE measures were ranked with respect to their predictive effectiveness for each department, the analytical measure was significantly more often the best or second best predictor of faculty ratings than of first‐year average, while the verbal and quantitative measures tended to be the best predictors about equally often for ratings and grades. This suggests that the ratings may be more reflective of analytical ability than of verbal or quantitative ability. However, the verbal measure was, on average, more highly correlated with faculty ratings of students' analytical skills than was the analytical measure. This suggests that faculty ratings of students' analytical skills may have been influenced by students' verbal reasoning skills. This failure to find unequivocal evidence of discriminant validity of the ratings may reflect problems with the ratings, with the way in which faculty rated students, or with the discriminant validity of the analytical measure, A recommendation was made to continue research on the development of these scales.
AUTOMATIC SCORING OF PAPER‐AND‐PENCIL FIGURAL RESPONSES
Authors: Martinez Michael E. , Ferris John J. , Kraft William , Manning Winton H. 
Published: August 2014

ABSTRACTLarge‐scale testing is dominated by the multiple‐choice question format. Widespread use of the format is due, in part, to the ease with which multiple‐choice items can be scored automatically. This paper examines automatic scoring procedures for an alternative item type: figural response. Figural response items call for the completion or modification of figural material, including illustrations, diagrams, and graphs. Twenty‐five science items were written in cooperation with the National Assessment of Educational Progress (NAEP) and printed with an ink that was invisible to scanning equipment. The items were answered with pencils; response sheets were scanned and the resulting data were processed by computer‐based scoring algorithms. The paper describes the technology that lead to successful pilot scoring of seven items for ten subjects. Implications of this technology for the future of large‐scale testing are discussed.
THE EFFECTS ON OBSERVED‐ AND TRUE‐SCORE EQUATING PROCEDURES OF MATCHING ON A FALLIBLE CRITERION: A SIMULATION WITH TEST VARIATION1
Authors: Eignor Daniel R. , Stocking Martha L. , Cook Linda L. 
Published: August 2014

ABSTRACTTwo recent simulation studies were conducted to aid in the diagnosis and interpretation of equating differences found between random and matched (nonrandom) samples for four commonly used equating procedures: Tucker, Levine equally reliable, and Chained equipercentile observed‐score procedures and the 3PL IRT true‐score equating procedure. For these simulations, test forms were equated to themselves, a situation that does not pattern reality. In the current simulation, test variation was added as an additional variable for study. The results of the current simulation confirmed the results of the previous two simulations and support the prediction based on theoretical grounds that observed‐score equating methods, such as Tucker and Chained equipercentile, are more affected by sample variation than are a true‐score method (IRT) or an observed‐score method based on true‐score assumptions (Levine equally reliable). The results further suggest that matching equating samples on the basis of a fallible measure of ability, such as anchor test score, is not advisable for any equating method studied except possibly the Tucker method.
AN INVESTIGATION OF THE USE OF SIMPLIFIED IRT MODELS FOR SCALING AND EQUATING THE TOEFL TEST
Authors: Way Walter D. , Reese Clyde M. 
Published: August 2014

ABSTRACTThe purpose of this study was to explore the use of two alternative item response theory estimation models in the scaling and equating of TOEFL – a modified one‐parameter model (M1PL) and a modified two‐parameter model (M2PL) – and to compare item scaling and test equating results based on these two alternative models with results based on the three‐parameter model (3PL) that is currently being used to scale and equate the TOEFL. The study employed a design in which a typical TOEFL equating was simulated using artificial data. The simulated equatings were compared in terms of correlations between estimated and generating parameters, model‐data fit, and concordance of simulated score conversions with conversions based on the generating parameters.The results of the study clearly indicated that the 3PL model performed better than the M1PL and M2PL models on the basis of each of the evaluation criteria. There was also evidence that the M2PL model performed better than the M1PL model, particularly in terms of model‐data fit and in the weighted root mean square difference statistics used to evaluate the simulated score conversions. The results of the study also indicated that discrepancies between score conversions based on the M1PL and M2PL model and those based on the 3PL model tended to occur at the lower and upper ends of the score scales. Finally, the results of the study for the 3PL model indicated that while correlations between item parameter estimates and generating parameters tended to be affected by sample size, neither the quality of model‐data fit nor the quality of simulated equatings appeared to be sensitive to the different sample sizes used in the study.
ESSAYS AND MULTIPLE‐CHOICE TESTS AS PREDICTORS OF COLLEGE FRESHMAN GPA
Authors: Bridgeman Brent 
Published: August 2014

ABSTRACTThe incremental validity of a short holistically‐scored expository essay for predicting freshman grade point average was explored in two samples. In one of the samples the essay was administered to incoming freshman at state colleges as part of a basic skills assessment battery. In the second sample the essay was part of an achievement test that is one of the admissions tests used by highly selective colleges. In both samples, the essay added essentially nothing to what could be predicted from high school grade point average, Scholastic Aptitude Test scores, and a multiple‐choice test of writing‐related skills.
DIFFERENCES AMONG RACIAL AND ETHNIC GROUPS IN MEAN SCORES ON THE GRE AND SAT: CROSS‐SECTIONAL COMPARISONS
Authors: Pennock‐Román María 
Published: August 2014

ABSTRACTThe objective here is to highlight the limitations of making longitudinal inferences about racial/ethnic group differences in test scores using cross‐sectional, published data provided by the Scholastic Aptitude Test (SAT) and Graduate Record Examinations (GRE) programs. Published data from the two programs are not comparable for a variety of reasons. For example, they use different definitions of demographic group membership (e.g., citizenship status), and these differences have a large effect on the population statistics. Furthermore, one year of GRE data corresponds to many years of SAT data.The lack of comparability of the two sets of data is illustrated by calculating mean differences between the non‐Hispanic White group and minority groups on test scores (in standard deviation units) under two conditions. First, standardized group mean differences are derived from statistics published for one year of GRE data and for one year of SAT data. Second, differences are derived after making corresponding groups in the two populations more comparable according to residency, citizenship status, and year of baccalaureate degree. Estimates of the relative distance between groups change under the two conditions.Even after making the aforementioned corrections, the two data sets are not comparable because of possible unequal distributions of undergraduate major in the two populations and possible variability across racial/ethnic groups in the characteristics of students who choose to apply to graduate school. Thus, longitudinal changes in group differences cannot be studied using cross‐sectional comparisons of the two populations.
AN ANALYSIS OF GRADUATE SCHOOL CAREERS IN THREE UNIVERSITIES: DIFFERENCES IN ATTAINMENT PATTERNS ACROSS ACADEMIC PROGRAMS AND DEMOGRAPHIC GROUPS
Authors: Zwick Rebecca 
Published: August 2014

ABSTRACTThe graduate careers of nearly 5,000 Ph.D.‐seeking students from 11 departments in each of three major universities were investigated, with a special focus on minority students. Minorities and women were found to be underrepresented in graduate school and to have generally lower candidacy and graduation rates than their White and male counterparts. In two of the three schools, foreign students had higher candidacy and graduation rates than did White Americans. Also, in two of the three schools, the percentage of foreign students increased substantially in recent years.A more general finding was that the candidacy and graduation rates in the eight years following matriculation were higher in quantitatively oriented departments than in the humanities and social sciences. In general, undergraduate grades and Graduate Record Examinations scores had only a minimal association with the attainment of candidacy and graduation. Among these academically select students, nonacademic factors may play a crucial role in determining who ultimately attains the doctoral degree.
RESPONSE LATENCY MEASURES FOR BIOGRAPHICAL INVENTORIES
Authors: Stricker Lawrence J. , Alderton David L. 
Published: August 2014

ABSTRACTThis study assessed the usefulness of response latency data for biographical inventory items in enhancing the inventory's validity. The Armed Services Applicant Profile (ASAP) was computer administered to Navy recruits, and the regular score, latency‐weighted scores, and measures of deviant latencies were obtained. The latency‐weighted scores did not improve the ASAP's validity in predicting six‐month retention, when used instead of or in addition to the regular score, and the deviant latency measures did not function as suppressor or moderator variables to increase the ASAP's validity. But subgroups of items with differing latencies varied systematically in their internal‐consistency reliability (with increased reliability for subgroups with shorter latencies), and a small subgroup of items with moderate latencies was almost as valid as the regular score, suggesting that latency data may be useful in writing and selecting inventory items.
NEW VIEWS OF STUDENT LEARNING: IMPLICATIONS FOR EDUCATIONAL MEASUREMENT
Authors: Masters Geofferey N. , Mislevy Robert J. 
Published: August 2014

ABSTRACTRecent research in cognitive psychology has drawn attention to the important role that students' personal understandings and representations of subject matter play in the learning process. This paper briefly reviews some of this research, and contrasts the kind of learning that results in an individual's changed conception or view of a phenomenon with the more passive, additive kind of learning assessed by most traditional achievement tests. To be consistent with a view of learning as an active, constructive process, educational tests are required which focus on key concepts in an area of learning, and which take into account the variety of types and levels of understanding that students have of those concepts. In these tests, scoring responses right and wrong is likely to be less appropriate than using students' answers to infer their levels of understanding. This will require not only imaginative new types of test items, but statistical models that permit inferences about students' understandings once their responses have been observed. Psychometric approaches are sketched to construct measures of achievement from such tests.
SEMANTIC AND STRUCTURAL FACTORS AFFECTING THE PERFORMANCE OF MATCHED BLACK AND WHITE EXAMINEES ON ANALOGY ITEMS FROM THE SCHOLASTIC APTITUDE TEST
Authors: Freedle Roy , Kostin Irene 
Published: August 2014

ABSTRACTAnalogy items from the Scholastic Aptitude Test (SAT) were evaluated for differential performance by Black and White examinees. Black and White examinees were first matched for overall SAT‐V scores prior to conducting item analyses. A content and psycholinguistic analysis of 220 disclosed SAT analogy items was performed. Regression analyses indicate that Black examinees consistently perform differentially better than matched White examinees on the hard analogy items! However, for easy items, particularly those that involve “science” content, White examinees appear consistently to perform differentially better than matched Black examinees. In addition, semantic relationships dealing with part/whole relationships in the item stem also contributed negatively to Black examinee percent correct responses. Three variables (item difficulty, science content, part/whole relationship) together account for 30% of the variance between the two ethnic (Black and White) groups. Of these three significant predictors, two are semantic (part/whole and science content) while the third (item difficulty) reflects a non‐semantic factor. Several hypotheses are advanced to explain these findings.