bluuebunny/crossref_metadata_2025_split huggingface dataset:
THE RELATIONSHIP OF GRE GENERAL TEST ITEM‐TYPE PART SCORES TO UNDERGRADUATE GRADES
Authors: Wilson Kenneth M. 
Published: September 2014

ABSTRACTThis study was undertaken (a) to assess the relative contribution of a vocabulary score made up of GRE General Test antonyms and analogies and a reading comprehension score made up of GRE sentence completions and reading comprehension sets to prediction of an academic criterion (self‐reported undergraduate grade point average) and (b) to assess patterns of criterion‐related validity for item‐type part scores on the GRE quantitative and analytical measures as well.The study was based on data from GRE files for 9,375 examinees in 12 fields of study representing 437 undergraduate departments from 149 colleges and universities. All data were standardized within each undergraduate department and then pooled for analysis by field.There were differences by major field in average performance on the various item‐type part scores within each test. The reading comprehension subtest was found to carry most of the predictive load in the GRE verbal measure (consistent with findings for the reading comprehension subscore on the SAT verbal measure). Item‐type part scores on the other measures also exhibited differential patterns of relationships with the self‐reported undergraduate grade point average.The findings suggest that the different item types within the respective broad ability measures may be tapping somewhat unique skills and abilities and that further exploration of their potential contribution is in order.
FACTOR STRUCTURE OF THE GRE GENERAL TEST FOR OLDER EXAMINEES: IMPLICATIONS FOR CONSTRUCT VALIDITY
Authors: Stricker Lawrence J. , Rock Donald A. 
Published: September 2014

ABSTRACTThe aim of this study was to appraise the extent to which the GRE General Test has the same construct validity for older test takers as it has for other examinees by comparing the interrelationships among the test items for subgroups of examinees defined by age. The items in the test were factor analyzed for three samples of examinees, 20‐29 years old, 30‐39 years old, and 40‐49 years old. The same verbal, quantitative, and analytical ability factors were found in each sample, but the intercorrelations of the factors increased slightly, though consistently, with the age of the sample. These results suggest that the convergent validity of the test is similar across these age groups, but the discriminant validity is somewhat different for the older examinees. It was also found, in each sample, that the items in the analytical measure had a substantial amount of specific variance and that the analytical ability factor was highly correlated with the other factors, implying that these items have relatively little in common with each other and overlap excessively with verbal and quantitative ability.
TEST SPEEDEDNESS UNDER NUMBER‐RIGHT SCORING: AN ANALYSIS OF THE TEST OF ENGLISH AS A FOREIGN LANGUAGE
Authors: Bejar Isaac I. 
Published: September 2014

ABSTRACTThis investigation aimed to develop a new methodology for assessing the speededness of right‐scored tests such as the Test of English as a Foreign Language (TOEFL). A new methodology is needed because traditional procedures of assessing speededness assume that the test is scored under formula‐scoring instructions. The approach to the problem used in this study capitalizes on the fact that Item Response Theory is used in equating TOEFL. Two indices were investigated and validated against artificial data and real data in a way that was consistent with the ETS criteria for declaring forms speeded. Both indices were able to identify data that contained the speed component. The applications of these indices to several forms suggest that speed does not play a significant role in the TOEFL as a whole. However, Section 2 appeared to be affected by speed in administrations where pretest items are included. This may have been due to the slightly shorter time per item allowed when pretest items are included. It was recommended that one of the indices studied be used on an operational basis for a trial period and that the time limit for Section 2 be increased when pretest items are present or that the number of items be reduced to achieve the same effect.
FIELD TRIAL OF A USER‐ORIENTED ADAPTATION OF THE INVENTORY OF DOCUMENTED ACCOMPLISHMENTS AS A TOOL IN GRADUATE ADMISSIONS
Authors: Baird Leonard L. 
Published: September 2014

ABSTRACTA user‐oriented application survey was constructed, building on the work of the project to develop an inventory of documented accomplishments. The survey was sent to a small group of interested departments. The experiences and reactions of these departments to the structured admissions survey were reviewed, and the survey revised accordingly. Eighteen departments in three universities agreed to use this revised instrument as part of their regular admissions procedures. They were subsequently polled for their reactions to the form's utility, content, interpretability, advantages over current procedures, and comparability with the personal statement. In general, it appeared unlikely that a single form could be developed that would have appeal and applicability across fields.
XTREE: A MULTIVARIATE GRAPHICAL ICON WITH SPECIAL APPLICABILITY IN MONTE CARLO STUDIES1
Authors: Thissen David , Wainer Howard 
Published: September 2014

ABSTRACTComputer simulations are often used to compare statistical estimators. A common structure (e.g., Andrews, et. al., 1972) of these Monte Carlo studies is to compare a number of estimators for a number of distributions under a number of side conditions (i.e., various sample sizes). The investigator is faced with a substantial problem in trying to interpret voluminous output. The problem is exacerbated if there are multiple dependent variables (e.g., variance and bias). In this paper we propose a graphical icon, the XTREE, which aids perception and interpretation of such results. The method is illustrated with data from two recently published studies. A computer program that draws this display is provided.
LANGUAGE FACTORS AND HISPANIC FRESHMEN'S STUDENT PROFILE
Authors: Duran Richard P. , Enright Mary K. , Rock Donald A. 
Published: September 2014

ABSTRACTThis study investigated the role of language factors as potential determinants of Hispanics' educational progress. Hispanic as well as non‐Hispanic educators have argued that if we knew more about Hispanics' language background and language skills, we might improve the assessment of their cognitive and academic skills. More specifically, the purpose of this study was to investigate the relationship of language background factors to measures of college aptitude as measured by the Scholastic Aptitude Test (SAT) and other College Board Admissions Testing Program achievement and placement tests. Language background factors that were studied included amount of exposure to both Spanish and English, frequency of use of the two languages, and proficiency in each language.The students surveyed were freshmen in the fall of 1982 who were attending a sample of four‐year institutions that were participating in the College Board's Summary Reporting Service. The data analyzed included test scores, responses to the Student Descriptive Questionnaire (SDQ), and a special questionnaire supplement that included 36 items dealing with language background and language proficiency questions.The major questions that were addressed by this study were: (1) How did Hispanic freshmen vary with respect to their perceptions of language use and proficiency? (2) What were the relationships between their self‐perceptions of their language skills and their test scores? (3) Were there specific language survey questions that would add to the utility of the SDQ in predicting SAT scores?The principal findings of the study indicated that a large number of non‐SDQ language questions were more highly related to SAT and TSWE test performance than was the existing SDQ question 38 (Is English your best language?). Language survey items probing preferences for using Spanish versus English and freshmen's self‐ratings of English language proficiency seemed to be the most outstanding predictors of SAT‐verbal scores as well as TSWE scores. Many of the language survey questions improved the prediction of the SAT scores by 10 percent or more when used in combination with the SDQ question 38. In addition, the importance of language survey questions as predictors of SAT‐verbal scores varied by Spanish language group, with language survey items being most important for Puerto Ricans.
THE PSYCHOMETRIC CHARACTERISTICS OF THE SAT FOR NINE HANDICAPPED GROUPS
Authors: Bennett Randy Elliot , Rock Donald A. , Kaplan Bruce A. 
Published: September 2014

ABSTRACTThis study examined the psychometric characteristics of the Scholastic Aptitude Test (SAT) administered under special conditions for nine handicapped groups. Information about test characteristics is central to judging the accuracy and fairness of scores from SAT special administrations.Four psychometric characteristics were studied: level of test performance, test reliability, speededness, and extent of unexpected differential item performance. Psychometric comparisons were made between a nonhandicapped sample and each of nine different handicapped classifications. These contrasts were carried out twice; that is, they were replicated across two forms of the same test. The use of two samples taking different forms served to increase confidence in the stability of results and their applicability to other forms of the SAT.Results of the study showed that visually impaired students and those with physical handicaps achieved mean scores generally comparable to students taking the SAT in national administrations. Learning disabled and hearing impaired students scored lower than their nondisabled peers. Differences between Verbal and Mathematical performance were also comparable to those for the nondisabled reference group in all but the hearing impaired‐regular type test and visually impaired‐braille test samples. Hearing impaired‐regular students scored higher on Mathematical than on Verbal relative to their nondisabled peers, while visually impaired‐braille students showed no consistent superiority for Mathematical over Verbal.Analysis of test reliability revealed no practical differences in measurement precision across groups. Data on test speededness showed no evidence of disadvantage for disabled students; the amount of extended time allotted through special administrations appears to allow roughly equivalent proportions of handicapped and nondisabled examinees to complete the test.Because of the large number of groups and test items involved, unexpected differential item performance was examined through a two‐stage procedure. The first stage centered on the performance of item clusters. Individual items composing clusters showing questionable performance were then examined. This two‐stage procedure revealed only a few instances of differential item performance localized to visually impaired students taking the braille test.It is concluded that, with the exception of performance level, the psychometric characteristics of the SAT are generally comparable for the handicapped and nondisabled groups studied. These results lend support to the contention that scores from special administrations are fair and accurate measures of the developed scholastic abilities of handicapped students. Further studies of these scores–in particular, their factor structure and predictive validity–should provide additional information about their meaning for handicapped students.
THE INTERNAL CONSTRUCT VALIDITY OF THE SAT ACROSS HANDICAPPED AND NONHANDICAPPED POPULATIONS
Authors: Rock Donald A. , Bennett Randy Elliot , Kaplan Bruce A. 
Published: September 2014

ABSTRACTThis study investigated the comparability of SAT Verbal and Mathematical scores for one nonhandicapped and nine handicapped populations. A simple two‐factor model based on Verbal and Mathematical item parcels was posed and tested for invariance across populations with respect to: (1) the number and intercorrelation of factors, (2) the pattern of factor loadings, and (3) the equality of scale units.The two common‐factor model provided a reasonable fit in all populations, with the mathematical reasoning factor generally showing a better fit to the population data than the verbal reasoning factor. Compared with the nonhandicapped population, these factors tended to be less correlated in most of the handicapped groups. This somewhat greater specificity implies the increased likelihood of achievement growth in one area independent of the other, suggesting that the two scores be interpreted separately rather than as an SAT composite.With respect to the pattern of factor loadings, some indication of the presence of two verbal method factors was discovered. Antonym items and, to a lesser extent, reading comprehension items formed such factors. While present to a certain extent for all groups, the method variance seemed to be somewhat greater for the learning disabled populations.Finally, there was evidence that multiple‐choice Mathematical items led to different observed score scale units for the learning disabled students taking a cassette administration. Because of the small sample sizes on which this finding is based, it should be treated with caution. However, if dependable, the finding suggests that Mathematical scores may underestimate the reasoning ability of these students.
RELATIONSHIPS OF TEST ITEM CHARACTERISTICS TO TEST PREPARATION/TEST PRACTICE EFFECTS: A QUANTITATIVE SUMMARY
Authors: Powers Donald E. 
Published: September 2014

ABSTRACTTen studies of the effects of test preparation or test practice were located that provided effect estimates for individual test item types. These item‐level effects were related to a number of test item characteristics, including item format, difficulty level, and complexity of test directions. Strong relationships were noted between test preparation/practice effects and (a) the length and complexity of test directions and (b) the item format employed. Suggestions are made for further research on the relationship of test item characteristics to the susceptibility of tests to coaching and practice.
A PSYCHOMETRIC ANALYSIS OF A THREE‐DIMENSIONAL SPATIAL TASK
Authors: Bejar Isaac I. 
Published: September 2014

ABSTRACTThis paper considers the feasibility of incorporating research results from cognitive science into the modeling of performance on psychometric tests and the construction of test items. The paper focuses on the feasibility of modeling performance on a three‐dimensional rotation task within the context of Item Response Theory (IRT). Three‐dimensional items were chosen because there is a rich literature on the mental models that are used in their solution. To test the feasibility of psychometrically modeling performance on this item type an 80‐item, three‐dimensional rotation test was constructed. An inexpensive computer system was also developed to administer the test and record performance, including response‐time data. Data were collected on high school juniors and seniors. As expected, angular disparity was a potent determinant of item difficulty. The applicability of IRT to these data was investigated by dichotomizing response time at several points and applying standard item parameter estimation procedures. It was concluded that an approach to psychometric modeling that explicitly incorporates information on the mental models examiners use in solving an item is not only workable, but also essential for future developments in psychometrics.
LARGE‐SCALE EDUCATIONAL ASSESSMENT AS POLICY RESEARCH: ASPIRATIONS AND LIMITATIONS1
Authors: Messick Samuel 
Published: September 2014

ABSTRACTCritical features important for transforming large‐scale educational assessment into effective policy research are examined. These include a developmental orientation to time‐ordered data, empirically‐grounded construct interpretations of measures and relationships, speculative inquiry into the role of diverse contributing factors, and appraisal of alternative perspectives on both the questions and the findings.
THE PREDICTIVE VALIDITY OF THE SCHOLASTIC APTITUDE TEST FOR DISABLED STUDENTS
Authors: Braun Henry , Ragosta Marjorie , Kaplan Bruce 
Published: September 2014

ABSTRACTIn two separate rounds of data collection, validity data on disabled students were obtained from 145 institutions with validity data on nonhandicapped students. First year grade point averages (FYAs) were obtained for almost 1,000 disabled students who had taken special test administrations of the SAT with extra time and for more than 650 disabled students who had taken standard test administrations. Empirical Bayes procedures were used in conjunction with the sample of nonhandicapped students to develop separate regression equations for each of the 145 institutions. The focus of this study was whether regression equations based on data from nonhandicapped students predict the performance of handicapped students as well as performance of the nonhandicapped.Consistent with findings from other reports in this series the SAT performance of visually impaired and physically handicapped people was not very different from that of the nonhandicapped students. The SAT scores of learning‐disabled students were considerably lower and those of hearing‐impaired students even lower.A pattern of over‐ and underprediction was evident in FYA predictions based on high school grades alone. Disabled students earning the lowest high school grade point averages tended to be underpredicted–i.e. predicted to earn FYAs lower than their actual FYAs–while disabled students earning the highest high school grades tended to be overpredicted.A second pattern emerged from predictions based on SAT scores alone. Except for hearing‐impaired students, SAT scores from special test administrations have a strong tendency to overpredict the college performance of students with disabilities. The effect is strongest for those with learning disabilities.Using both high school grades and SAT scores to predict the college performance of students from special test administrations results in good overall predictions, but only because overprediction in some areas is offset by underprediction in others. The overprediction of the strongest third of the candidates is balanced by the underprediction of the weakest third.
THE FEASIBILITY OF RESCALING TEST SCORES OF HANDICAPPED EXAMINEES
Authors: Powers Donald E. , Willingham Warren W. 
Published: September 2014

ABSTRACTA number of testing programs offer handicapped examinees the opportunities to take admissions tests under nonstandard conditions. Typically, the test scores based on these nonstandard administrations have been flagged to indicate to test users that the scores may not be comparable to those from standard administrations. In order to avoid thus identifying handicapped applicants, a panel established by the National Academy of Sciences suggested the possibility of rescaling the scores of handicapped students to make grade predictions comparable for handicapped and nonhandicapped students. This report examines this possibility and concludes that the approach is not feasible technically and furthermore that it would also have a number of potentially serious undesirable side effects.
TEST PREPARATION FOR THE GRE ANALYTICAL ABILITY MEASURE: DIFFERENTIAL EFFECTS FOR SUBGROUPS OF GRE TEST TAKERS
Authors: Powers Donald E. 
Published: September 2014

ABSTRACTA previous study of the initial version of the GRE analytical ability measure (Powers & Swinton, 1984) revealed practically and statistically significant effects of test familiarization on analytical test scores. (Two susceptible item types were subsequently removed from the test.) Data from this study were reanalyzed for evidence of differential effects for subgroups of examinees classified by age, ethnicity, degree aspiration, English language dominance, and performance on other sections of the GRE General Test.The results suggested little, if any, difference among subgroups of examinees with respect to their response to the particular kind of test preparation considered in the study. Within the limits of the data, no particular subgroup appeared to benefit significantly more or significantly less than any other subgroup.
CAN WE ACCURATELY ASSESS CHANGES IN MINORITY PERFORMANCE ON THE SAT?
Authors: Wainer Howard 
Published: September 2014

ABSTRACTThe performance of minority examinees on the SAT is carefully monitored by the national educational media. Changes of 10 or 15 points over a five year period are interpreted as having a significant and important relationship to the educational process. A crucial assumption underlying the validity of this inference is that the performance of an examinee is unrelated to that examinee choosing to identify his or her ethnicity. In this paper we show that this assumption is false and that the potential errors introduced by it dwarf the changes being interpreted as real.
EELWORMS, BULLET HOLES, AND GERALDINE FERRARO
Authors: Wainer Howard 
Published: September 2014

ABSTRACTThere is no safety in numbers. When data are gathered from a sample in which the selection criteria are unknown many problems can befall the unwary investigator. In this paper we explore some of these problems and discuss some solutions. Our principal example is drawn from data from students who choose to take the College Board's Scholastic Aptitude Test (the SAT). We explore methods of covariance adjustment as well as more explicitly model‐based adjustment methods. Among the latter we discuss Heckman's Selection Model, Rubin's Mixture Model, and Tukey's Simplified Selection Model.
A FAST SCORING ALGORITHM FOR MAXIMUM LIKELIHOOD ESTIMATION IN UNBALANCED MIXED MODELS WITH NESTED RANDOM EFFECTS
Authors: Longford Nicholas 
Published: September 2014

SUMMARYA fast Fisher scoring algorithm for maximum likelihood estimation in unbalanced mixed models with nested random effects is described. The algorithm uses explicit formulae for the inverse and the determinant of the covariance matrix, given by LaMotte (1972), and avoids inversion of large matrices. Description of the algorithm concentrates on computational aspects for large sets of data.