bluuebunny/crossref_metadata_2025_split huggingface dataset:
ATTITUDES TOWARD BORROWING AND PARTICIPATION IN POSTSECONDARY EDUCATION
Authors: Ekstrom Ruth B. 
Published: August 2014

ABSTRACTHigh school seniors who answered that they would choose to borrow, when asked what they would do if college costs were $1,500 more than they, their family, and a scholarship could provide, were significantly more likely to attend college in the next four years than were students who answered that they would choose other options (delaying college entrance, attending a less expensive college, or getting a job). This held true even after controlling for other variables such as educational aspirations, tested achievement, influence from others, and socioeconomic status. This finding supports the contention that students who are reluctant to borrow are less likely to enroll in postsecondary education. Among the students who did enroll in college, willingness to borrow was significantly associated with attendance at a four‐year college rather than a two‐year college and with full‐time rather than part‐time attendance. This may mean that recent changes in financial aid packaging, with fewer grants and more loans, have had an effect not only on college enrollments but also on the type of college that students attend and on the proportion of students enrolling in college on a part‐time basis. The results also suggest that, all else being equal, students who are knowledgeable about financial aid sources may be more likely to enroll in college than are students with less knowledge of these sources.
A REVIEW OF AUTOMATICALLY SCORABLE CONSTRUCTED‐RESPONSE ITEM TYPES FOR LARGE‐SCALE ASSESSMENT
Authors: Martinez Michael E. , Bennett Randy Elliot 
Published: August 2014

ABSTRACTThe use of automated scanning of test sheets, beginning in the 1930s, led to widespread use of the multiple‐choice format in standardized testing. New forms of automated scoring now hold out the possibility of making a wide range of constructed‐response item formats feasible for use on a large‐scale basis. We describe new developments in five domains: mathematical reasoning, algebra problem solving, computer science, architecture, and natural language. For each one, we describe the task as presented to the examinee, the methods used to score the response, and the psychometric properties of the item responses. We then highlight general challenges and issues spanning these technologies. We conclude by offering our views on the ways in which such technologies are likely to shape the future of testing.
FIGURAL‐RESPONSE ASSESSMENT: SYSTEM DEVELOPMENT AND PILOT RESEARCH IN CELL AND MOLECULAR BIOLOGY
Authors: Martinez Michael E. , Jenkins Jeffrey B. 
Published: August 2014

ABSTRACTThe purpose of the GRE figural‐response project was to design a prototype assessment system for delivering and scoring figural‐response items in the domain of cell and molecular biology, and to begin to investigate properties of the item format This report describes progress to date in an effort that is intended to be continuous and lead to program implementation. We first describe features of the delivery system and give sample items; ancillary developments, such as a tutorial, are also noted. Then, findings from a pilot research study are described. The essence of the pilot study was to examine the relationships between two item types (figural‐response and open‐ended verbal questions) and measures of figural and verbal ability. The data hint that whereas verbal items draw from verbal ability, figural‐response items draw from figural and verbal ability. The report concludes with a discussion of possible new directions for research, development, and eventual program use of the item format.
ASSESSING THE CLASSROOM PERFORMANCE OF BEGINNING TEACHERS: EDUCATORS' APPRAISAL OF PROPOSED EVALUATION CRITERIA
Authors: Powers Donald E. 
Published: August 2014

ABSTRACTSurveys were conducted of two multi‐state samples of educators, who provided ratings of the importance of two preliminary versions of the classroom performance assessment criteria being developed for The Praxis Series: Professional Assessments for Beginning Teachers™. There was considerable agreement about the importance of the criteria across several classifications of educators according to ethnicity, instructional level, years of teaching experience, subject area, and orientation to teaching.
COGNITIVE TASK ANALYSIS, INTERFACE DESIGN, AND TECHNICAL TROUBLESHOOTING
Authors: Steinberg Linda S. , Gitomer ew H. 
Published: August 2014

ABSTRACTWe propose a model of the interface design process that makes use of two interdependent levels of cognitive analysis: 1) the study of the criterion task through an analysis of expert/novice differences and; 2) the evaluation of the working user interface design through the application of a practical interface analysis methodology (GOMS model). We review this dual analysis in the context of HYDRIVE, a video‐disc based intelligent tutoring system designed to facilitate the development of troubleshooting skills for aircraft hydraulics systems. The initial cognitive task analysis enabled the identification of critical troubleshooting skills and troubleshooting procedures. We find, though, mat even with an in‐depth initial cognitive task analysis, the GOMS interface analysis resulted in significant and beneficial design changes.
EXAMINEE BACKGROUND CHARACTERISTICS AND GRE GENERAL TEST PERFORMANCE
Authors: Stricker Lawrence J. , Rock Donald A. 
Published: August 2014

ABSTRACTThe relationships between examinee background characteristics and performance on the GRE General Test were appraised by a structural equation modeling analysis. The examinees' initial characteristics (sex, ethnicity, parental education, geographic region, and age) had modest relationships with their test performance. Of these, parental education had the most consistent and strongest association. Sex also had an appreciable association, but it was limited to the quantitative score. College‐related characteristics (college major and the institution's public vs. private control, Carnegie classification, selectivity, and Ph.D. productivity) and undergraduate grade‐point average (GPA) generally had stronger and more pervasive relationships with test performance than did the examinees' initial characteristics, not only mediating the associations of the examinees' initial characteristics with test performance but also making independent contributions in their own right. The associations were especially strong for school quality (a composite of public vs. private control, selectivity, and Ph.D. productivity), college major, and undergraduate GPA.
PARTICIPATION IN RELIGIOUS ACTIVITIES BY COLLEGE‐BOUND HIGH SCHOOL SENIORS
Authors: Grandy Jerilee 
Published: August 2014

ABSTRACTUnder a grant from the Lilly Endowment, ETS analyzed the 1989–90 Scholastic Aptitude Test (SAT) database to create profiles of students who participate in religious activities or organizations while in high school. Analyses included the comparison of profiles of students participating in religious activities versus those who did not, based on the entire student population. Then two denominational case studies of Presbyterian students and Roman Catholic students sought to answer questions of specific concern to policy makers in those denominations.On the average, test takers who participated in religious activities differed from those who did not participate in the following ways:• Their families had more education and a higher income, their SAT scores were higher, their grades in math, science, and writing were higher, more had taken honors courses, and they had higher degree aspirations.• Students who participated in religious activities also participated more frequently in other activities, especially community or service activities.• Students who participated in religious activities more often chose humanities, social sciences, health sciences, and education as college majors. They chose business less often than did students who did not participate in religious activities.Additional information from the case study of Presbyterians was the following:• If students were not participating in religious activities or organizations by ninth grade, they generally did not join later in high school.• On the average, students who remained in religious activities or organizations throughout high school earned higher grades and test scores than those who dropped out.The case study of Roman Catholic test takers yielded the following findings:• A smaller percentage of Roman Catholic students participated in religious activities than did SAT takers as a whole. Catholic students attending religiously affiliated high schools participated in religious activities at about the same rate as mainline Protestant students attending all types of schools.• Roman Catholic students attending religiously affiliated schools reported greater participation in religious activities and community activities than did Catholic students in public schools, but the number planning to participate in religious activities in college was not significantly higher for those attending religiously affiliated schools than for those in public schools.• A comparison among Roman Catholic ethnic groups attending religiously affiliated schools showed Asian Americans reporting the highest levels of participation in religious activities in high school, participation in community service activities in high school, intention to participate in religious activities in college, and intention to attend a religiously affiliated college.
THE MATTER OF STYLE: MANIFESTATIONS OF PERSONALITY IN COGNITION, LEARNING, AND TEACHING
Authors: Messick Samuel 
Published: August 2014

ABSTRACTThe historical roots of cognitive styles are traced in differential psychology, psychoanalytic ego psychology, gestalt and cognitive‐developmental psychology to illuminate the varied theoretical issues that energize (and fragment) style research. Optimal measurement of cognitive styles as information‐processing regularities and as intraindividual contrasts of abilities or other attributes is discussed in terms of both typical performance and contrasted performance on measures of opposing ends of bipolar style dimensions, with special emphasis on the contrasted measurement of field independence versus field dependence. The role of styles as both performance variables and competence variables in learning and teaching is examined, as are various critiques of style research that appear to be excessively polarized in either supporting or undercutting styles as meaningful constructs. This polarization appears to reflect different stances not just with respect to scientific evidence but with respect to ideology. Concluding remarks broach the issue of how styles are organized, not just within an information‐processing framework but within the structure of personality.
THE MATTER OF STYLE: MANIFESTATIONS OF PERSONALITY IN COGNITION, LEARNING, AND TEACHING
Authors: Messick Samuel 
Published: August 2014

ABSTRACTThis project has been an outgrowth of the recently completed report Change in Field of Study from Undergraduate to Graduate School: Creation of a GRE Data Base for Studying Talent Flow by Jerilee Grandy and Nancy Robertson. Both examinee‐level and matrix data files created and analyzed under the original project were rearranged to make them more accessible to researchers. These multiyear files (covering testing years 1977–78 through 1986–87) have also been documented so that they can be easily located and analyzed. Extension of the data base through 1991–92 is underway and will be completed in 1993.
GENERALIZABILITY, VALIDITY, AND EXAMINEE PERCEPTIONS OF A COMPUTER‐DELIVERED FORMULATING‐HYPOTHESES TEST
Authors: Bennett Randy Elliot , Rock Donald A. 
Published: August 2014

ABSTRACTFormulating‐Hypotheses (F‐H) items present a situation and ask the examinee to generate as many explanations for it as possible. This study examined the generalizability, validity, and examinee perceptions of a computer‐delivered version of the task. Eight F‐H questions were administered to 192 graduate students. Half of the items restricted examinees to 7 words per explanation and half allowed up to 15 words. Generalizability results showed high interrater agreement, with tests of between two and four items scored by one judge achieving coefficients in the .80s. As in studies of paper‐and‐pencil versions, validity analyses found that although F‐H was highly reliable, it was only weakly related to GRE General Test scores, differing from that test primarily in relating more strongly to a measure of ideational fluency. Versions of F‐H based on different response limits tapped somewhat different abilities, with items employing the 15‐word constraint appearing more useful for graduate assessment. These items added to conventional measures in explaining school performance and creative expression. Finally, although the overwhelming majority of examinees found the F‐H interface easy to use, some experienced difficulty, suggesting the possibility that computer familiarity constitutes a source of irrelevant variance in F‐H scores.
PROFICIENCY SCALING BASED ON CONDITIONAL PROBABILITY FUNCTIONS FOR ATTRIBUTES
Authors: Tatsuoka Kikumi K. , Birenbaum Menucha , Lewis Charles , Sheehan Kathleen M. 
Published: August 2014

ABSTRACTThis study introduces procedures for constructing a proficiency scale for a large‐scale test by applying Tatsuoka's Rule Space Model. The SAT Mathematics (SAT M), Section 2, is used for illustrating the process and the results. A task analysis is summarized in a mapping sentence, and then 14 processes and content attributes are identified for explaining the underlying cognitive aspects of the examinees' performance on the SAT M. Analysis results show that almost 98% of 2334 examinees are successfully classified into one of 468 cognitive states. The cognitive states are characterized by mastery or non‐mastery of the 14 attributes. Attribute Characteristic Curves, which are conditional probability functions defined on the SAT Scale, are introduced and used for interpreting an examinees' proficiency. Prototypes of a student's performance report and a group performance report are given as examples of possible ways for summarizing the analysis results.
RELIABILITY OF ESSAY RATING AND SCORE ADJUSTMENT
Authors: Longford N. T. 
Published: August 2014

ABSTRACTA model‐based approach to rater reliability for essays read by multiple readers is presented. Variation of rater severity (between‐rater variation) and rater inconsistency (within‐rater variation) is considered in presence of between‐examinee variation. An additive variance component model is posited and the method of moments for its estimation described. The model involves no distributional assumptions. Minimum mean squared error estimators of examinees' true scores and readers' severities are derived. Model diagnostic procedures are an integral component of the approach. The methods are illustrated on data from standardized educational tests.
DERIVING COMPARABLE SCORES FOR COMPUTER ADAPTIVE AND CONVENTIONAL TESTS: AN EXAMPLE USING THE SAT1,2
Authors: Eignor Daniel R. 
Published: August 2014

ABSTRACTProcedures used to establish the comparability of scores derived from the College Board Admissions Testing Program (ATP) computer adaptive SAT prototype and the paper‐and‐pencil SAT are described in this report. Both the prototype, which is made up of Verbal and Mathematical computer adaptive tests (CATs), and a form of the paper‐and‐pencil test were administered to just greater than 500 examinees using a random groups counterbalanced design. Both linear and equipercentile procedures were used for equating in each of the separate testing orders (paper‐and‐pencil then CAT or CAT then paper‐and‐pencil). Data were not pooled across the orders because the groups were not randomly equivalent due to administrative problems. The linear procedure was chosen for each test (Verbal or Mathematical) for each order, and results from the two orders were averaged. The final Verbal and Mathematical CAT conversions were quite similar to the paper‐and‐pencil conversions, although the two conversions for Verbal and two conversions for Mathematical did differ by as much as 20 scaled score points in certain regions of the scale.
CASE STUDIES IN COMPUTER ADAPTIVE TEST DESIGN THROUGH SIMULATION1,2
Authors: Eignor Daniel R. , Stocking Martha L. , Way Walter D. , Steffen Manfred 
Published: August 2014

ABSTRACTThe extensive computer simulation work done in developing the computer adaptive versions of the Graduate Record Examinations (GRE) Board General Test and the College Board Admissions Testing Program (ATP) SAT is described in this report. Both the GRE General and SAT computer adaptive tests (CATs), which are fixed length in nature, were developed from pools of items that were calibrated using the three‐parameter logistic IRT model and item selection was based on the recently developed weighted deviations algorithm (see Swanson and Stocking, 1992), which simultaneously deals with content, statistical, and other constraints in the item selection process. For the GRE General CATs (Verbal, Quantitative, and Analytical), item exposure was controlled by using an extension of an approach originally developed by Sympson and Hetter (1988). For the SAT CATs (Verbal and Mathematical), item exposure was controlled by using a less complex randomization approach. Lengths of the CATs were determined so that CAT reliabilities matched or exceeded comparable full length paper‐and‐pencil test reliabilities.
PASSAGE DEPENDENCE OF THE NEW SAT® READING COMPREHENSION QUESTIONS
Authors: Powers Donald E. , Wilson Susan T. 
Published: August 2014

ABSTRACTIt has been reasonably well established that test takers can, to varying degrees, answer some reading comprehension questions correctly without reading the passages on which the questions are based. This is true even for carefully constructed measures such as the College Board's SAT. As a result, the use of reading scores, including those from the SAT, as valid indicators of reading comprehension has been challenged.The major aim of this study was to determine the strategies by which examinees are able to achieve better‐than‐chance performances without reading the passages. The focus of the research was a sample of reading comprehension questions similar to those that will be used in the revised SAT, to be introduced in 1994. Sets of reading comprehension questions were administered, without the passages, to a sample of verbally able students in eight secondary schools across the country. After attempting the task, these students were asked to complete a questionnaire describing the strategies they had used. The most often cited strategies involved choosing answers on the basis of consistency with other questions and reconstructing the main theme of a missing passage from all the questions and answers in a set. These strategies were also more likely to result in successful performance on individual test items than were any of the many other possible (and less construct‐relevant) strategies. Implications are discussed with regard to the construct validity of the new SAT reading comprehension questions and the advice that should be given to prospective SAT takers.
THREE PRACTICAL ISSUES FOR MODERN ADAPTIVE TESTING ITEM POOLS1
Authors: Stocking Martha L. 
Published: August 2014

ABSTRACTAs adaptive testing moves towards operational implementation in large scale testing programs, where it is important that adaptive tests be as parallel as possible to existing linear tests, a number of practical issues arise. This paper concerns three such issues. First, optimum item pool size is difficult to determine in advance of pool construction. Retrospective results are analyzed for five operational pools; these analyses indicate that item pools of the size and quality of six to eight linear tests are adequate to support adaptive tests of roughly half the length of a parallel linear test. Second, item pools may not support sufficiently low exposure rates for items or a sufficiently small amount of test overlap to maintain test security when testing is conducted on a continuous basis. Various simulations suggest that multiple pools which can be chosen randomly before adaptive testing begins provide a satisfactory solution. Finally, over time it will be necessary to refresh or replace operational item pools. Issues that must be considered in refreshing or replacing item pools are discussed, and guidance is given for establishing benchmark values from initial pools as standards that must be met for adaptive tests from new pools to be considered parallel to adaptive tests from existing pools.
EFFECT OF RASCH CALIBRATION ON ABILITY AND DIF ESTIMATION IN COMPUTER‐ADAPTIVE TESTS
Authors: Zwick Rebecca , Thayer Dorothy T. , Wingersky Marilyn 
Published: August 2014

ABSTRACTA simulation study of methods of assessing differential item functioning (DIF) in computer‐adaptive tests (CATs) was conducted by Zwick, Thayer and Wingersky (in press; 1993). Results showed that modified versions of the Mantel‐Haenszel and standardization methods work well with CAT data. In that study, data were generated using the three‐parameter logistic (3PL) model and this same model was assumed in obtaining item parameter estimates. In the current study, 3PL item response data were used, but the Rasch model was assumed in obtaining item parameter estimates, which, in turn, determined the information table to be used in the item selection algorithm. New Rasch‐based expected true scores were obtained for each examinee, based on responses to the CAT items. As in the previous study, the DIF statistics were highly correlated with the generating DIF, and the means and standard deviations of these statistics across items were close to their nominal values. There was, however, a tendency for DIF statistics to be slightly smaller in magnitude than in the 3PL analysis, resulting in a lower probability of detecting items with extreme DIF. This reduced sensitivity appeared to be related to a degradation in the accuracy of matching. Expected true scores from the Rasch‐based CAT tended to be biased downward, particularly for lower‐ability examinees. Unlike the Rasch CAT scores, Rasch expected true scores based on nonadaptive administration of all pool items behaved quite well, as did the nonadaptive and CAT‐based expected true scores obtained using the 3PL model.
DIF ANALYSIS FOR PRETEST ITEMS IN COMPUTER‐ADAPTIVE TESTING
Authors: Zwick Rebecca , Thayer Dorothy T. , Wingersky Marilyn 
Published: August 2014

ABSTRACTA simulation study of methods of assessing differential item functioning (DIF) in computer‐adaptive tests (CATs) was conducted by Zwick, Thayer and Wingersky (in press; 1993). Results showed that modified versions of the Mantel‐Haenszel and standardization methods work well with CAT data. DIF methods were also investigated for nonadaptive “pretest” items, for which item parameter estimates were assumed unavailable. The pretest DIF statistics were generally well‐behaved, but the Mantel‐Haenszel DIF statistics tended to have larger standard errors for the pretest items than for the CAT items. The current extension of the earlier work addressed the effect of using alternative matching methods for pretest items. Using a more elegant matching procedure did not lead to a reduction of the Mantel‐Haenszel standard errors and produced DIF measures that were nearly identical to those from the earlier study. Further investigation showed that the Mantel‐Haenszel standard errors tended to be larger when items were administered to examinees with a wide ability range, whereas the opposite was true of the standard errors of the standardization DIF statistic. Some theoretical findings were obtained that appear to explain this phenomenon.