bluuebunny/crossref_metadata_2025_split huggingface dataset:
THE GRADED UNFOLDING MODEL: A UNIDIMENSIONAL ITEM RESPONSE MODEL FOR UNFOLDING GRADED RESPONSES
Authors: Roberts James S. , Laughlin James E. 
Published: August 2014

ABSTRACTBinary or graded disagree‐agree responses to attitude items are often collected for the purpose of attitude measurement. Although such data are sometimes analyzed with cumulative measurement models, recent investigations suggest that unfolding models are more appropriate (Roberts, 1995; Van Schuur & Kiers, 1994). Advances in item response theory (IRT) have led to the development of several parametric unfolding models for binary data (Andrich, 1988; Andrich & Luo, 1993; Hoijtink, 1991), but IRT models for unfolding graded responses have not been addressed in the psychometric literature. A parametric IRT model for unfolding either binary or graded responses was developed in this study. The model, called the graded unfolding model (GUM), is a generalization of Andrich & Luo's (1993) hyperbolic cosine model for binary data. A joint maximum likelihood procedure was implemented to estimate GUM parameters, and a subsequent recovery simulation showed that reasonably accurate estimates could be obtained with minimal data demands (e.g., as few as 100 subjects and 15 to 20 6‐category items). The applicability of the GUM to common attitude testing situations was illustrated with real data on student attitudes toward capital punishment. Index terms: attitude measurement, graded unfolding model, hyperbolic cosine model, ideal point process, item response theory, Likert scale, Thurstone scale, unidimensional scaling, unfolding model.
BRIDGING COGNITION AND PERSONALITY IN EDUCATION: THE ROLE OF STYLE IN PERFORMANCE AND DEVELOPMENT
Authors: Messick Samuel 
Published: August 2014

ABSTRACTIn an attempt to illuminate the interrelatedness of noncognitive and cognitive domains – or, more pointedly, of affection, conation, and cognition – emphasis was placed on variables that not only operate across these domains but are also potentially integrative. Prominent among such variables are personal styles, particularly cognitive styles. The properties and problems of cognitive styles are examined, with special emphasis on field independence versus field sensitivity and on two stylistic dimensions of attentional scanning. The role of cognitive styles as both competence variables and performance variables is addressed, along with the difficulty of disentangling style from ability because of reciprocal determinism in their development. The educational implications of cognitive styles are explored, especially as they bear on the problem of the match between student characteristics and educational experiences and on the value‐laden nature of style‐based pedagogical decisions.
DIFFERENCES IN THE SURVEY RESPONSES OF ASIAN AMERICAN AND WHITE SCIENCE AND ENGINEERING STUDENTS
Authors: Grandy Jerilee 
Published: August 2014

ABSTRACTA recent survey of gender and ethnic differences among science and engineering majors suggested that Asian American students were less content with their undergraduate experiences and with their field of study than were White students or students of other ethnic groups. Asian Americans appeared to enjoy their studies less and to feel more pressured by family to continue their science or engineering studies. Their responses indicated that they were less encouraged by professors to continue in their field, and they gave lower ratings to their relationships with professors. This study reports the results of additional analyses aimed at better understanding the responses of Asian Americans.Results of the additional analyses indicated that Asian American students tended to endorse the middle response on a 5‐point Likert scale more than White students did, and to avoid extreme responses. This response style accounted, in part, for the lower ratings Asian Americans gave to their undergraduate experiences. When items were rescored, collapsing the 5‐point scale to a 3‐point scale, Asian American ratings were still lower. These results indicated that there was at least one type of Asian American response set affecting the comparison of Asian Americans with Whites. Furthermore, Asian Americans were either less content with their studies, or, the tendency to respond more neutrally to questions of judgment was another type of response set.A hypothesis was posed that students in large research universities would be less content than students in other institutions, and that because a greater proportion of Asian Americans than Whites attend large research universities, the relative discontent of Asian Americans might be attributable to the type of institution attended. This hypothesis was not supported by the data.Response styles that are unique to specific ethnic groups may lead researcher's to draw erroneous conclusions from survey data. A number of recommendations are made for further research to determine how general the Asian American response set is, and to attempt to account for the lower satisfaction ratings of Asian Americans even when a correction is made for a response set.
OPTIMAL DESIGN OF ITEM POOLS FOR COMPUTERIZED ADAPTIVE TESTS
Authors: Stocking Martha L. , Swanson Len 
Published: August 2014

ABSTRACTMethods of optimal test assembly have served in the past as the foundation for the development of methods of assembling adaptive tests on‐the‐fly, A particular model and heuristic that facilitate the assembly of adaptive tests of the same complex structure as linear tests have already been developed (Swanson & Stocking, 1993; Stocking & Swanson, 1993). Similar methods can be used in the assembly of item pools designed to support adaptive testing. In the continuous testing environment of adaptive testing, optimal item pool design must simultaneously take into account the competing goals of reducing item exposure to enhance item security and maximizing this exposure to make the greatest possible use of every item. An extension of previous methods of optimal design to the issues of item pool design in the context of adaptive testing is developed in this paper, with examples.
DESCRIBING AND CATEGORIZING DIF IN POLYTOMOUS ITEMS
Authors: Zwick Rebecca , Thayer Dorothy T. , Mazzeo John 
Published: August 2014

ABSTRACTThe purpose of this project was to evaluate statistical procedures for assessing differential item functioning (DIF) in polytomous items (items with more than two score categories). Three descriptive statistics—the Standardized Mean Difference, or SMD (Dorans & Schmitt, 1991), and two procedures based on SIBTEST (Shealy & Stout, 1993) were considered, along with five inferential procedures—two based on SMD, two based on SIBTEST, and the Mantel (1963) method. The DIF procedures were evaluated through applications to simulated data, as well as data from ETS tests.The simulation included conditions in which the two groups of examinees had the same ability distribution and conditions in which the group means differed by one standard deviation. When the two groups had the same distribution, the descriptive index that performed best was the SMD. When the two groups had different distributions, a modified form of the SIBTEST DIF effect size measure tended to perform best. The five inferential procedures performed almost indistinguishably when the two groups had identical distributions. When the two groups had different distributions and the studied item was highly discriminating, the SIBTEST procedures showed much better Type I error control than did the SMD and Mantel methods, particularly in short tests. The power ranking of the five procedures was inconsistent; it depended on the direction of DIF and other factors.Routine application of these polytomous DIF methods at ETS seems feasible in cases where a reliable test is available for matching examinees. For the Mantel and SMD methods, Type I error control may be a concern under certain conditions. In the case of SIBTEST, the current version cannot easily accommodate matching tests that do not use number‐right scoring. Additional research in these areas is likely to be useful.
WORK VALUES OF COLLEGE BOUND AND OTHER HIGH SCHOOL STUDENTS
Authors: Stricker Lawrence J. 
Published: August 2014

ABSTRACTThis study identified the salient work values of college bound and noncollege bound high school students from their perceptions of occupations. Multidimensional scaling analyses of the students' judgments about the similarity of occupations extracted three factors in the college bound sample and seven in the noncollege bound sample. The same blue collar dimension appeared in both samples; additional dimensions in the noncollege bound sample were fragments of the broader dimensions in the college bound sample. These results indicate that the perceptions were more differentiated for the noncollege bound students than for the other students, but did not reflect different aspects of occupations, implying that the same work values were salient for both groups.
AN INVESTIGATION OF THE ROLE OF EDUCATIONAL BACKGROUND ON PERFORMANCE ON THE GRE® ANALYTICAL MEASURE
Authors: Enright Mary K. , Bejar Isaac I. 
Published: August 2014

ABSTRACTThere have been a number of reports that educational background is related to performance on reasoning problems similar to those used on the GRE analytical measure. The purpose of this research was to evaluate whether educational background as defined by broad undergraduate major fields was related to differential difficulty for specific items on the GRE analytical measure. An index of differential difficulty, MH‐DDIF, was calculated for the reasoning items from 13 GRE forms. We found only low to moderate levels of MH‐DDIF for the two item types on the analytical measure. A higher proportion of logical reasoning items than analytical reasoning items had moderate MH‐DDIF. The relationship of item classification attributes to MH‐DDIF was evaluated. In addition, other attributes potentially related to MH‐DDIF for logical reasoning items were identified and their ability to predict MH‐DDIF was evaluated with a cross‐validation design. The attributes identified as potentially related to MH‐DDIF for items from one subset of GRE forms did not predict MH‐DDIF for items in a second subset. Thus, a strong link between item performance and educational background was not observed within item types. Because of the failure to cross‐validate the potential predictors of DIF, other factors that contribute to MH‐DDIF are briefly considered. The relationship of these results to other research on the effect of educational background is discussed. We also discuss implications for test design and how test development practices are effective in minimizing the impact of educational background within item types.
AUTOMATIC SCORING OF ADVANCED PLACEMENT BIOLOGY ESSAYS
Authors: Burstein Jill , Kaplan Randy , Wolff Susanne , Lu Chi 
Published: August 2014

ABSTRACTThe Natural Language Processing Initiative (NLPI) at ETS currently does research and development for computer‐assisted scoring of natural language constructed‐responses. The purpose of NLPI's work is to develop computer‐assisted methods for scoring so that computer‐administered natural language constructed‐response items can be integrated into ETS' tests which can be scored efficiently with regard to time and cost. Until recently, NLPI's automated scoring efforts were primarily devoted to the development of computer programs used to score short‐answer responses constructed‐responses of up to 15 words. In previous studies, we have developed scoring systems for main idea item types (e.g., GUIDES items) and divergent thinking item types (e.g., the Formulating‐Hypotheses item (FH)). In the current study, we applied similar techniques to those used to score short‐answer responses in order to implement a prototype to automatically score AP Biology essays. This paper reports on the development process involved in building a linguistically‐based, computer scoring prototype for AP Biology essays and the results of the scoring system. The prototype was used to score essays that had been assigned an Excellent rating (9 or 10 points) or a Poor rating (0 ‐ 3 points) by human graders.1 The results reported in the paper illustrate agreement between human rater scores and computer‐based scores. The results of the study are promising and would encourage continued research in automated scoring of essays with highly analytical rubrics.
DEVELOPMENT OF A SCALE FOR ASSESSING THE LEVEL OF COMPUTER FAMILIARITY OF TOEFL EXAMINEES
Authors: Eignor Daniel , Taylor Carol , Kirsch Irwin , Jamieson Joan 
Published: August 2014

ABSTRACTThe increasing use of computer‐based testing raises concerns about equity and bias. Specifically, many in the field of language testing are concerned that the introduction of a computer‐based TOEFL test in 1998 may introduce bias by confounding the measurement of English‐language proficiency with level of computer familiarity. As the first step in a series of research efforts to address this question, a 23‐item questionnaire focusing on individuals' access to, attitude toward, and experience with using computers as well as related technology was developed and administered to TOEFL test takers in April and May of 1996 as part of a TOEFL test administration. Results of this survey administration are documented in a paper by Kirsch, Jamieson, Taylor, and Eignor (1998).This paper has two purposes. The first is to explain decisions as to which questionnaire items would subsequently be used together as a score to separate TOEFL examinees into levels of computer familiarity. The second purpose is to provide details on the procedures used to assess the underlying factor structure of the complete questionnaire. Factor analysis procedures (specifically, a principal factors approach with both varimax orthogonal and promax oblique rotations) were used for both purposes.Eleven items, which loaded heavily on the first factor of a two‐factor common factor solution with promax oblique rotation, were selected to create a computer familiarity score. This score was then used to classify TOEFL examinees into one of three familiarity groups: low, moderate, and high. A four‐factor common factor solution with promax oblique rotation was seen as providing the best accounting of the underlying factor structure of the complete questionnaire.
DEVELOPMENT, SELECTION AND VALIDATION OF A SET OF COGNITIVE AND LINGUISTIC ATTRIBUTES FOR THE SAT I VERBAL: SENTENCE COMPLETION SECTION1
Authors: Buck Gary , VanEssen Tom , Tatsuoka Kikumi , Kostin Irene , Lutz Donna , Phelps Matthew 
Published: August 2014

ABSTRACTThe purpose of this study is to use the rule space procedure to develop a general set of cognitive and linguistic attributes that account for performance on the Sentence Completion Section of the SAT I Verbal. The overall research strategy was to develop a set of cognitive and linguistic attributes on one form of the Sentence Completion Section, the experimental form, and then to examine the generalizability of these by applying the same set of attributes to a different form of the test, the generalizability form. Results produced a set 20 cognitive and linguistic attributes, along with 3 interaction attributes, that were able to classify 97% of test‐takers into their latent knowledge states on the experimental form; the same set of attributes were able to classify 98% of test takers into their latent knowledge states on the generalizability form. The analyses provide an indication of the important determinants of success on the SAT Sentence Completion task, and provide evidence that these are relatively stable across forms.
EFFECTS OF EXAMINEE CHOICE ON A TEST OF DIVERGENT THINKING
Authors: Powers Donald E. , Bennett Randy Elliot 
Published: August 2014

ABSTRACTUnder some circumstances, allowing examinees to choose which items to respond to may increase test validity. In this study, we explored how choice affected examinee performance and related characteristics for a test of the ability to generate hypotheses about a given situation. Four forms of this experimental test were randomly spiraled within a large sample of students taking the computer‐based GRE General Test, with two of the forms allowing examinee choice. Results suggested that items were, in fact, differentially attractive to gender, ethnic, undergraduate major, and ability groups, indicating that, in some form, choice was operating. Further, performance was significantly higher when a given item was chosen than when it was required. Effects of choice on subgroup performance were generally inconclusive. Finally, there was some indication that internal consistency was improved by the choice option.
PREPARING FOR THE SAT® I: REASONING TEST—AN UPDATE
Authors: Powers Donald E. 
Published: August 2014

ABSTRACTTo document the extent of special test preparation for the SAT® I: Reasoning Test, we surveyed a stratified random sample of some 6,700 students who registered to take the SAT I in 1995–96. A smaller companion survey sought information about special preparation programs from a stratified random sample of secondary schools whose students take the SAT I. The objectives were to:

determine the availability, and incidence of use, of a variety of programs and resources designed to prepare students to take the SAT I;
describe some of the salient features of these resources; and
estimate the amount of time (and money) that students spend on preparing for the test.
Though the surveys differed slightly from similar surveys conducted in 1986–87, they were designed generally to enable comparison with the results of the earlier surveys.The student survey found that prospective SAT I takers participate, to varying degrees, in a variety of preparation activities. Taking the PSAT/NMSQT for practice and using the test familiarization materials provided by the College Board are the most frequently used strategies. Other commercially available books and texts used in regular courses are also consulted relatively frequently. Engagement is much less frequent with such resources as test‐preparation software, special programs given either at school or outside school, or private tutoring. When particular programs or resources were available, cost was cited less often than some other factor as a reason for not using the resource. This was true for each of several resources, including coaching courses. About 12 percent of all students in the survey said they had attended preparation or coaching sessions outside school, where this minority of students paid, on average, about $400.On average, students currently spend a total of approximately 11 hours preparing for the SAT I, about the same amount (10 hours) that students reported in the 1986–87 survey. Currently, about 10 percent of all students report spending 54 hours or more preparing for the test (about the same as the 58 hours reported in 1986–87). Now, approximately 3 percent of test takers say they do not devote any time at all to SAT preparation (compared with 9 percent in 1986–87).The results of the school survey revealed that a slight majority (52 percent) of all secondary schools now offer programs to prepare students for the SAT I, about the same proportion (49 percent) as in 1986–87.
TEST TAKERS' JUDGMENTS ABOUT GRE WRITING TEST PROMPTS
Authors: Powers Donald E. , Fowles Mary E. 
Published: August 2014

ABSTRACTThis study gathered the judgments of various GRE test takers about a total of 78 essay prompts that were being considered for possible use in a GRE writing test. The objective was to determine the kinds of prompts and topics on which examinees feel they can write strong essays, as well as those that they perceive as more difficult. The study identified several features that underlie examinee assessments of essay prompts.Some of the study participants also wrote essays on a small subset of the prompts, and their opinions of the prompts were compared to the scores that GRE readers assigned to their essays. This comparison revealed only a weak and inconsistent relationship between writers' judgments about the prompts and their performance on those prompts.The report discusses the implications of the findings for the development of the GRE writing test and for advising GRE examinees on how to approach the test.
SOME PATHS TOWARD MAKING PRAXIS SCORES MORE USEFUL
Authors: Wainer Howard , Sheehan Kathleen , Wang Xiaohui 
Published: August 2014

ABSTRACTIn this study we describe an analytic method for aiding in the generation of subscales that characterize the deep structure of tests. In addition we also derive a procedure for estimating scores for these scales that are much more statistically stable than subscores computed solely from the items that are contained on that scale. These scores achieve their stability through augmentation with information from other related information on the test. These methods were used to complement each other on a data set obtained from a Praxis administration. We found that the deep structure of the test yielded ten subscales, and that because the test was essentially unidimensional ten subscores could be computed; all with very high reliability. This result was contrasted with the calculation of six traditional subscales based on surface features of the items. These subscales also yielded augmented subscores of high reliability.
CONSEQUENCES OF TEST INTERPRETATION AND USE: THE FUSION OF VALIDITY AND VALUES IN PSYCHOLOGICAL ASSESSMENT
Authors: Messick Samuel 
Published: August 2014

ABSTRACTThis paper addresses the role of social values in educational and psychological measurement, with special attention to the consequences of testing as validity evidence. Test use depends for its legitamicy on adherence to established principles of reliability, validity, and fairness. These principles are organized within an overarching conception of construct validity. Desired reliability depends on the meaning of the construct being measured while test fairness depends on comparable construct validity across groups or settings. In the name of fairness, value judgements need to be made throughout the testing process, as they do in the name of validity throughout the validation process.
DETERMINANTS OF PERSISTENCE OF GRADUATE STUDENTS IN PH.D. PROGRAMS
Authors: Attiyeh Gregory M. 
Published: August 2014

ABSTRACTThis study utilizes longitudinal data on individual students enrolled in doctoral programs from 1989 to 1993 to identify patterns and determinants of their persistence. For each of five disciplines (biochemistry, economics, English, mathematics, and mechanical engineering), a survival model is specified to explain student persistence through the first four years of graduate study. Variables hypothesized to explain persistence include student aptitude and achievement, financial support, and demographic characteristics, as well as characteristics of the programs in which students were enrolled. The statistical estimates reveal significant differences across disciplines, but several general results are apparent: a student with given characteristics is more likely to persist at more selective institutions; other things being equal, a student is more likely to persist if he or she has greater financial support, higher GRE Verbal or Quantitative scores, and a master's degree. Student citizenship, gender, ethnicity, and age are not consistently related to persistence across disciplines.