bluuebunny/crossref_metadata_2025_split huggingface dataset:
POTENTIAL IMPACT OF CONTEXT EFFECTS ON THE SCORING AND EQUATING OF THE MULTISTAGE GRE® REVISED GENERAL TEST
Authors: Davey Tim , Lee Yi‐Hsuan 
Published: August 2014

ABSTRACTBoth theoretical and practical considerations have led the revision of the Graduate Record Examinations® (GRE®) revised General Test, here called the rGRE, to adopt a multistage adaptive design that will be continuously or nearly continuously administered and that can provide immediate score reporting. These circumstances sharply constrain the available options for equating or linking together the thousands of unique test forms that will need to be administered each year. The only practical method is to pre‐equate each form through item response theory (IRT) methods. Unfortunately, doing so means that item position and context effects are both vitally important and potentially difficult to control. This study examines item position effects in a GRE context and attempts to assess the extent to which they may affect the rGRE. It was found that position effects are in fact present in GRE data and that they can be particularly troublesome under multistage testing. Pretest strategies are able to mitigate effects to some degree, but complete control will require careful attention be paid to the time limits and associated test configuration details in the design of the rGRE.
HIGH SCHOOLS THAT WORK: PROGRAM DESCRIPTION, LITERATURE REVIEW, AND RESEARCH FINDINGS
Authors: Young John W. , Cline Frederick , King Teresa C. , Jackson Avis D. , Timberlake Allison 
Published: August 2014

ABSTRACTInterest in setting high educational standards is at the forefront of many school reform programs in the United States. States' pursuit of high standards can be advanced through implementation of programs such as High Schools That Work (HSTW), which emphasizes high expectations and rigorous academics for all students, including Career and Technical Education students. High Schools That Work is the largest comprehensive school reform program for high schools in the United States, with over 1,000 schools in more than 30 states currently participating. This report describes the program, previous research, and recent research studies conducted by ETS to ensure validity for the HSTW program.
THE IMPACT OF CHANGES IN THE TOEFL® EXAM ON TEACHING IN A SAMPLE OF COUNTRIES IN EUROPE: PHASE 3, THE ROLE OF THE COURSEBOOK PHASE 4, DESCRIBING CHANGE
Authors: Wall Dianne , Horák Tania 
Published: August 2014

ABSTRACTThe aim of this report is to present the findings of the 3rd and 4th phases of a longitudinal study into the impact of changes in the TOEFL® exam on teaching in test preparation classrooms. Phase 1 (2003–2004) described the type of teaching taking place in 12 TOEFL preparation classrooms before the introduction of the new TOEFL. Phase 2 (2004–2006) followed 6 of the Phase 1 teachers as they became aware of the requirements of the new test and faced the challenges of designing new courses to help students to prepare for it effectively. The objectives of the Phase 3 study (2006–2007) were to analyze the coursebooks that 4 of these teachers were using as they continued to prepare students for the TOEFL computer‐based test (CBT) and began to plan courses for the TOEFL iBT®, and to find out how the teachers were using the coursebooks as they developed their courses and planned individual classroom sessions. The coursebook analysis revealed that the TOEFL iBT coursebooks differed considerably from the TOEFL CBT coursebooks in terms of content, with the inclusion of integrated writing tasks and independent and integrated speaking tasks and the absence of attention to grammatical form on its own. They did not differ greatly in terms of their general methodological approach, however. Information about how the teachers used their coursebooks was gathered via tracking questions and tasks eliciting self‐report data. The coursebooks seemed to be playing an important role in shaping the teachers' understanding of the requirements of the new TOEFL, and the teachers depended on them heavily as they developed their courses and planned their lessons. The objectives of Phase 4 (2007–2008) were to interview and observe 3 of the same teachers, to find out what their preparation classes looked like 1 year after the introduction of the TOEFL iBT in their countries. While some aspects of teaching seemed not to have changed greatly, considerable changes occurred in the amount of attention the teachers paid to the development of speaking and to the integration of different skills. The teachers differed from each other in how much they had changed their methods to develop their students' language skills. The report concludes with a discussion of the role the new test, new coursebooks, and other factors in the educational context played in shaping current practices in these TOEFL preparation classrooms.
EVALUATING ACADEMIC PROGRESS WITHOUT A VERTICAL SCALE
Authors: Yen Wendy M. , Lall Venessa F. , Monfils Lora 
Published: August 2014

ABSTRACTAlternatives to vertical scales are compared for measuring longitudinal academic growth and for producing school‐level growth measures. The alternatives examined were empirical cross‐grade regression, ordinary least squares and logistic regression, and multilevel models. The student data used for the comparisons were Arabic Grades 4 to 10 in Qatar, and results were examined in the scale score and performance level metrics. It is found that vertical scales and cross‐grade regressions can show different results at the individual student level, but at the school level, the different measures of growth were strongly correlated, particularly in the scale score metric. Differences between the methods appear more likely in the performance level metric than the scale score metric and for grade pairs with more extreme performance.
WHY THE MAJOR FIELD TEST IN BUSINESS DOES NOT REPORT SUBSCORES: RELIABILITY AND CONSTRUCT VALIDITY EVIDENCE
Authors: Ling Guangming 
Published: August 2014

ABSTRACTTo assess the value of individual students' subscores on the Major Field Test in Business (MFT Business), I examined the test's internal structure with factor analysis and structural equation model methods, and analyzed the subscore reliabilities using the augmented scores method. Analyses of the internal structure suggested that the MFT Business measures a unidimensional construct, which does not support reporting individual students' subscores. Augmented score analyses revealed that the observed total score could approximate the true subscore more accurately than the observed subscores, which do not support reporting individuals' subscores either. The results from these two approaches provided consistent evidence in support of the current practice of not reporting individuals' subscores on the MFT Business. The relationship between the two approaches is discussed, followed by a recommendation of an alternate method for future research.
THE USE OF QUALITY CONTROL AND DATA MINING TECHNIQUES FOR MONITORING SCALED SCORES: AN OVERVIEW
Authors: von Davier Alina A. 
Published: August 2014

ABSTRACTMaintaining comparability of test scores is a major challenge faced by testing programs that have almost continuous administrations. Among the potential problems are scale drift and rapid accumulation of errors. Many standard quality control techniques for testing programs, which can effectively detect and address scale drift for small numbers of administrations yearly, are not always adequate to detect changes in a complex, rapid flow of scores. To address this issue, Educational Testing Service has been conducting research into applying data mining and quality control tools from manufacturing, biology, and text analysis to scaled scores and other relevant assessment variables. Data mining tools can identify patterns in the data and quality control techniques can detect trends. This type of data analysis of scaled scores is relatively new, and this paper gives a brief overview of the theoretical and practical implications of the issues. More in‐depth analyses to refine the approaches for matching the type of data from educational assessments are needed.
THE LANGUAGE MUSESM SYSTEM: LINGUISTICALLY FOCUSED INSTRUCTIONAL AUTHORING
Authors: Burstein Jill , Shore Jane , Sabatini John , Moulder Brad , Holtzman Steven , Pedersen Ted 
Published: August 2014

ABSTRACTIn the United States, English learners (EL) often do not have the academic language proficiency, literacy skills, cultural background, and content knowledge necessary to succeed in kindergarten through 12th grade classrooms. This leads to large achievement gaps. Also, classroom texts are often riddled with linguistically unfamiliar elements, including: unfamiliar vocabulary, idioms, complex phrases or sentences, morphologically complex words, and unfamiliar discourse relations. Lack of familiarity with linguistic elements may result in gaps in a learner's comprehension of key content. It is not feasible for teachers to develop additional curriculum for the needs of all ELs in a classroom (who often come from culturally and linguistically diverse backgrounds.) However, it is feasible for teachers to develop instructional scaffolding (support) that helps ELs and can be used with all students. To develop effective scaffolding, teachers need to be able to reliably identify linguistic features in texts that could interfere with content comprehension. Language MuseSM is a web‐based application designed to support teachers in the identification of linguistic features in texts and in the development of linguistically focused instructional scaffolding. With regard to system itself, we will discuss (a) the system's motivation, (b) the system's linguistic feedback and instructional authoring components, which are driven by natural language processing, and (c) the system's infrastructure for capturing teachers' system use. In addition, we will also discuss preliminary pilot study findings with three teacher professional development programs. These findings suggest that exposure to Language Muse's linguistic feedback can support teachers in the development of lesson plan scaffolds designed to address language learning needs.
ETS RESEARCH ON COGNITIVE, PERSONALITY, AND SOCIAL PSYCHOLOGY: I
Authors: Stricker Lawrence J. 
Published: August 2014

ABSTRACTThis is an account of a portion of the research on cognitive, personality, and social psychology at ETS since the organization's inception. The topics in cognitive psychology are the structure of abilities; in personality psychology, response styles and social and emotional intelligence; and in social psychology, prosocial behavior and stereotype threat. Research on motivation is also covered.
EXAMINING EVIDENCE OF RELIABILITY, VALIDITY, AND FAIRNESS FOR THE SUCCESSNAVIGATOR™ ASSESSMENT
Authors: Markle Ross , Olivera‐Aguilar Margarita , Jackson Teresa , Noeth Richard , Robbins Steven 
Published: August 2014

ABSTRACTThe SuccessNavigator™ assessment is an online, 30 minute self‐assessment of psychosocial and study skills designed for students entering postsecondary education. In addition to providing feedback in areas such as classroom and study behaviors, commitment to educational goals, management of academic stress, and connection to social resources, it is also designed to predict a range of early academic outcomes. By indicating students' likely success, advisors, faculty, and staff can target their interactions with students to increase their likelihood of success. This report outlines evidence of reliability, validity, and fairness to demonstrate the appropriateness of SuccessNavigator for these purposes.
ASSESSING THE TEST INFORMATION FUNCTION AND DIFFERENTIAL ITEM FUNCTIONING FOR THE TOEFL JUNIOR® STANDARD TEST
Authors: Young John W. , Morgan Rick , Rybinski Paul , Steinberg Jonathan , Wang Yuan 
Published: August 2014

ABSTRACTThe TOEFL Junior® Standard Test is an assessment that measures the degree to which middle school‐aged students learning English as a second language have attained proficiency in the academic and social English skills representative of English‐medium instructional environments. The assessment measures skills in three areas: listening comprehension, language form and meaning, and reading comprehension. This study focused on two specific psychometric characteristics of the assessment: (a) For which segments of its score scales does the assessment provide sufficient information to support the use of the scores in placement decisions? (b) Do items exhibit significant (or C‐level) differential item functioning (DIF) when comparisons are made between test‐takers from different countries? For the first question, both of the forms we analyzed appear to provide sufficient information to support placement decisions across the majority of the score scale for all 3 sections of the assessment. For the second question, we found that a moderate number of items exhibited significant DIF, while the linguistic analyses conducted on the DIF results showed plausible construct‐relevant explanations for most of the findings.
EVALUATION OF A CONDITION‐ADAPTIVE TEST OF READING COMPREHENSION FOR STUDENTS WITH READING‐BASED LEARNING DISABILITIES
Authors: Stone Elizabeth , Cook Linda , Laitusis Cara 
Published: August 2014

ABSTRACTThis study presents secondary analyses on a 2–stage test of reading comprehension for students with reading‐based learning disabilities (RLD). The present paper describes student perceptions of the test and its features as well as analyses focused on the routing test and associated cut score. The routing test contained typical state assessment content and was designed to route the RLD participants into 1 of 2 accessible 2nd‐stage tests, 1 accommodated and 1 not accommodated, and we refer to this structure as condition‐adaptive. The accommodated test was presented with a read‐aloud accommodation, and an oral reading fluency subtest was also administered as part of that test to evaluate decoding skill that may have been masked by the read‐aloud accommodation. This design allowed for componentwise measurement of reading proficiency at the 2nd stage with the goal of creating an accessible tailored test that would provide more comparable accommodated and nonaccommodated test scores for possible use for accountability. Overall, student perceptions differed depending on which 2nd‐stage test was taken, but most students indicated that they tried as hard on this field test as they did on other tests, and many expressed a preference for the read‐aloud accommodation. We found that routing test length was more strongly correlated with 2nd‐stage test performance for the higher performing RLD students, that relatively similar routing decisions could be made using a shorter routing test made up of a smaller subset of the available passages, and that the percentage of students routed into each 2nd‐stage test was sensitive to perturbations of the cut score.
SYNTHESIZING FRAMEWORKS OF HIGHER EDUCATION STUDENT LEARNING OUTCOMES
Authors: Markle Ross , Brenneman Meghan , Jackson Teresa , Burrus Jeremy , Robbins Steven 
Published: August 2014

ABSTRACTThe public, education, and workforce sectors all have expressed interest regarding the key knowledge, skills, and abilities that enable individuals to be productive members of society. Although past efforts have attempted to create frameworks of student learning outcomes, the results have varied due to different perspectives and goals. Thus, the purpose of this paper was to gather and review relevant higher education frameworks, determine their commonalities and create domains, and identify the assessments that Educational Testing Service (ETS) has developed in each of the domains. After a thorough review of the relevant frameworks, seven key domains were identified: creativity, critical thinking, teamwork, communication, digital and information literacy, citizenship, and life skills. Also discussed were the issues of education versus work contextualization and the assumption of foundational quantitative reasoning and literacy skills informing these seven domains.
AUTOMATED SCORING OF MATHEMATICS TASKS IN THE COMMON CORE ERA: ENHANCEMENTS TO M‐RATER IN SUPPORT OF CBAL™ MATHEMATICS AND THE COMMON CORE ASSESSMENTS
Authors: Fife James H. 
Published: August 2014

ABSTRACTThe m‐rater scoring engine has been used successfully for the past several years to score CBAL™ mathematics tasks, for the most part without the need for human scoring. During this time, various improvements to m‐rater and its scoring keys have been implemented in response to specific CBAL needs. In 2012, with the general move toward creating innovative tasks for the Common Core assessment initiatives, in traditional testing programs, and with potential outside clients, and to further support CBAL, m‐rater was enhanced in ways that move ETS's automated scoring capabilities forward and that provide needed functionality for CBAL: (a) the numeric equivalence scoring engine was augmented with an open‐source computer algebra system; (b) a design flaw in the graph editor, affecting the way the editor graphs smooth functions, was corrected; (c) the graph editor was modified to give assessment specialists the option of requiring examinees to set the viewing window; and (d) m‐rater advisories were implemented in situations in which m‐rater either cannot score a response or may provide the wrong score. In addition, 2 m‐rater scoring models were built that presented some new challenges.
ETS CONTRIBUTIONS TO THE QUANTITATIVE ASSESSMENT OF ITEM, TEST, AND SCORE FAIRNESS
Authors: Dorans Neil J. 
Published: August 2014

ABSTRACTQuantitative fairness procedures have been developed and modified by ETS staff over the past several decades. ETS has been a leader in fairness assessment, and its efforts are reviewed in this report. The first section deals with differential prediction and differential validity procedures that examine whether test scores predict a criterion, such as performance in college, across different subgroups in a similar manner. The bulk of this report focuses on item level fairness, or differential item functioning, which is addressed in the various subsections of the second section. In the third section, I consider research pertaining to whether tests built to the same set of specifications produce scores that are related in the same way across different gender and ethnic groups. Limitations with the approaches reviewed here are discussed in the final section.
READING FOR UNDERSTANDING: HOW PERFORMANCE MODERATORS AND SCENARIOS IMPACT ASSESSMENT DESIGN
Authors: O'Reilly Tenaha , Sabatini John 
Published: August 2014

ABSTRACTThis paper represents the third installment of the Reading for Understanding (RfU) assessment framework. This paper builds upon the two prior installments (Sabatini & O'Reilly, 2013; Sabatini, O'Reilly, & Deane, 2013) by discussing the role of performance moderators in the test design and how scenario‐based assessment can be used as a tool for assessment delivery. Performance moderators are characteristics of students that impact reading performance but are not considered a part of the reading construct. These include (a) background and prior knowledge, (b) metacognitive and self‐regulatory strategies and behavior, (c) reading strategies, and (d) student motivation and engagement. In this paper, we argue there is added value in incorporating performance moderators into a reading test design. We characterize added value with respect to the validity of the claims derived from test scores, the interpretation of the test scores, and the relevance to instruction. As a second aim, we present a case for using scenario‐based assessments and how they can be used to integrate into the test design both the performance moderators as well as other features that make the assessment more instructionally relevant.
Noncontrast myocardial T1 mapping using cardiovascular magnetic resonance for iron overload
Authors: Sado Daniel M. , Maestrini Viviana , Piechnik Stefan K. , Banypersad Sanjay M. , White Steven K. , Flett Andrew S. , Robson Matthew D. , Neubauer Stefan , Ariti Cono , Arai Andrew , Kellman Peter , Yamamura Jin , Schoennagel Bjoern P. , Shah Farrukh , Davis Bernard , Trompeter Sara , Walker Malcolm , Porter John , Moon James C. 
Published: August 2014

PurposeTo explore the use and reproducibility of magnetic resonance‐derived myocardial T1 mapping in patients with iron overload.Materials and MethodsThe research received ethics committee approval and all patients provided written informed consent. This was a prospective study of 88 patients and 67 healthy volunteers. Thirty‐five patients underwent repeat scanning for reproducibility. T1 mapping used the shortened modified Look–Locker inversion recovery sequence (ShMOLLI) with a second, confirmatory MOLLI sequence in the reproducibility group. T2* was performed using a commercially available sequence. The analysis of the T2* interstudy reproducibility data was performed by two different research groups using two different methods.ResultsMyocardial T1 was lower in patients than healthy volunteers (836 ± 138 msec vs. 968 ± 32 msec, P < 0.0001). Myocardial T1 correlated with T2* (R = 0.79, P < 0.0001). No patient with low T2* had normal T1, but 32% (n = 28) of cases characterized by a normal T2* had low myocardial T1. Interstudy reproducibility of either T1 sequence was significantly better than T2*, with the results suggesting that the use of T1 in clinical trials could decrease potential sample sizes by 7‐fold.ConclusionMyocardial T1 mapping is an alternative method for cardiac iron quantification. T1 mapping shows the potential for improved detection of mild iron loading. The superior reproducibility of T1 has potential implications for clinical trial design and therapeutic monitoring. J. Magn. Reson. Imaging 2015;41:1505–1511. © 2014 Wiley Periodicals, Inc.
Luminescence chronology of late Quaternary moraines and Last Glacial Maximum equilibrium‐line altitude reconstruction from Parlung Zangbo Valley, south‐eastern Tibetan Plateau
Authors: CHEN RENRONG , ZHOU SHANGZHE , LAI ZHONGPING , OU XIANJIAO , CHEN RONG , DENG YINGBIN 
Published: August 2014

ABSTRACTOptically stimulated luminescence dating was applied to samples from glacial sediments in the Parlung Zangbo Valleys (Daba and Charao) of the south‐eastern Tibetan Plateau to better constrain the timing of Late Quaternary glaciation, which is thought to have been linked to the Indian monsoon. The single aliquot regenerative‐dose protocol on quartz was used for equivalent dose (De) determination using small aliquots. For samples from the Daba Valley, the comparison of ages using two grain size fractions, middle (38–63 µm) and coarse (90–250 µm) were conducted. The coarse grain ages in the Daba Valley (P3) and Charao Valley (P7) range from 16.4 to 26.0 ka, and are consistent with previously published cosmogenic radionuclide exposure ages in the Baiyu Valley (PB1). Our dating results reveal that these moraines were formed during the Last Glacial Maximum (LGM). We then reconstruct the past equilibrium‐line altitudes (ELAs) during the LGM for five glacial valleys in the study area, which range from 3286 to 4014 m with an average value of 3730 m, while the contemporary average ELA is 4655 m. After correction for terrain uplift (about 8 m since the LGM), the difference in the ELA values between the LGM and today is 917 m. According to these data, the estimated temperature during the LGM is 6.3 °C lower than present for the Parlung Zangbo River Valley.