bluuebunny/crossref_metadata_2025_split huggingface dataset:
TIGHT BUT LOOSE: SCALING UP TEACHER PROFESSIONAL DEVELOPMENT IN DIVERSE CONTEXTS
Authors: Wylie E. Caroline , Goe Laura , Leusner Dawn Marie , Lyon Christine J. , Tocic Cynthia , Wylie E. Caroline , Cleland Donna , Gannon Maureen , Ellsworth Judith , Heritage Margaret , Maher Jeff , Mardy Diane , Popham W. James , Snodgrass Donna , Taylor Gayle , Thompson Marnie , Wiliam Dylan 
Published: August 2014

ABSTRACTThis series of papers was originally presented as a symposium at the annual meetings of the American Educational Research Association (AERA) and the National Council on Measurement in Education (NCME) held between April 9, 2007, and April 13, 2007, in Chicago, IL. The authors represent school districts and departments of education across the United States, as well as researchers at Cleveland State University, Educational Testing Service (ETS), the Institute for Education in London, and the University of Wyoming at Laramie. All of the current ETS staff, along with Dylan Wiliam and Marnie Thompson, worked at ETS for several years on an iterative research and development program, out of which grew the Keeping Learning on Track® (KLT) program.These papers represents the thinking about the theory behind the KLT program, describes the range of contexts used to implement the program, and illustrates the inherent tensions between the desire to maintain fidelity to a theory of action and the need to demonstrate flexibility in order to accommodate local situations. Papers 2 through 6 present descriptions of five implementations in chronological order.
RESPONSE TO ASSESSMENT FEEDBACK: THE EFFECTS OF GRADES, PRAISE, AND SOURCE OF INFORMATION
Authors: Lipnevich Anastasiya A. , Smith Jeffrey K. 
Published: August 2014

ABSTRACTThis experiment involved college students (N= 464) working on an authentic learning task (writing an essay) under 3 conditions: no feedback, detailed feedback (perceived by participants to be provided by the course instructor), and detailed feedback (perceived by participants to be computer generated). Additionally, conditions were crossed with 2 factors of grade (receiving grade or not) and praise (receiving praise or not). Detailed feedback specific to individual work was found to be strongly related to student improvement in essay scores, with the influence of grades and praise more complex. Overall, detailed, descriptive feedback was found to be most effective when given alone, unaccompanied by grades or praise. The results have implications for theory and practice of assessment.
EFFECTS OF CALCULATOR AVAILABILITY ON GRE® QUANTITATIVE QUESTIONS
Authors: Bridgeman Brent , Cline Frederick , Levin Jutta 
Published: August 2014

ABSTRACTIn order to estimate the likely effects on item difficulty when a calculator becomes available on the quantitative section of the Graduate Record Examinations® (GRE®‐Q), 168 items (in six 28‐item forms) were administered either with or without access to an on‐screen four‐function calculator. The forms were administered as a special research section at the end of operational tests, with student volunteers randomly assigned to the calculator or no‐calculator groups. Usable data were obtained from 13,159 participants. Test development specialists were asked to rate which items they thought would become easier with a calculator. In general, the specialists were successful in identifying the items with relatively large calculator effects, though even these effects were quite small. An increase of only about four points in the percent correct should suffice for the items identified as likely to show calculator effects with no adjustment needed for the majority of the items. Introduction of a calculator should have little or no effect on gender and ethnic differences.
SAMPLE‐SIZE REQUIREMENTS FOR AUTOMATED ESSAY SCORING
Authors: Haberman Shelby J. , Sinharay Sandip 
Published: August 2014

ABSTRACTSample‐size requirements were considered for automated essay scoring in cases in which the automated essay score estimates the score provided by a human rater. Analysis considered both cases in which an essay prompt is examined in isolation and those in which a family of essay prompts is studied. In typical cases in which content analysis is not employed and in which the only object is to score individual essays to provide feedback to the examinee, it appears that several hundred essays are sufficient. For application of one model to a family of essays, fewer than 100 essays per prompt may often be adequate. The cumulative logit model was explored as a possible replacement of the linear regression model usually employed in automated essay scoring; the cumulative logit model performed somewhat better than did the linear regression model.
THE ASSESSMENT OF INFORMATION LITERACY: A CASE STUDY
Authors: Katz Irvin R. , Elliot Norbert , Attali Yigal , Scharf Davida , Powers Donald , Huey Heather , Joshi Kamal , Briller Vladimir 
Published: August 2014

ABSTRACTThis study presents an investigation of information literacy as defined by the ETS iSkills™ assessment and by the New Jersey Institute of Technology (NJIT) Information Literacy Scale (ILS). As two related but distinct measures, both iSkills and the ILS were used with undergraduate students at NJIT during the spring 2006 semester. Undergraduate students (n = 331), first through senior years, took the iSkills and submitted portfolios to be judged by the ILS. First‐year students took the Core iSkills assessment, which was designed to provide administrators and faculty with an understanding of the information and communication technology (ICT) literacy of a student doing entry‐level coursework (n = 155). Upper classmen took the more difficult Advanced iSkills assessment, appropriate for rising juniors (n = 176). Across all class levels, iSkills scores varied as expected. First‐year basic skills writing students performed at lower levels than first‐year students enrolled in traditional composition and cultural history courses; seniors performed at higher levels than sophomores and juniors. Because the NJIT ILS scores were designed to be curriculum sensitive, portfolio scores did not similarly follow grade levels. Analyses revealed weak correlations between portfolio and Core iSkills scores and moderate correlations between portfolio and Advanced iSkills scores. As two associated yet distinct systems of inquiry designed to explore undergraduate student performance, the ETS iSkills assessment and the NJIT ILS—taken both individually and together—yield important information regarding student performance.
COMPARING MULTIPLE‐GROUP MULTINOMIAL LOG‐LINEAR MODELS FOR MULTIDIMENSIONAL SKILL DISTRIBUTIONS IN THE GENERAL DIAGNOSTIC MODEL
Authors: Xu Xueli , von Davier Matthias 
Published: August 2014

ABSTRACTThe general diagnostic model (GDM) utilizes located latent classes for modeling a multidimensional proficiency variable. In this paper, the GDM is extended by employing a log‐linear model for multiple populations that assumes constraints on parameters across multiple groups. This constrained model is compared to log‐linear models that assume separate sets of parameters to fit the distribution of latent variables in each group of a multiple‐group model. Estimation of these constrained log‐linear models using iterative weighted least squares (IWLS) methods is outlined and an application to NAEP data exemplifies the differences between constrained and unconstrained models in the presence of larger numbers of group‐specific proficiency distributions.The use of log‐linear models for the latent skill space distributions using constraints across populations allows for efficient computations in models that include many proficiency distributions.
DEVELOPMENT OF APPROXIMATIONS TO POPULATION INVARIANCE INDICES
Authors: Liu Jinghua , Zhu Xiaowen 
Published: August 2014

ABSTRACTThe purpose of this paper is to explore methods to approximate population invariance without conducting multiple linkings for subpopulations. Under the single group or equivalent groups design, no linking needs to be performed for the parallel‐linear system linking functions. The unequated raw score information can be used as an approximation. For other linking functions that are nonparallel‐linear, linking only needs to be conducted for the total population. The difference of the standardized mean differences between each subpopulation and the total population across the old form and the new form can be used as an approximation of population invariance. Under the nonequivalent groups with anchor test design, conducting separate subpopulation linking and comparing them to the total population linking may still be the best way to estimate population invariance.
DIF DETECTION WITH SMALL SAMPLES: APPLYING SMOOTHING TECHNIQUES TO FREQUENCY DISTRIBUTIONS IN THE MANTEL‐HAENSZEL PROCEDURE
Authors: Yu Lei , Moses Tim , Puhan Gautam , Dorans Neil 
Published: August 2014

ABSTRACTAll differential item functioning (DIF) methods require at least a moderate sample size for effective DIF detection. Samples that are less than 200 pose a challenge for DIF analysis. Smoothing can improve upon the estimation of the population distribution by preserving major features of an observed frequency distribution while eliminating the noise brought about by irregular data points. This study applied smoothing techniques to frequency distributions and investigated the impact of smoothed data on the Mantel‐Haenszel (MH) DIF detection in small samples. Eight sample‐size combinations were randomly drawn from a real data set to make the study realistic and were replicated 80 times to produce stable results. The population DIF results were used as the criteria to evaluate sample estimates using root‐mean square difference (RMSD), bias analysis, and Type II error rate. Loglinear smoothing was found to provide slight to moderate improvements in MH DIF estimation with small samples.
APPLYING CONTENT SIMILARITY METRICS TO CORPUS DATA: DIFFERENCES BETWEEN NATIVE AND NON‐NATIVE SPEAKER RESPONSES TO A TOEFL® INTEGRATED WRITING PROMPT
Authors: Deane Paul , Gurevich Olga 
Published: August 2014

ABSTRACTFor many purposes, it is useful to collect a corpus of texts all produced to the same stimulus, whether to measure performance (as on a test) or to test hypotheses about population differences. This paper examines several methods for measuring similarities in phrasing and content and demonstrates that these methods can be used to identify population differences between native and non‐native speakers of English in a writing task.
LINKING WITH CONTINUOUS EXPONENTIAL FAMILIES: SINGLE‐GROUP DESIGNS
Authors: Haberman Shelby J. 
Published: August 2014

ABSTRACTContinuous exponential families are applied to linking forms via a single‐group design. In this application, a distribution from the continuous bivariate exponential family is used that has selected moments that match those of the bivariate distribution of scores on the forms to be linked. The selected continuous bivariate distribution then yields continuous univariate marginal distributions for the two forms. These marginal distributions then provide distribution functions and quantile functions that may be employed in equating. Normal approximations are obtained for the sample distributions of the conversion functions.
SUBSCORES AND VALIDITY
Authors: Haberman Shelby J. 
Published: August 2014

ABSTRACTIn educational testing, subscores may be provided based on a portion of the items from a larger test. One consideration in evaluation of such subscores is their ability to predict a criterion score. Two limitations on prediction exist. The first, which is well known, is that the coefficient of determination for linear prediction of the criterion score by the subscore cannot exceed the reliability coefficient of the subscore. The second limitation is on incremental validity. The coefficient of determination for linear prediction of the criterion score by both the total score and the subscore is at least as great as the coefficient of determination for linear prediction of the criterion score by only the total score. Incremental validity may be measured by the difference between these two coefficients of determination. This difference is no greater than the reliability of the residual from linear prediction of the subscore by the total score.
MONITORING AND FOSTERING LEARNING THROUGH GAMES AND EMBEDDED ASSESSMENTS
Authors: Shute Valerie J. , Ventura Matthew , Bauer Malcolm , Zapata‐Rivera Diego 
Published: August 2014

ABSTRACTTo reveal what is being learned during the gaming experience, this report proposes an approach for embedding assessments in immersive games, drawing on recent advances in assessment design. Key to this approach are formative assessment to guide instructional experiences and evidence‐centered design to systematically analyze the assessment argument (including the claims about the learner and the evidence that supports or fails to support those claims). Elements of this approach that have been applied in a nongame setting are shown and ideas are discussed for applying the approach to an existing immersive game setting. Finally, the report offers suggestions for extending and applying this approach for existing games and the design of new ones.
FIRST LANGUAGE OF EXAMINEES AND ITS RELATIONSHIP TO EQUATING
Authors: Liang Longjuan , Dorans Neil J. , Sinharay Sandip 
Published: August 2014

ABSTRACTTo ensure fairness, it is important to better understand the relationship of language proficiency with the standard procedures of psychometric analysis. This paper examines how equating results are affected by an increase in the proportion of examinees who report that English is not their first language, using the analysis samples for a large‐scale reading and mathematics test. The results indicate that equating is not affected much by an increase in the proportion of those examinees.
MODELS FOR EFFECTIVE AND SCALABLE TEACHER PROFESSIONAL DEVELOPMENT
Authors: Thompson Marnie , Goe Laura 
Published: August 2014

ABSTRACTThe minute‐to‐minute and day‐by‐day use of assessment for learning holds great potential to change the trajectory of student learning in U.S. classrooms. But without effective and scalable systems of professional development that actually lead teachers to adopt these practices, the utility and impact of assessment for learning will be quite limited. Drawing on learning theory, expertise research, and research on effective professional development, this paper presents the theoretical and empirical basis behind an evolving program of professional development that employs school‐embedded teacher learning communities as a central component, called Keeping Learning on Track®. Implementations of the Keeping Learning on Track program in two districts and a consortium of districts are described in detail.
TEACHER PROFESSIONAL DEVELOPMENT FOCUSED ON FORMATIVE ASSESSMENT: CHANGING TEACHERS, CHANGING SCHOOLS
Authors: Wylie E. Caroline , Lyon Christine J. , Goe Laura 
Published: August 2014

ABSTRACTThis paper outlines an approach to improving learning and teaching that combines two strong research bases: The research on formative assessment or assessment for learning provides information about what to change; research on teacher learning communities guides decisions about how to change. In this paper we describe the content and process for one model of teacher professional development. Two case studies for two school districts engaged in this process of teacher change are then presented along with descriptions of the observed effects on both the participating teachers and their larger school contexts.
FIRST LANGUAGE OF EXAMINEES AND ITS RELATIONSHIP TO DIFFERENTIAL ITEM FUNCTIONING
Authors: Sinharay Sandip , Dorans Neil J. , Liang Longjuan 
Published: August 2014

ABSTRACTTo ensure fairness, it is important to better understand the relationship of language proficiency to standard psychometric analysis procedures. This paper examines how results of differential item functioning (DIF) analysis are affected by an increase in the proportion of examinees who report that English is not their first language in the analysis sample of a large‐scale assessment. The results vary by group. In some combinations of focal/reference groups, the magnitude of DIF is not appreciably affected by whether the DIF is performed on examinees whose first language is not English. In other groups, the first language status matters. The results vary by type of test as well. In addition, the magnitude of DIF for some items is substantially affected by whether the DIF is performed on examinees whose first language is not English.
SELF‐CONSTRUCTS AND ANXIETY ACROSS CULTURES
Authors: Lee Jihyun 
Published: August 2014

ABSTRACTThis study examined the factorial structure of three related constructs, math self‐concept, math self‐efficacy, and math anxiety, across 41 countries. One factorial structure was achieved at both between‐ and within‐country levels. Within‐country variations of the self‐constructs were also noted in relation to math performance: Self‐concept showed the strongest correlations in some Western European countries, but it was self‐efficacy that demonstrated the strongest correlations among Asian and Eastern European countries. Findings also indicate that students in some Asian countries have low math self‐concept and math self‐efficacy and high math anxiety despite high math performance, while students in some Western European countries have high math performance and low levels of math anxiety.
USE OF GENERALIZED RESIDUALS TO EXAMINE GOODNESS OF FIT OF ITEM RESPONSE MODELS
Authors: Haberman Shelby J. 
Published: August 2014

ABSTRACTGeneralized residuals are a tool employed in the analysis of contingency tables to examine goodness of fit. They may be applied to item response models with little complication. Their use is illustrated with testing data from operational programs. Models considered include the Rasch model and the two‐parameter logistic model.
THE MISSING DATA ASSUMPTIONS OF THE NONEQUIVALENT GROUPS WITH ANCHOR TEST (NEAT) DESIGN AND THEIR IMPLICATIONS FOR TEST EQUATING
Authors: Sinharay Sandip , Holland Paul W. 
Published: August 2014

ABSTRACTThe nonequivalent groups with anchor test (NEAT) design involves missing data that are missing by design. Three popular equating methods that can be used with a NEAT design are the poststratification equating method, the chain equipercentile equating method, and the item‐response‐theory observed‐score‐equating method. These three methods each make different assumptions about the missing data in the NEAT design. Though studies have compared the equating performance of the three methods under the NEAT design, none has examined the missing data assumptions and their implications for such comparisons. The missing data assumptions can affect equating studies because it is necessary to fill in the missing data or their distribution in some way in order to have a true, or criterion, equating function to compare the accuracy and bias of the different methods. If the missing data or their distribution are filled in using missing data assumptions that correspond to a given method, that may favor that method in any comparison with the others. This paper first describes the missing data assumptions of the three equating methods and then performs a fair comparison of the 3 methods using data from 3 different operational tests. For each data set, we examine how the 3 equating methods perform when the missing data satisfy the assumptions made by only 1 of these equating methods. The chain equating method is somewhat more satisfactory overall than the other methods in our fair comparison of the methods; hence, we recommend that equating practitioners seriously consider the chain equating method when using the NEAT design. In addition, we conclude that the results from the different equating methods will tend to agree with each other when proper equating conditions are in place. Moreover, to uncover problems that might not reveal themselves otherwise, it is important for operational testing programs to apply multiple equating methods and study the differences among their results.
THE TOEIC® SPEAKING AND WRITING TESTS: RELATIONS TO TEST‐TAKER PERCEPTIONS OF PROFICIENCY IN ENGLISH
Authors: Powers Donald E. , Kim Hae‐Jin , Yu Feng , Weng Vincent Z. , VanWinkle Waverely 
Published: August 2014

ABSTRACTTo facilitate the interpretation of test scores from the new TOEIC® speaking and writing tests as measures of English‐language proficiency, we administered a self‐assessment inventory to TOEIC examinees in Japan and Korea, to gather their perceptions of their ability to perform a variety of everyday English‐language tasks. TOEIC scores related relatively strongly to test taker self‐reports for both speaking and writing tasks. The results were extraordinarily consistent, with examinees at each higher TOEIC score level being more likely to report that they could successfully accomplish each of the everyday language tasks in English. The pattern of correlations also suggested modest discriminant validity of the new TOEIC speaking and writing measures, suggesting that both measures contribute uniquely to the assessment of English‐language proficiency.
UNTYING THE KNOT: REVIEW OF RESEARCH ON THE INFLUENCE OF PARENTAL INVOLVEMENT ON STUDENTS' ACADEMIC ACHIEVEMENT AT THE SECONDARY SCHOOL LEVEL
Authors: Shute Valerie J. , Hansen Eric G. , Underwood Jody 
Published: August 2014

ABSTRACTThe idea that parental involvement engenders students' academic achievement is so intuitively appealing that society in general, and educators in particular, have considered parental involvement an important ingredient for the remedy of many ills in education today. In the 1980s and early 1990s, a number of studies were published that suggested the importance of parental involvement. In response, in the mid‐1990s, the popular press, policy makers, and school administrators promoted parental involvement. Legislation was enacted, such as the Goals 2000: Educate America Act and the reauthorized Elementary and Secondary Education Act (ESEA). However, in the late 1990s, research began appearing that questioned the preeminence of the parental role. A landmark in this debate was Judith Harris's 1998 book, The Nurture Assumption, in which Harris argued that parents have little or no influence over the long‐term development of their children's personality or academic success. Other studies have followed, suggesting that parental involvement may not matter as much or in the ways that had been believed. This paper reviews the research literature on the influence of parental involvement on academic achievement, with particular attention to the secondary school level.
CONSTRUCTION OF CHAINED TRUE SCORE EQUIPERCENTILE EQUATINGS UNDER THE KERNEL EQUATING (KE) FRAMEWORK AND THEIR RELATIONSHIP TO LEVINE TRUE SCORE EQUATING
Authors: Chen Haiwen (Henry) , Holland Paul 
Published: August 2014

ABSTRACTIn this paper, we develop a new chained equipercentile equating procedure for the nonequivalent groups with anchor test (NEAT) design under the assumptions of the classical test theory model. This new equating is named chained true score equipercentile equating. We also apply the kernel equating framework to this equating design, resulting in a family of chained true score equipercentile equating functions, which include the Levine true score equating model as a special case.