ethanporcaro/eric-fulltext-meta huggingface dataset:
Measurement of New Attributes for Chile's Admissions System to Higher Education. Research Report. ETS RR-11-18
Author: Santelices, Maria Veronica
Date: 2011-04-00
Type: Numerical/Quantitative Data
Subjects: Foreign Countries,College Entrance Examinations,Test Construction,Test Validity,Test Reliability,Scoring,Critical Thinking,Standardized Tests,Factor Analysis,Correlation,High School Seniors,College Freshmen,Teacher Attitudes,Questionnaires

This paper presents the development and initial validation of new measures of critical thinking and noncognitive attributes that were designed to supplement existing standardized tests used in the admissions system for higher education in Chile. The importance of various facets of this process, including the establishment of technical rigor and political support of instrument development, piloting, and implementation is also emphasized because the results from this pilot study could affect the admissions process for some of the most prestigious and competitive universities in the country. The new instruments were pilot tested on a group of high school seniors (n = 1,568) and first-year college students (n = 1,443) during 2008. The sample included students from 20 secondary schools and 4 universities. Initial results were promising and showed that the measures were effectively assessing attributes not currently considered in the Chilean higher education admissions process. In addition, performance on the new instruments was found to be less correlated with students' sociodemographic characteristics than were measures currently used for undergraduate admissions. Appended are: (1) Factor Analysis of Critical Thinking Essay; (2) Factor Analysis of the Personal Reflection Instrument; (3) Pearson Partial Correlation Coefficients Among Standardized Questionnaires Controlling for Social Desirability; (4) Factor Analysis of All Likert Scale Self-Ratings' Items Combined; and (5) Factor Analysis of Teacher Survey. (Contains 19 tables, 4 figures and 20 notes.)
Repeater Effects on Score Equating for a Graduate Admissions Exam. Research Report. ETS RR-11-17
Author: Yang, Wen-Ling
Date: 2011-04-00
Type: Reports - Research
Subjects: Equated Scores,College Entrance Examinations,Graduate Study,Differences

Using self-reported but empirically verified repeater groups, we analyzed vast amounts of real test data across a wide range of administrations from a graduate admissions examination that was administered in a non-English language to investigate repeater effects on score equating using the nonequivalent groups with anchor test (NEAT) design. Both linear and nonlinear equating models were considered in deriving the equating functions for various study groups. We evaluated scaled score differences between equating in the total group, the repeater group, and the first-timer group using statistics of simple differences and subpopulation invariance measures developed and used widely in the last 10 years. Standard errors of statistics summarizing scaled score differences were estimated using a simulation approach to provide statistical criteria for evaluating the significance of equating differences. In addition, we used scaled score differences that were critical to admissions screening as criteria for evaluating the practical significance of equating differences. To put the investigation of repeater effects in proper perspective, we analyzed the repeater data for an in-depth understanding of repeater performance trends. Overall, we found no significant effects of repeater performance on score equating for the exam being studied. Although many of the equating differences were practically significant, most of the practically significant differences were not statistically significant. However, further research with larger repeater samples is recommended to help explain the practical significance of equating differences consistently observed in this study for the repeater group. Potential problems associated with small repeater study sample sizes, issues of the practical criterion for evaluating the significance of equating differences, and study limitations are also discussed. (Contains 8 tables, 25 figures and 8 notes.)
The Evaluation of Bias of the Weighted Random Effects Model Estimators. Research Report. ETS RR-11-13
Author: Jia, Yue
Date: 2011-04-00
Type: Reports - Research
Subjects: Computation,Statistical Bias,Sampling,Statistical Analysis,Probability,Correlation,National Competency Tests,Reading Tests,Grade 4,Sample Size

Estimation of parameters of random effects models from samples collected via complex multistage designs is considered. One way to reduce estimation bias due to unequal probabilities of selection is to incorporate sampling weights. Many researchers have been proposed various weighting methods (Korn, &amp; Graubard, 2003; Pfeffermann, Skinner, Holmes, Goldstein, &amp; Rasbash, 1998) in estimating the parameters of hierarchical models, including random effects models as a special case. In this paper, the bias of the weighted analysis of variance (ANOVA) estimators of the variance components for a two-level, one-way random effects model is evaluated. For these estimators, analytic bias expressions are first developed, the expressions are then used to examine the impact of sample size, intraclass correlation coefficient (ICC), and the sampling design on the bias of the estimators. In addition, two-stage sampling designs are considered, with a general probability design at the first stage (Level 2) and simple random sampling without replacement (SRS) at the second stage (Level 1). The study shows that first-order weighted variance component estimators perform well when for moderate cluster sizes and ICC values. However, noticeable estimation bias can be found with this weighting method for small cluster sizes (less than 20), particularly when ICC is small (less than 0.2). In such scenarios, scaled first-order weighted estimators can be an alternative. This paper is discussed in the context of National Assessment of Educational Progress (NAEP) 2003 4th Grade Reading National and State Assessment data, with Level 1 being the student level and Level 2 being the school level. Bias Expression of First-Order Weighted Estimators is appended. (Contains 4 tables and 4 figures.)
High Schools That Work: Program Description, Literature Review, and Research Findings. Research Report. ETS RR-11-33
Author: Young, John W.
Date: 2011-08-00
Type: Reports - Research
Subjects: High Schools,Program Descriptions,Vocational Education,Literature Reviews,Research Reports,Comprehensive Programs,Effective Schools Research,Educational Practices,Educational Change,Outcomes of Education,Program Implementation,Correlation,Educational Improvement,Improvement Programs,Educational Research

Interest in setting high educational standards is at the forefront of many school reform programs in the United States. States' pursuit of high standards can be advanced through implementation of programs such as High Schools That Work (&quot;HSTW&quot;), which emphasizes high expectations and rigorous academics for all students, including Career and Technical Education students. High Schools That Work is the largest comprehensive school reform program for high schools in the United States, with over 1,000 schools in more than 30 states currently participating. This report describes the program, previous research, and recent research studies conducted by ETS to ensure validity for the &quot;HSTW&quot; program. Appended are: (1) Score Changes by School Quartiles and ANOVA Results for All Indices; and (2) Changes in &quot;HSTW&quot; Indices from 2008 to 2010. (Contains 32 tables.)
Applying Knowledge of Quantitative Design and Analysis
Author: Baskas, Richard S.
Date: 2011-08-28
Type: Reports - Research
Subjects: Statistical Analysis,Research Design,Journal Articles,Research Methodology,Definitions,Measurement,Validity,Reliability,Sampling,Ethics,Comparative Analysis

This study compared and contrasted two quantitative scholarly articles in relation to their  research designs. Their designs were analyzed by the comparison of research references and research specific vocabulary to describe how various research methods were used. When researching and analyzing quantitative scholarly articles, it is imperative to understand how research designs can be the same, but their methods of analysis can differ depending on the study, and how vocabulary describes the research being conducted. The course instructor provided two quantitative articles which were analyzed by research references provided by the student. A literature review of the two articles and that of the research references were conducted to provide a more in-depth explanation of the research designs. Analysis revealed that even though both articles shared the same research design, various analytical techniques that are used can produce different perspectives of the results.
The Use of Narrative: Gender Differences and Implications for Motivation and Learning in a Math Game. CRESST Report 804
Author: Bittick, Sarah Joy
Date: 2011-08-00
Type: Reports - Research
Subjects: Educational Games,Females,Student Experience,Gender Differences,Student Motivation,Personal Narratives,Video Games,Disproportionate Representation,Masculinity,Femininity,Comparative Analysis,Remedial Programs,Mathematics Instruction,Algebra,Student Attitudes,Secondary School Mathematics,High Schools

The use of a narrative in educational contexts has been found to increase learners' experience of flow or absorption in a task. This increased experience of flow can in turn result in increased retention and learning outcomes. However, narrative can also be polarizing particularly in the male-dominated realm of video game play due to gender stereotyping or underrepresentation of females in games. Considering studies on narrative, flow, and gender differences in video games, it is expected that masculine and feminine narratives used in an educational game would result in differences in learning for the two genders. This study implemented two narratives into a rational numbers math video game to compare engagement and learning outcomes between genders and in comparison to a non-narrative version of the game. One hundred seventy-seven students enrolled in a remedial math course, pre-algebra, or Algebra 1 participated in the main study; this included 85 males, 80 females, and 12 students who did not report their gender. Results indicate that narrative did increase student experience of flow and positive perceptions of the game, especially when males were matched with the masculine version of the game. Increased learning outcomes took place only when students were placed in the masculine narrative and when males were matched to the masculine narrative. Appended are: (1) Narrative Screens; (2) Rubric for Open-Ended Narrative Question; (3) Background Information Survey; (4) Video Game Experience Survey; and (5) Study Missing Data Analyses. (Contains 16 tables and 5 figures.) [This paper was supported under the Educational Research and Development Centers Program.]
Teaching English as a Foreign Language from a New Literacy Perspective: A Guide for Egyptian EFL Student Teachers (Chapter One: Setting the Scene)
Author: Abdallah, Mahmoud M. S.
Date: 2011-00-00
Type: Guides - Classroom - Teacher
Subjects: Foreign Countries,English (Second Language),Second Language Instruction,Preservice Teacher Education,Student Teachers,Information Technology,Telecommunications,Computer Mediated Communication,Technological Advancement,Social Networks,Educational Change

The article presents Chapter One of my recent the book entitled, &quot;Teaching English as a Foreign Language from a New Literacy Perspective: A Guide for Egyptian EFL Student Teachers.&quot; The chapter deals with the new reality of Teaching English as a Foreign in this ICT-dominated age in which the Internet has been playing a vital role in changing the nature of the English language, and subsequently the methods and techniques that should be used for teaching/learning it. It presents some recent approaches to literacy that have recently come to the fore to encompass the new literacy practices imposed by some new technological innovations, especially the Internet. Those practices extend beyond the traditional print-based practices that are still dominant in the Egyptian schools and English-language teacher education programmes. Moreover, the chapter presents a comprehensive list of those Web-based new literacies that pre-service English language teachers need in this age, especially in Egypt, so that they become able to cope with the new literacy challenges imposed by the Web as well as the informal, out-of-school literacy practices they have been increasingly involved in. This list is the main contribution of my recently finished PhD study conducted at the Graduate School of Education, College of Social Sciences and International Studies, University of Exeter, UK (Main source: Abdallah, M. M. S. (2011). &quot;Web-based New Literacies and EFL Curriculum: A Design Study for Expanding EFL Student Teachers' Language-Related Literacy Practices in an Egyptian Pre-service Teacher Education Programme.&quot; PhD Thesis. Graduate School of Education, College of Social Sciences and International Studies, University of Exeter, UK. Available at: http://hdl.handle.net/10036/3202. (Contains 2 tables.)
Can Smoothing Help When Equating with Unrepresentative Small Samples? Research Report. ETS RR-11-09
Author: Puhan, Gautam
Date: 2011-03-00
Type: Reports - Research
Subjects: Equated Scores,Data Analysis,Accuracy,Sample Size,Tests,Statistical Bias,Error of Measurement

The study evaluated the effectiveness of log-linear presmoothing (Holland &amp; Thayer, 1987) on the accuracy of small sample chained equipercentile equatings under two conditions (i.e., using small samples that differed randomly in ability from the target population &quot;versus&quot; using small samples that were distinctly different from the target population). Results showed that equating with small samples (e.g., N less than 50) using either raw or smoothed score distributions can result in a substantial amount of random equating error (although smoothing reduced random equating error). Even with samples sizes of 100, the random equating error was quite large (greater than the difference that matters or DTM) for almost all score points. Moreover, when the small samples were unrepresentative of the target population, which is quite likely for small samples, the amount of equating bias (in addition to random equating error) was considerably large for both the raw and smoothed equatings. It was concluded that although presmoothing helped reduce random equating error, it is unlikely to reduce equating bias caused by using an unrepresentative sample. Other alternatives to the small sample equating problem that focus more on improving data collection than on improving existing equating methods are discussed. Appended are: (1) Score Distribution of Full Sample; and (2) Conditional Standard Errors of Equating. (Contains 3 tables, 7 figures and 2 notes.)
Performance and Passing Rate Differences of African American and White Prospective Teachers on Praxis[TM] Examinations: A Joint Project of the National Education Association (NEA) and Educational Testing Service (ETS). Research Report. ETS RR-11-08
Author: Nettles, Michael T.
Date: 2011-03-00
Type: Reports - Research
Subjects: African American Teachers,Educational Testing,Teacher Supply and Demand,Standard Setting,Program Effectiveness,Bachelors Degrees,Educational Attainment,Preservice Teachers,Whites,African Americans,Racial Differences,Comparative Analysis,Academic Achievement,Teacher Competencies,Teacher Certification,Pedagogical Content Knowledge,Teaching Skills,Interviews,Achievement Gap,Mathematics Instruction,Social Studies,Language Arts,English Instruction,Majors (Students),Minority Group Teachers,Teaching Experience,Masters Degrees,Socioeconomic Status,Associate Degrees,College Choice,Science Instruction,Regression (Statistics),American Indians,Hispanic Americans

This report focuses on two aspects of teacher supply and quality: (a) the under-representation of racial/ethnic minorities, especially African Americans in the teaching pool, and (b) teacher candidates' performance on licensure assessments, including general skills tests in reading, writing, and mathematics (known as Praxis I[R]) and selected tests covering areas of content and pedagogy (known as Praxis II[R]). The report presents trends in the overall and minority teaching workforce, utilizes some of the available Praxis[TM] program data, combined with interviews of faculty and students involved in college and university teacher preparation programs, to address supply, achievement, and performance gaps between prospective minority and White teachers. African American candidates are the focal group in this report, in order to gauge the effect that Praxis has on the supply of prospective African American teachers and to generate ideas for closing gaps. Other minority Praxis test-taking populations in the selected states are not included because these are not large enough to produce sufficient data from which to make valid inferences. Appended are: (1) Primer on Standard Setting; (2) Description of Selected Background Variables and Classifications Used in the Quantitative Analyses of Praxis I and Selected Praxis II Tests; (3) Statistical Formulas; (4) Summary of Regression Analyses Using Praxis I and Selected Praxis II Tests and Background Information Questionnaire (BIQ) Variables; (5) Script Inviting Campuses to Participate in the Joint Research Project of the National Education Association (NEA) and Educational Testing Service (ETS) to Understand Achievement Gaps in Teacher Licensure and Certification Tests; (6) National Education Association (NEA)/Educational Testing Service (ETS) Interview Guides; and (7) Lloyd Bond Findings and Recommendations. (Contains 31 tables and 11 notes.)
An Examination of the Link between Rater Calibration Performance and Subsequent Scoring Accuracy in Graduate Record Examinations[R] (GRE[R]) Writing. Research Report. ETS RR-11-03
Author: Ricker-Pedley, Kathryn L.
Date: 2011-02-00
Type: Reports - Research
Subjects: Scoring,Accuracy,College Entrance Examinations,Writing Tests,Reliability

A pseudo-experimental study was conducted to examine the link between rater accuracy  calibration performances and subsequent accuracy during operational scoring. The study asked 45 raters to score a 75-response calibration set and then a 100-response (operational) set of responses from a retired Graduate Record Examinations[R] (GRE[R]) writing prompt that had been vetted to determine correct scores prior to the study. The study found a positive relationship between calibration accuracy and scoring accuracy. Results suggest that a longer calibration test is more reliable and has a stronger correlation with operational scoring exact agreement; however, even tests as short as 10 responses had a reasonable correlation with operational accuracy, and the shorter test classified raters accurately 87% of the time when compared to the 75-response criterion. The additional requirement of no discrepant scores during calibration had better correlation with exact agreement, but much poorer classification accuracy in the shorter tests. This research suggests that calibration is useful for screening raters in its current form, but that further investigation of passing standards and of less controlled operational scoring conditions is required. (Contains 16 tables, 3 figures and 2 notes.)
Equating Subscores Using Total Scaled Scores as an Anchor. Research Report. ETS RR-11-07
Author: Puhan, Gautam
Date: 2011-03-00
Type: Reports - Research
Subjects: Equated Scores,Scaling,Raw Scores,Methods,Comparative Analysis

Because the demand for subscores is ever increasing, this study examined two different  approaches for equating subscores: (a) equating a subscore on the new form to the same subscore in the old form using internal common items as the anchor to conduct the equating, and (b) equating a subscore on the new form to the same subscore in the old form using equated total scores as the anchor to conduct the equating. Equated total scores can be used as an anchor to equate the subscores because the total equated scores are comparable across both the new and the old forms. Data from 2 tests (Tests X and Y) were used to conduct the study, and results showed that when the number of internal common items was large (approximately 50% of the total subscore), then using common items to equate the subscores was preferable. However, when the number of common items was small (approximately 25% of the total subscore, which is common practice), then using total scaled scores (TSS) to equate the subscores was preferable. Using raw subscores (not equating) resulted in a considerable amount of bias for both tests. Indices to measure equating variability and bias are appended. (Contains 4 tables, 5 figures and 4 notes.)
Equating of Subscores and Weighted Averages under the NEAT Design. Research Report. ETS RR-11-01
Author: Sinharay, Sandip
Date: 2011-01-00
Type: Numerical/Quantitative Data
Subjects: Equated Scores,Weighted Scores,Tests,Statistical Analysis

Recently, the literature has seen increasing interest in subscores for their potential diagnostic values; for example, one study suggested the report of weighted averages of a subscore and the total score, whereas others showed, for various operational and simulated data sets, that weighted averages, as compared to subscores, lead to more accurate diagnostic information. To report weighted averages, the averages should be comparable across different test forms; that is, the averages should be equated. This report discusses how to equate weighted averages. Results from operational and simulated data sets demonstrate the small error found when equating weighted averages. Description of the Methodology of Haberman (2008b) is appended. (Contains 17 tables, 12 figures and 5 notes.)
Principles and Practices of Test Score Equating. Research Report. ETS RR-10-29
Author: Dorans, Neil J.
Date: 2010-12-00
Type: Reports - Descriptive
Subjects: Equated Scores,Methods,Data Collection,Data Processing,Best Practices

Score equating is essential for any testing program that continually produces new editions of a test and for which the expectation is that scores from these editions have the same meaning over time. Particularly in testing programs that help make high-stakes decisions, it is extremely important that test equating be done carefully and accurately. An error in the equating function or score conversion can affect the scores for all examinees, which is both a fairness and a validity concern. Because the reported score is so visible, the credibility of a testing organization hinges on activities associated with producing, equating, and reporting scores. This paper addresses the practical implications of score equating by describing aspects of equating and best practices associated with the equating process. (Contains 4 tables and 4 notes.)