ethanporcaro/eric-fulltext-meta huggingface dataset:
Cluster Randomized Trial of a Large-Scale Education Initiative in the Democratic Republic of Congo: Pilot Year Impacts on Teacher Development
Author: Wolf, Sharon
Date: 2014-00-00
Type: Reports - Research
Subjects: Foreign Countries,Randomized Controlled Trials,Developing Nations,Intervention,Teacher Motivation,Job Satisfaction,Professional Development,Educational Practices,Low Income,War,Teacher Burnout,Pilot Projects,Teacher Attitudes,Elementary School Teachers

A wealth of research, primarily in high income countries, has accumulated in recent years evaluating teacher effectiveness and the processes through which teachers' performance and job satisfaction can be improved (e.g., Pianta, Mashburn, Downer, Hamre &amp; Justice, 2008; Ross, 1992; 1995). Much less is known about how these processes operate for teachers in low-income countries, where schools may be located in high conflict areas with low resources that can pose strains on teachers, students and their interactions. In the past several years, there has been a growing interest in rigorous evaluation research in developing countries to help inform policy and practice. This paper applies rigorous methodology to consider the question of whether and how an intervention can improve teachers' motivation, job satisfaction and professional development in the Democratic Republic of the Congo (DRC). The current study is part of OPEQ (Opportunities for Equitable Access to High Quality Basic Education), a 5-year project aimed at improving teachers' motivation and performance, and elementary school children's numeracy, literacy and socio-emotional wellbeing in 3 provinces of the DRC (i.e., Katanga, North Kivu and South Kivu), through curriculum development, teacher training and community mobilization. The main purpose of the current paper is to better understand teacher practices and classroom processes in low-income, war-afflicted countries such as the DRC. It presents baseline information on teachers' criticism of school, burnout, motivation and job satisfaction; examines how these processes can be affected by an intervention that targets teachers' and students' social emotional needs, and provides instructional support to teachers in math and reading; and presents preliminary impacts on teachers' burnout, motivation and job satisfaction after one pilot year of program implementation. An appendix contains a table and figure.
Assessing Teacher Effectiveness through Dual-Rater Classroom Observations: Researchers and District Staff Partnering to Create Calibrated Performance Evaluations
Author: Manzeske, David P.
Date: 2014-00-00
Type: Reports - Research
Subjects: Teacher Effectiveness,Teacher Evaluation,Interrater Reliability,Classroom Observation Techniques,Performance Based Assessment,Elementary School Teachers,Middle School Teachers,Pilot Projects,Scoring Rubrics,Federal Legislation,Educational Legislation,Elementary Secondary Education,Federal Aid,Item Response Theory,School Districts

Federal policies (e.g., 2002 reauthorization of the Elementary and Secondary Education Act [ESEA] and the American Recovery and Reinvestment Act) posit that teacher quality is a potential leverage point for improving student achievement (U.S. Department of Education, 2010). Moreover, in the Race to the Top competition, teacher effectiveness must be based, in part, on teacher performance measured by classroom observations. This has driven many districts to adopt teacher classroom observation rubrics to meet the Race to the Top requirement. Without clear guidance on how to rate teachers and without proper calibration activities, scores on these rubrics can become upwardly biased, leading to an inability to distinguish among teachers at different performance levels (see, for example, Weisberg, Sexton, Mulhern, &amp; Keeling, 2009). When a rubric is used inconsistently, teachers may not receive useful feedback, and the rubric could lack teacher buy-in, resulting in views that the evaluation does not provide credible information. In partnership with district central office staff, a study was conducted in a set of 20 elementary and middle schools in the western United States to investigate the use of a classroom observation rubric within the context of a district's pilot teacher performance evaluation system. The district wanted to know whether peer raters would use the observation rubric differently compared with principals, whether interrater reliability differed by rater type, and whether district-selected raters, on average, were more or less lenient than principals in rating other teachers. Supplemental tables and figures are appended.
Financial Aid Professionals at Work in 1999-2000: Results from the 2001 Survey of Undergraduate Financial Aid Policies, Practices, and Procedures
Author: College Entrance Examination Board
Date: 2002-00-00
Type: Tests/Questionnaires
Subjects: Student Financial Aid,Student Financial Aid Officers,Higher Education,Eligibility,Debt (Financial),Demography,Salaries,Administrator Characteristics,Surveys

This report first describes the aid application procedures aid administrators used to determine award eligibility, and the use of professional judgment to adjust students' aid application data. Next, the report describes the number of students who received financial aid, the funding and distribution of institutional scholarships, and student loan borrowing and cumulative debt. The study then shows the percentage of aid applicants who had financial need and the proportion of need met by total financial aid. The final section of the report describes the salary by type of institution, and technology used by aid offices to process applications. The following are appended: (1) Undergraduate Admissions Selectivity Definitions; and (2) 2001 Survey of Undergraduate Financial Aid Policies, Practices, and Procedures (SUFAPPP).
Comparison of Student-Level and School-Level Data in a National Impact Evaluation
Author: Velez, Melissa
Date: 2014-00-00
Type: Reports - Research
Subjects: National Surveys,Program Effectiveness,Program Evaluation,Comparative Analysis,Student Records,Intervention,Access to Information,Middle Schools,Case Studies,Information Utilization,Multiple Regression Analysis

One of the primary obstacles facing education researchers today is the struggle to obtain student-level data from states, districts, and schools. Researchers typically face one of two scenarios; they must either (1) work with contractors hired by the state or district to handle data requests who can be prohibitively expensive or (2) invest significant amounts of time making data requests from overworked school district staff that can take a year or longer. Due to budget cuts and waning resources, district staff juggle many competing demands and typically prioritize district-level needs above those of external evaluators. It is becoming increasingly clear the research community must look for methodological alternatives that allow the continuation of rigorous research about what works in education. This study explores the possibility of using publicly available school-level data instead of student-level data to conduct an impact analysis of a national education intervention. The authors use a national evaluation of an Expanded Learning Time (ELT) initiative in middle schools across the country as a case-study for these methods. One table is appended.
Consequences of Not Accounting for One-Group Clustering in Meta-Analysis
Author: Citkowicz, Martyna
Date: 2014-00-00
Type: Reports - Research
Subjects: Meta Analysis,Accountability,Cluster Grouping,Effect Size,Research Design,Error of Measurement,Bias,Experimenter Characteristics,Control Groups,Experimental Groups,Research Methodology,Research Problems,Robustness (Statistics)

Meta-analyses are syntheses of effect-size estimates obtained from a collection of studies to summarize a particular field or topic (Hedges, 1992; Lipsey &amp; Wilson, 2001). These reviews are used to integrate knowledge that can inform both scientific inquiry and public policy, therefore it is important to ensure that the estimates of the effect sizes are both robust and free of bias. One such scenario under which bias can occur is from a failure to account for clustering. Clustering occurs when, for example, schools are assigned to conditions and students are grouped together within each school. Occasionally, study designs are such that clustering exists in one condition but not the other. These studies may be referred to as one-group cluster-randomized designs or partially clustered designs. Despite recent advances, researchers do not yet have an understanding of the degree to which &quot;not&quot; accounting for clustering will bias the overall meta-analytic mean effect and its variance. The purpose of this study is to investigate the consequences of not accounting for one-group clustering in meta-analyses. The authors conducted a series of simulations in order to find out how distorted meta-analytic results can be when analysts fail to correct effect sizes for one-group clustering. The results of these analyses, in addition, demonstrate the extent to which researchers may be confident in previously conducted meta-analyses that did not have the tools to adjust the effects and variance in such designs. Data tables are appended.
Distortions in Distributions of Impact Estimates in Multi-Site Trials: The Central Limit Theorem Is Not Your Friend
Author: May, Henry
Date: 2014-00-00
Type: Reports - Research
Subjects: Educational Research,Program Effectiveness,Research Problems,Computation,Statistical Distributions,Randomized Controlled Trials,Reading Programs,Grade 1,Elementary School Students,Intervention,Monte Carlo Methods,Maximum Likelihood Statistics,Bayesian Statistics,Hierarchical Linear Modeling

Interest in variation in program impacts--How big is it? What might explain it?--has inspired recent work on the analysis of data from multi-site experiments. One critical aspect of this problem involves the use of random or fixed effect estimates to visualize the distribution of impact estimates across a sample of sites. Unfortunately, unless the sample sizes within sites are enormous (e.g., &gt;1,000), the shape of the distribution of estimates is dominated by the normal distribution as a result of imperfect reliability in site-specific estimates. The Central Limit Theorem explains why this occurs and why the problem is unavoidable. The purpose of this paper is to demonstrate the scope and severity of this problem, and to provide cautionary guidance regarding the interpretation of distributions of site-specific impact estimates in multi-site experiments. Furthermore, the paper explores methods for recovering the distribution of true site-specific impacts when that distribution is non-normal. For illustrative purposes, data from the multi-site randomized trial under the Reading Recovery Investing in Innovation (i3) Scale-Up is used in addition to generalized Monte Carlo simulation. Reading Recovery is an early reading intervention for first-graders who are reading substantially below grade level. Data figures are appended.
Efficacy of Online Algebra I for Credit Recovery for At-Risk Ninth Graders: Consistency of Results from Two Cohorts
Author: Heppen, Jessica
Date: 2012-00-00
Type: Reports - Research
Subjects: High School Students,Grade 9,High School Freshmen,Public Schools,Academic Failure,Algebra,At Risk Students,Core Curriculum,Credits,Cohort Analysis,Repetition,School Holding Power,Electronic Learning,Online Courses,Randomized Controlled Trials,Summer Schools,Outcomes of Education,Scores,Student Attitudes

The consequences of failing core academic courses during the first year of high school are dire. In the Chicago Public Schools (CPS), only about one-fifth of off-track freshmen--students who fail more than one semester of a core academic course and/or fail to earn enough credits to be promoted to 10th grade--graduate high school, compared with over 80% of on-track freshmen (Allensworth &amp; Easton, 2005, 2007). Failure of Algebra I is particularly problematic. In CPS, only 13% of students who fail both semesters of Algebra I in 9th grade graduate in 4 years, and the largest share of 9th grade algebra failures occur in the second semester of the course. Elucidating the ways that students can get back on track is of the utmost policy importance. Credit recovery is one strategy to deal with high failure rates. The primary goal of credit recovery programs is to give students an opportunity to retake classes that they failed in an effort to get them back on track and keep them in school (Watson &amp; Gemin, 2008). As schools across the nation struggle to keep students on track and re-engage students who are off track, online learning has emerged as a promising and increasingly popular strategy for credit recovery. Despite the growing use of online courses for credit recovery, the evidence base is thin. This paper describes the design, implementation, and results of a randomized control trial that was designed to address this gap. The primary intent of the proposed paper is to share findings to date for the two cohorts of students who participated in two Algebra I credit recovery courses (one online and one face-to-face) as part of this trial--first time freshmen in 2010-11 at 15 CPS high schools and first-time freshmen at 13 CPS high schools in 2011-12. Tables and figures are appended.
Evidence of Deeper Learning Outcomes: Findings from the Study of Deeper Learning
Author: Taylor, James
Date: 2014-00-00
Type: Reports - Research
Subjects: Critical Thinking,Outcomes of Education,High School Students,Quasiexperimental Design,Thinking Skills,Knowledge Level,Interpersonal Competence,College Admission,Graduation Rate,International Assessment,Achievement Tests,Reading Tests,Mathematics Tests,Science Tests,Effect Size,Student Motivation,Self Efficacy,Statistical Distributions,Self Control,Locus of Control,Persistence,Learner Engagement,Academic Achievement,Student Surveys

The &quot;Study of Deeper Learning: Opportunities and Outcomes&quot;, funded by the William and Flora Hewlett Foundation, is a &quot;proof-of-concept&quot; study to determine whether students attending high schools with a mature and at least moderately well-implemented approach to promoting deeper learning experience greater deeper learning opportunities and outcomes than they would have had they not attended these schools. The study includes a set of high schools belonging to networks that are aligned with a deeper learning approach. The purpose of this paper is to provide quasi-experimental empirical evidence on whether or not students who experienced these network schools' approaches achieved better outcomes than they would have otherwise. Specifically, did students attending these network schools: (1) score higher on measures of content knowledge and critical thinking skills associated with the OECD PISA-based Test for Schools; (2) report higher levels of intra- and inter-personal competencies and dispositions; (3) attain a higher rate of graduation; or (4) attain a higher rate of post-secondary matriculation than they would have if they had not attended the network schools? This analysis relies on student-level data from schools located in six districts across two states: California and New York. The primary findings include: (1) Students in participating network high schools scored significantly higher on the OECD PISA-based Test for Schools in reading, mathematics, and science than students in the paired non-network schools. Effect sizes for all three test subjects were similar in magnitude, with the scores of students attending network schools exceeding the scores of students in non-network schools by approximately 0.11 standard deviations; and (2) Network students reported higher levels of academic engagement, motivation to learn, self-efficacy and collaboration skills relative to similar students who attended comparison schools. Effect sizes ranged from 0.12 standard deviations (for motivation to learn) to 0.20 standard deviations (for academic engagement). On the other intrapersonal competencies (perseverance, self-management, creative thinking skills, and locus of control) there were no significant differences between network and comparison students. Exhibits are appended.
Effects of Expanding Summer Credit Recovery in Algebra
Author: Allensworth, Elaine
Date: 2014-00-00
Type: Reports - Research
Subjects: Algebra,Summer Science Programs,Program Improvement,Grade 9,Remedial Mathematics,Intervention,Educational Practices,Pretests Posttests,Cohort Analysis,Extended School Day,Program Effectiveness,Program Evaluation

In Chicago, over a quarter of students fail at least one semester of algebra in their ninth grade year, and only 13% of students who fail both semesters of Algebra I in ninth grade graduate in 4 years. Offering credit recovery options is one strategy to deal with high failure rates. The primary goal of credit recovery programs is to give students an opportunity to retake classes that they failed in an effort to get them back on track and keep them in school (Watson &amp; Gemin, 2008). While it seems like a good idea, the pay-off may not actually be large for a number of reasons: few students who failed in the prior year may show up in the summer for credit recovery; few students may pass even if they do show up; and the gains of attending summer school for learning and for credit accumulation may be very small compared to students' initial deficits in skills or the number of total credits they eventually need to recover. This study examines the benefits of offering expanded credit recovery options for ninth grade algebra, relative to business as usual (i.e., the summer programming schools would offer in the absence of efforts to expand credit recovery). The study incorporates all regular neighborhood high schools in Chicago (76) and all first-time ninth grade students who entered these schools (about 4,000 students in each cohort) between fall 2009 and fall 2011 who failed second semester algebra (Algebra IB). Tables and figures are appended.
Challenges When Using the Regression Discontinuity Design in Educational Evaluations: Lessons from the Transition to Algebra Study
Author: Louie, Josephine
Date: 2014-00-00
Type: Reports - Research
Subjects: Regression (Statistics),Research Design,Curriculum Evaluation,Algebra,Readiness,Supplementary Education,Randomized Controlled Trials,Secondary School Mathematics,Grade 9,High School Students,Evaluation Problems

Recent federal legislation, such as the &quot;No Child Left Behind Act&quot; of 2001 and the &quot;Education Sciences Reform Act (ESRA)&quot; of 2002, has insisted that educational evaluations use rigorous research designs with quantitative outcome measures. In particular, the Institute of Education Sciences (IES) at the U.S. Department of Education has promoted the use of &quot;randomized controlled trials&quot; (RCTs) as the preferred method for educational evaluations that seek to determine the causal effect of interventions. This paper describes a regression discontinuity (RD) study conducted to evaluate the impact of Transition to Algebra (TTA)--a supplemental algebra-readiness curriculum. The TTA curriculum was developed over a period of two years and then evaluated through a formal field test in two Massachusetts high schools in 2011-2012. Outcome data was collected from 85 students who had been properly assigned to the TTA intervention (the &quot;core&quot; treatment group); 9 students who were eligible for the intervention but had been randomly excluded (the &quot;core RCT&quot; comparison group); and 27 students who were ineligible for the TTA intervention due to high Grade 8 Massachusetts Comprehensive Assessment System (MCAS8) scores (the &quot;core RD&quot; comparison group). Outcomes were measured using a modified version of the MCAS test, labeled MCASmod. The research team's experience offers lessons about the implementation of RD studies and the need to work closely with practitioner partners to carry out evaluation designs that address both stakeholder concerns and broader goals of generating rigorous evidence to learn what works in education. Tables and figures are appended.
A Bayesian Nonparametric Causal Model for Regression Discontinuity Designs
Author: Karabatsos, George
Date: 2013-00-00
Type: Reports - Research
Subjects: Regression (Statistics),Bayesian Statistics,Nonparametric Statistics,Causal Models,Research Design,Schools of Education,Teacher Education Curriculum,Preservice Teachers

The regression discontinuity (RD) design (Thistlewaite &amp; Campbell, 1960; Cook, 2008) provides a framework to identify and estimate causal effects from a non-randomized design. Each subject of a RD design is assigned to the treatment (versus assignment to a non-treatment) whenever her/his observed value of the assignment variable equals or exceeds a cutoff value. The RD design provides a &quot;locally-randomized experiment&quot; under remarkably mild conditions, so that the causal effect of treatment outcomes versus non-treatment outcomes can be identified and estimated at the cutoff (Lee, 2008). Such effect estimates are similar to those of a randomized study (Goldberger, 2008/1972). As a result, since 1997, at least 74 RD-based empirical studies have emerged in the ?fields of education, political science, psychology, economics, statistics, criminology, and health science (see van der Klaauw, 2008; Lee &amp; Lemieux, 2010; Bloom, 2012; Wong et al. 2013; Li et al., 2013). Polynomial and local linear models are standard for RD designs (Bloom, 2012; Imbens &amp; Lemieux, 2008). However, these models can produce biased causal effect estimates, due to the presence of outliers of treatment outcomes; and/or due to incorrect choices of the bandwidth parameter for the local linear model. Currently, the correct choice of bandwidth has only been justified by large-sample theory (Imbens &amp; Kalyanaraman, 2012), and the local linear model for quantile regression (Frandsen et al., 2012) suffers from the &quot;quantile crossing&quot; problem. The authors introduce a novel formulation of their Bayesian nonparametric regression model (BLIND, 2012), which provides causal inference for RD designs. It is an infi?nite-mixture model, that allows the entire probability density of the outcome variable to change ?flexibly as a function of the assignment variable. Moreover, the Bayesian model can provide inferences of causal effects, in terms of how the treatment variable impacts the mean, variance, a quantile, distribution function, probability density, hazard function, and/or any other chosen functional of the outcome variable. Moreover, the accurate causal effect estimation relies on a predictively-accurate model for the data. The Bayesian nonparametric regression model attained best overall predictive performance, over many real data sets, compared to many other regression models (BLIND, 2012). Finally, the authors illustrate their Bayesian model through the causal analysis of two real educational data sets. Figures are appended.