ethanporcaro/eric-fulltext-meta huggingface dataset:
Using Dirichlet Processes for Modeling Heterogeneous Treatment Effects across Sites
Author: Miratrix, Luke
Date: 2016-00-00
Type: Reports - Research
Subjects: Probability,Models,Statistical Distributions,Bayesian Statistics,Hierarchical Linear Modeling,Simulation,Data Analysis,Error of Measurement,Measurement,Statistical Analysis,Randomized Controlled Trials

Modeling the distribution of site level effects is an important problem, but it is also an incredibly difficult one. Current methods rely on distributional assumptions in multilevel models for estimation. There it is hoped that the partial pooling of site level estimates with overall estimates, designed to take into account individual variation as compared to site level variation, does not distort the overall distribution of the Empirical Bayes site-level estimates too badly. The authors plan on investigating this claim, and comparing the results of multilevel modeling to multilevel modeling where the assumption of the normally distributed site level effects is relaxed by replacing it with an unspecified distribution assigned a Dirichlet process prior. The authors hope, via simulations and analysis of actual data sets, to identify where multilevel modeling does work in practice, where it is likely to lead to erroneous conclusions, and what requirements one needs to use these approaches with reasonable levels of confidence. It may be the case that instead of attempting to measure the distribution of site level effects, they should instead attempt to directly measure summary statistics of these distributions (such as skewness and variance). These are important next steps, once they establish how this approach generally works in practice. One figure is appended. [SREE documents are structured abstracts of SREE conference symposium, panel, and paper or poster submissions.]
What Works to Improve Reading Outcomes in Latin-America? A Systematic Review of the Evidence
Author: de Hoop, Thomas
Date: 2016-00-00
Type: Information Analyses
Subjects: Reading Achievement,Latin Americans,Elementary School Students,Reading Research,Outcomes of Education,Reading Programs,Reading Instruction,Teaching Methods,Reading Improvement,Quasiexperimental Design,Qualitative Research,Intervention,Correlation,Check Lists,Statistical Analysis,Meta Analysis,Foreign Countries,Literature Reviews

Improvements in students' learning achievement have lagged behind in low-and middle-income countries despite significant progress in school enrollment numbers. Large-scale early grade reading assessments (e.g., &quot;Annual Status of Education Report&quot; [ASER], 2013; EdData II, n.d.) have shown low reading rates and worryingly high &quot;zero&quot; scores in reading assessments from across the world, including Latin-America. Numerous initiatives are underway globally to try to improve children's literacy development. Although several studies focus on the impact of these initiatives in low-and middle-income countries in Latin-America, to the best of the authors' knowledge, there exists no systematic synthesis of the evidence on what works to improve early grade reading outcomes in the Latin-American and Caribbean region. This systematic review synthesizes the existing literature on what works to improve early grade reading outcomes in the Latin-American and Caribbean (LAC) region. The review addresses the following research questions: (1) What is the impact of reading programs, practices, policies and products aimed at improving the reading skills for children from birth through grade 3 on reading outcomes in the LAC region; and (2) What are the gaps in the evidence base on early grade reading in the LAC region? The authors synthesize the evidence from experimental and quasi-experimental studies to address these research questions. In addition, they synthesize the qualitative evidence associated with reading outcomes in the LAC region and evidence linked to reading outcomes but not associated with a specific intervention. [SREE documents are structured abstracts of SREE conference symposium, panel, and paper or poster submissions.]
On the Appropriateness of Surveying Students in 4th and 5th Grades
Author: Luppescu, Stuart
Date: 2016-00-00
Type: Reports - Research
Subjects: Grade 4,Grade 5,Elementary School Students,Public Schools,Academic Achievement,Predictor Variables,Age Differences,Student Characteristics,Instructional Program Divisions,Psychometrics,Student Surveys,Urban Schools,Grade 6,Grade 7,Grade 8,Administrators,Teachers,Parents,Educational Environment,Peer Influence,English,Language Arts,Mathematics Instruction,School Safety,Trust (Psychology)

The UChicago Consortium on School Research has been surveying students, teachers, and principals in the Chicago Public Schools (CPS) since 1991 using what is now known as the 5Essentials surveys. The developers categorized and grouped survey measures into what they refer to as &quot;The Five Essentials for School Improvement.&quot; The value of these surveys is that they measure school organizational quality that predicts student academic improvement, which has been demonstrated convincingly (e.g., Bryk, et al. 2010). What used to be just a CPS instrument has since been adopted as a school improvement vehicle in several school districts across the United States. For example, the 5Essentials has been used in Minneapolis, Detroit, Florida, among other places. Since 2013, it has been administered across the state of Illinois. To address the concerns raised by school districts outside of Chicago, the current author sought to test whether the expansion of the student survey to students in grades 4 and 5 was feasible. There are number of valid reasons why there may be concerns about surveying students this young, including: (1) Younger students could have trouble reading and understanding survey question text because of lower levels of literacy; (2) Younger students could have less ability to accurately assess their observations of the context around them; and (3) Younger students could be less able to recognize and process their emotional reactions to these observations. Levenstein and Luppescu (2015) found evidence for the relationship between grade level and survey measure person fit, comparing students in the lower-middle grades and the upper middle grades. Al-Tayyip et al. (2002) found that lower levels of literacy interfered with respondents' being able to provide accurate survey data. The author takes this idea further, applying it to an investigation of the psychometric properties of survey measures obtained from students in grades 4 and 5. In order to avoid reporting back to K-5 schools on only three of the five essentials, this study tests whether students in grades 4 and 5 can provide reliable, valid survey responses on the 5Essentials. Tables and figures are appended. [SREE documents are structured abstracts of SREE conference symposium, panel, and paper or poster submissions.]
Parent Beliefs and Student Absences: Large Absence-Reduction Field Experiment
Author: Rogers, Todd
Date: 2016-00-00
Type: Reports - Research
Subjects: Parent Attitudes,Attendance,Dropout Rate,Parent School Relationship,Educational Strategies,Intervention,Metropolitan Areas,School Districts,Elementary School Students,Middle School Students,High School Students,Student Records,Enrollment,Cost Effectiveness,Social Work,Truancy,Telephone Surveys,Regression (Statistics),Control Groups,Experimental Groups,Letters (Correspondence),Least Squares Statistics

School attendance is a robust predictor of course performance, and it is consistently the strongest predictor of high school dropout, even more so than suspensions and test scores. Focusing on getting students to school is an essential part of decreasing high school dropout rates. What is concerning is that up to 20% of students miss essentially a month or more of schooling in a year. Recent work suggests that guardians are unaware of how their student's attendance compares to that of their classmates; moreover, they often are very miscalibrated in estimating how many days of school their own student has been absent. The objective of the project presented in this study is to motivate parents/guardians to improve student attendance through multiple communications during the school year. The project addresses the following research questions: (1) Does contacting guardians and encouraging them to improve their students' attendance reduce absences?; (2) Does communicating to guardians the total number of days their student missed reduce absences?; (3) Does communicating to guardians the total number of days their student missed as compared to the absences of a typical student reduce absences?; and (4) Do these interventions also impact the attendance of other students in the household not explicitly mentioned in the mailings? The study involves students and their guardians in all public elementary, middle, and high schools in a major metropolitan school district. This study incorporated various data sets exported directly from the administrative records of the school district. The data sets included student demographics and enrollment data, guardian contact information, and attendance data. This study identifies an easy-to-implement, extremely cost-effective way to reduce absenteeism. This intervention cost around $6 per incremental day of attendance it generated. Back-of-the-envelope estimates of the cost per incremental day of attendance from social workers and truancy officers is around $50-$100. Two tables are appended. [SREE documents are structured abstracts of SREE conference symposium, panel, and paper or poster submissions.]
Valuing a More Rigorous Review of Formative Assessment's Effectiveness
Author: Apthorp, Helen
Date: 2016-00-00
Type: Reports - Research
Subjects: Formative Evaluation,Academic Achievement,Classification,Intervention,Evaluation Methods,Elementary School Students,Learning Disabilities,Emotional Disturbances,Intellectual Disability,Evidence,Inferences,Correlation,Academic Standards,Outcomes of Education,Meta Analysis,Instructional Effectiveness,Literature Reviews

Prior reviews of evidence for the impact of formative assessment on student achievement suggest widely different estimates of formative assessment's effectiveness, ranging from 0.40 and 0.70 standard deviations in one review. The purpose of this study is to describe variability in the effectiveness of formative assessment for promoting student achievement by refining, updating and applying Black and Wiliam's (1998a) original typology of formative assessment interventions and including only those studies that meet rigorous evidence standards for supporting causal inferences. The systematic evidence review focuses on the effectiveness of formative assessment for improving the academic achievement of students in grades 1-6, including both regular education students and students with learning disabilities, emotional disturbance, and/or intellectual disabilities. Although findings from the study are currently under review, the process of preparing for and responding to reviews of the report posed several challenges and opportunities to consider possible solutions regarding communicating the value of a more precise estimate of formative assessment's effectiveness to practitioners and policy-makers. In conclusion, the formative assessment evidence review provides an opportunity to examine evidence that ordinarily is interpreted as strongly supporting the view that formative assessment is effective for improving student achievement. By extending conceptual frameworks for categorizing types of formative assessments presented in prior reviews and adopting rigorous standards to assess whether studies support causal inferences, this evidence review allows practitioners and researchers to examine relationships between effectiveness and both malleable and non-malleable factors. Figures are appended. [SREE documents are structured abstracts of SREE conference symposium, panel, and paper or poster submissions.]
Findings from a Multi-Year Scale-Up Effectiveness Trial of Open-Court Reading (Imagine It!)
Author: Vaden-Kiernan, Michael
Date: 2016-00-00
Type: Reports - Research
Subjects: Reading Programs,Program Effectiveness,Evidence Based Practice,Elementary School Students,Reading Difficulties,Prevention,Outcomes of Education,Reading Instruction,Instructional Effectiveness,Phonemic Awareness,Phonics,Reading Fluency,Vocabulary Development,Reading Comprehension,Randomized Controlled Trials,Experimental Groups,Control Groups,Pretests Posttests,Hierarchical Linear Modeling,Scores

This study addresses the effectiveness of a widely used core reading program that reflects the research-based practices recommended by the National Reading Panel. This and other similar programs are increasingly used to prevent reading difficulties and ensure that all children are reading at or above grade level by the end of third grade. Effective early reading instruction is critical for preventing later reading difficulties. With twothirds of 4th grade students failing to achieve proficiency in reading on the National Assessment of Educational Progress in 2013, the need to implement and test effective early reading programs is relevant and pressing. The &quot;Open Court Reading&quot; (OCR) program published by SRA/McGraw-Hill and widely used for almost 40 years is a phonics-based core-reading program for students in kindergarten to 6th grade that incorporates many of the instructional practices related to phonemic awareness, phonics, fluency, vocabulary, and text comprehension recommended by the National Reading Panel. In this study, an independent research team evaluated the effectiveness of the OCR program in a large national sample of elementary schools at scale, across diverse school populations and conditions, and with no more support than schools would have access to if they had selected OCR as their early reading curriculum apart from participation in a research project. The study participants included approximately 4,500 elementary school students and 1,200 teachers across 49 schools in 7 districts each year of the study. The evaluation of the OCR program involved two key elements: the multi-site cluster randomized trial (CRT) and the implementation study. Data from teachers and students in two cohorts (grades K&amp;3 and grades 1&amp;4) were gathered over two school years. Each year of data was collected as cross-sectional and each year is analyzed independently. This study provides preliminary evidence that the impacts of OCR are not significant on overall students' reading performance when implemented at scale in a large sample of schools after one or two years relative to other core reading curricula. However, there were positive differential impacts for Kindergarten students as well as Hispanic students. Tables are appended. [SREE documents are structured abstracts of SREE conference symposium, panel, and paper or poster submissions.]
Analyzing Empirical Evaluations of Non-Experimental Methods in Field Settings
Author: Steiner, Peter M.
Date: 2016-00-00
Type: Reports - Research
Subjects: Randomized Controlled Trials,Comparative Analysis,Research Design,Evaluation Methods,Field Studies,Statistical Analysis,Evaluation Criteria,Research Methodology,Statistical Significance,Probability,Sample Size

Despite recent emphasis on the use of randomized control trials (RCTs) for evaluating education interventions, in most areas of education research, observational methods remain the dominant approach for assessing program effects. Over the last three decades, the within-study comparison (WSC) design has emerged as a method for evaluating the performance of non-experimental methods in field settings. In a traditional WSC design, treatment effects from a randomized experiment (RE) are compared to those produced by a non-experimental approach that shares the same target population and intervention. The purpose of this paper is two-fold. First, it is to examine methods for assessing correspondence in experimental and non-experimental results in WSC designs, and their relative advantages and limitations. Second, it is to provide methods for calculating statistical power in WSC designs that evaluate the empirical performance of non-experimental methods. The authors apply multiple correspondence criteria to two WSC examples, highlighting contexts and conditions under which these methods converge at the same result, but also when they differ. Finally, the authors address correspondence criteria for WSC designs with independent experimental and nonexperimental conditions, and for cases when the conditions have dependent data structures. 2 figures are appended. [SREE documents are structured abstracts of SREE conference symposium, panel, and paper or poster submissions.]
The Development and Testing of a New Measure of Early Childhood Education Organizational Conditions
Author: Ehrlich, Stacy B.
Date: 2016-00-00
Type: Reports - Research
Subjects: Test Construction,Teacher Surveys,Parent Surveys,Early Childhood Education,Institutional Characteristics,Educational Environment,Test Validity,Test Reliability,Preschool Teachers,Interviews,Focus Groups

The current study sought to scientifically adapt the existing five essentials teacher survey and develop a new parent survey, and pilot test them for use in early childhood education settings. The adaptation and development of these &quot;Early Education Surveys&quot; occurred in 2014-15, was based strongly on Bryk et al.'s (2010) existing framework, and utilized the existing &quot;5 Essentials&quot; surveys, which are publicly-available through the Consortium on Chicago School Research's website, with author permission. This proposed paper presents the development process of these two surveys and pilot test findings. In the future, the authors' work will test whether these surveys measure organizational elements of a preschool program that are related to positive classroom and child outcomes via a validation study (to be conducted in 2015-16). Ultimately, the purpose of the &quot;Early Education Surveys&quot; is to provide reliable and valid survey data that will guide early childhood educators to generate continuous quality improvements in teaching and learning by strengthening the organizational conditions in which teachers work every day with children and families. Tables and figures are appended. [SREE documents are structured abstracts of SREE conference symposium, panel, and paper or poster submissions.]
Do Charter School Networks Deflate as They Expand? Trends in the Impacts of KIPP Schools during a Period of Rapid Growth in the KIPP Network
Author: Nichols-Barrer, Ira
Date: 2016-00-00
Type: Reports - Research
Subjects: Charter Schools,Networks,Network Analysis,Educational Trends,School Effectiveness,Evidence,Experimental Groups,Control Groups,Comparative Analysis,Middle School Students,Grade 5,Grade 6,Urban Schools,Program Effectiveness,Statistical Significance,Scores,Least Squares Statistics,Regression (Statistics)

The Knowledge Is Power Program (KIPP) is a national network of public charter schools whose stated mission is to help underserved students enroll in and graduate from college. As of 2014-2015 the network included 162 elementary, middle, and high schools serving 59,495 students. Questions remain, however, about the ability of charter school networks like KIPP to remain effective as they expand and serve larger numbers of students. In this paper, the authors present estimates of the impacts of KIPP middle schools during a period in which the network greatly expanded. In particular, they measure KIPP impacts between 2005 and 2014, a period in which the number of schools in the network increased from 40 to 140. They aim to explore whether there is any evidence of an increase, decline, or stability in KIPP impacts over this period and the extent to which variation in network-wide impacts is driven by changes to the composition of the network versus changes over time in the effectiveness of existing schools. The paper focuses on 37 KIPP middle schools open during the 2005-2014 period in 14 KIPP regions and 11 states. The study sample included a treatment group of students who entered a KIPP school for the first time in grade 5 or grade 6, and a matched comparison group of students in the same school districts who did not attend KIPP. In total, a sample of 20,312 treatment students and 20,312 matched comparison students were analyzed. The study relied on a matched comparison group design that used &quot;nearest neighbor&quot; matching to identify a similar comparison student for each treatment student entering a KIPP middle school in grade 5 or grade 6. Once the matched comparison group was identified, impacts were estimated using ordinary least squares (OLS) regressions that control for any remaining baseline differences between KIPP students and comparison students. The authors used de-identified, longitudinally-linked student-level data from jurisdictions (states or districts) hosting at least one KIPP school and able to provide student-level records at the time of data collection. The study results suggest that KIPP middle schools continued to produce positive impacts on student achievement outcomes even as the organization faced the pressures of growth in terms of student recruitment, hiring new principals and teachers, and managing turnover in the network's workforce over time. Figures are appended. [SREE documents are structured abstracts of SREE conference symposium, panel, and paper or poster submissions.]