ethanporcaro/eric-fulltext-meta huggingface dataset:
Causal Inference and the Comparative Interrupted Time Series Design: Findings from Within-Study Comparisons
Author: St. Clair, Travis
Date: 2014-00-00
Type: Reports - Research
Subjects: Measurement Techniques,Time,Randomized Controlled Trials,Comparative Analysis,Performance,Benchmarking,Grade 4,Grade 5,Grade 6,Scores,Achievement Tests,English,Language Arts,Mathematics

Researchers are increasingly using comparative interrupted time series (CITS) designs to estimate the effects of programs and policies when randomized controlled trials are not feasible. In a simple interrupted time series design, researchers compare the pre-treatment values of a treatment group time series to post-treatment values in order to assess the impact of a treatment, without any comparison group to account for confounding factors. The CITS design is a version of the ITS design in which both a treatment and a comparison group are evaluated both before and after the onset of a treatment. A growing body of literature is employing a within study comparison (WSC) methodology to examine the validity of the CITS model. WSC studies empirically estimate the extent to which a given observational study reproduces the results of a randomized controlled trial (RCT) when both share the same treatment group, and represent a rigorous method of evaluating non-experimental designs using real data. A number of recent within-study comparisons have demonstrated that CITS can produce estimates that are comparable to those from a randomized controlled trial (RCT) in practice. In the St. Clair et al. (2014) application, the authors found that correspondence with the RCT was possible when the CITS model accounted for baseline trends, but that additional time points could actually increase bias when the pre-treatment trend was not modeled correctly. Examination of the pretreatment trends in this data set showed clearly that in at least one of the outcomes the treatment and comparison groups had different slopes in the pretreatment period, and as a result the &quot;parallel trends&quot; assumption often invoked in the difference-in-difference literature was clearly violated. This paper employs a within study comparison (WSC) methodology to examine the performance of two approaches: (1) a more flexible modeling approach, which employs year fixed-effects rather than trying to parametrically model the pretest trend; and (2) match treatment and comparison cases to reduce reliance on modeling the pretreatment trend. The paper then compares the approaches to the performance of the baseline mean and baseline slope models across three datasets. The purpose of this research is two-fold: (1) to examine what approach, if any, works in the unclear functional form case; and (2) to examine the relative superiority of the different approaches across the three datasets in terms of both bias reduction and precision. Tables and figures are appended.
Comparing the Effectiveness of Head Start and State Pre-K Using a Propensity-Score Matching Regression Discontinuity Design
Author: Jenkins, Jade Marcus
Date: 2014-00-00
Type: Reports - Research
Subjects: Comparative Analysis,Program Effectiveness,Preschool Education,Public Education,Regression (Statistics),Statistical Analysis,State Programs,Federal Programs,Student Participation,Preschool Children,School Entrance Age,Skills,Educational Quality

This study is a secondary data analysis of data collected on preschool and kindergarten-aged children in Tulsa, Oklahoma in the fall of 2006 to study the effects of school and community-based public early learning programs on children's cognitive and social-emotional outcomes. Children in the study participated in the state-funded pre-k program, in the county Head Start (HS) program, or a combination of both. The present study aims to compare the overall effects of a high-quality state pre-k program and Head Start; compare the effects of different pathways into pre-k and Head Start; and to test whether two years of participation in public early learning programs (age three and age four participation) are more effective than one year (age four participation only). The overall sample includes 1,985 pre-k entrants; 739 HS entrants; and 4,121 kindergarten entrants (1,597 pre-k graduates; 474 HS graduates; and 2,050 who did not attend a public preschool program). The data come from four sources: direct cognitive assessments of children at the beginning of the school year; parent surveys collected at their child's cognitive assessment; social-emotional assessments conducted by each child's teacher; and administrative data from Tulsa Public Schools and Head Start. Using propensity-score matched groups, a regression discontinuity (RD) design was implemented to estimate program effects on outcomes at kindergarten entry. Findings show that while both pre-k and HS are effective at improving children's pre-academic skills at school entry after one year of participation, the high-quality Oklahoma pre-k program produces stronger gains in skills at the start of kindergarten than the Oklahoma HS program. Study limitations are discussed. Tables are appended.
A Case Study of Assessing an International Initiative on Curriculum Reform
Author: Reichman, Roxana G.
Date: 2014-00-00
Type: Reports - Research
Subjects: Foreign Countries,Educational Assessment,Case Studies,Curriculum Development,Colleges,Schools of Education,Multicultural Education,Graduate Students,Undergraduate Students,Cultural Pluralism,Models,Qualitative Research

The globalized world we live in has forced institutions of higher education and of teacher education to deal with different aspects of diversity. More and more institution have become involved in creating programs geared at promoting multiculturalism on campus, as well as in changing the curriculum in order to make it more relevant to students who have to be prepared to live and work in a multicultural society. Previous research has shown the importance of using diverse assessment techniques in order to make sure that the programs meet the needs of the faculty and of the students who take part in developing and teaching courses on this subject both at the undergraduate and at the graduate level (Krishnamurthi, 2003, Levitan &amp; Wolf, 1994, Brown, 2008). There are many definitions of multiculturalism but the common denominator stresses the importance of a conscious effort to assure that all people, regardless of race, religion, gender, etc get equal opportunities to live, study and work in any heterogeneous society. The objective of this paper is to present and examine a model used for the assessment process of the curriculum which has been developed for multicultural education and cultural diversity training by 60 consortium members over a year. The present study focuses specifically on the criteria for choosing this specific model, which borrowed from several models of assessing curriculum reform in the field of multicultural education in teacher education and in higher education. It is important to stress the fact that the curriculum developed by the consortium members does not only affect the institutions that will implement it, but it also has a domino effect because the teachers who are exposed to it will hopefully influence their schools at all levels and thus will have an impact on students of all ages.
A Study Examining the Dimensionality of Core Competencies Measure in Teacher Preparation Programs: Challenges and Lessons
Author: Kizil, Ruhan Circi
Date: 2014-00-00
Type: Reports - Research
Subjects: Teacher Competencies,Teacher Education Programs,Educational Quality,Educational Research,Federal Aid,Novices,Beginning Teachers,Teacher Surveys,Teacher Attitudes,Program Effectiveness,Computer Software,Factor Analysis,Tables (Data),Statistical Analysis,Pedagogical Content Knowledge,Intervention,Comparative Analysis,Models,Outcomes of Education,Program Evaluation

The evidence that teacher preparation programs have an impact on teacher quality is often limited. Progress in research on this topic will remain rather limited in its influence on practice until more proximal measures of teacher education outcomes can be established. The dearth of variables to measure the impact of teacher preparation programs on teacher skills constitutes a measurement problem. As a part of a three year Institute of Education Sciences (IES)-funded project, the big picture motivating questions for the present study are as follows: (1) What is the best characterization of the dimensional structure of the core competencies (CC) survey?; and (2) How does the choice of dimensional structure change inferences about differences in quality among teacher preparation programs in Colorado? This study utilized data collected from all teacher preparation programs in Colorado. There are two groups of self-selected participants in this study: novice teachers in their first three years of teaching (graduates); and respondents just completing their preparation programs at the time of survey administration (candidates). A total of 648 graduates from 18 programs responded to at least some portion of the graduate survey, and a total of 355 candidates from 13 programs responded to at least some portion of the candidate survey. Both surveys were administered using the survey software Qualtrics via the internet. Three approaches are used for exploring dimensional structure: (1) exploratory factor analysis (EFA); (2) confirmatory factor analysis (CFA); and (3) bi-factor analysis. Findings show the following: (1) on the basis of a purely exploratory approach, an argument can be advanced for collapsing CCs into an overall composite; (2) on the basis of a confirmatory approach, an argument can be advanced for reporting 8 dimensions; (3) the bi-factor approach can be seen as a compromise between the first two approaches; and (4) the consequence of decisions made about how to represent the dimensional structure of the instrument such that each different outcome measure lead to different results. Tables are appended.
Can We Improve Preschool Classroom Quality in Chile? A Cluster-Randomized Trial Evaluation of a Professional Development Program
Author: Leyva, Diana
Date: 2014-00-00
Type: Reports - Research
Subjects: Foreign Countries,Preschool Education,Educational Quality,Educational Improvement,Intervention,Program Effectiveness,Program Evaluation,Faculty Development,Preschool Teachers,Kindergarten,Language Skills,Health Promotion,Socioeconomic Influences,Public Schools,Randomized Controlled Trials,Language Proficiency,Second Language Learning,English (Second Language),Surveys

In recent years, governments across the world have increased their investments in early childhood education programs (Britto, Yoshikawa, &amp; Boller, 2011; Engle et al., 2011; Myers, 2006; UNESCO, 2006). This investment in preprimary education reflects the proven value of providing early learning opportunities in promoting later educational attainment and earnings (Cunha &amp; Heckman, 2006). While the initial focus for those interested in improving early childhood education in low and middle-income countries was access, the reform emphasis has progressively shifted to quality and equity, as overall access has improved in many nations. The intervention program &quot;Un Buen Comienzo&quot; (A Good Start) constitutes the first large-scale, randomized evaluation of an effort to improve the quality of preschool education in South America. &quot;Un Buen Comienzo&quot; (henceforth UBC) is an intensive two-year intervention that provides professional development program to prekindergarten and kindergarten teachers in Chile, with the goal of enhancing children's language, literacy, health and socio-emotional outcomes. This study provides useful data on classroom quality under usual conditions in government-funded pre-kindergarten and kindergarten classrooms in urban Chile. Tables are appended.
Measuring Educational Disadvantage of SAT® Candidates. Research Report No. 2002-1. ETS RR-02-01
Author: Stricker, Lawrence J.
Date: 2002-00-00
Type: Reports - Research
Subjects: Research Reports,College Entrance Examinations,Individual Differences,Disadvantaged,Factor Analysis,Performance Factors,Institutional Characteristics,Educational Environment,Family Environment,Parent Influence,Socioeconomic Status,Ethnicity,Racial Differences,Grades (Scholastic),Scores,Predictor Variables,Questionnaires,Student Records,Factor Structure,Correlation,Tables (Data),Student Surveys,Student Characteristics

This study explored individual differences in educational disadvantage-- deficits in formal and informal education in the school, home, and elsewhere--in the SAT test-taking population. Factor analysis identified six educational disadvantage factors--four concerning the students' schools and two the students' nativity and parenting--and one family socioeconomic status factor, race/ethnicity, high school grades, and SAT scores. The individual-differences perspective on disadvantage appears to be a viable one, and educational disadvantage seems to be a meaningful and useful construct. The following are appended: (1) Letters; and (2) Questionnaire.
Estimating the Impact of the PROMISE Scholarship Using Propensity Score Weighted Frontier Fuzzy Regression Discontinuity Design
Author: Shobo, Yetty
Date: 2014-00-00
Type: Reports - Research
Subjects: Program Effectiveness,Scholarships,Regression (Statistics),Research Design,Statistical Analysis,Computation,In State Students,College Students,Graduation Rate,Grade Point Average,College Credits

Regression discontinuity (RD), an &quot;as good as randomized,&quot; research design is increasingly prominent in education research in recent years; the design gets eligible quasi-experimental designs as close as possible to experimental designs by using a stated threshold on a continuous baseline variable to assign individuals to a &quot;treatment&quot;. Fuzzy RD in which the threshold does not perfectly predict treatment receipt is a subset of this increasingly popular design. In higher education, RD design is increasingly prominent in estimating the impact of merit-based aids on enrollment, school completion, grade point average (GPA), and other key outcomes. Merit-based aids are increasingly allocated based on individuals meeting multiple academic proficiency criteria. A careful examination of merit-based scholarship is particularly critical in West Virginia, a state with the lowest percent of adults 25 and older who have a Bachelor's degree (US Census, 2006). Started in 2002, the West Virginia PROMISE grant aimed to &quot;improve high school and post secondary academic achievement through scholarship incentives&quot; and &quot;promote access to higher education by reducing cost to students.&quot; Originally, the grant provided full tuition for attending public colleges (equivalent amount was provided for students in in-state not-for-profit private colleges) but it now provides a maximum of $4750 towards tuition. Academic eligibility to receive and continue receiving the four-year maximum grant is increasingly stringent. The present study utilizes propensity scoring technique in a frontier RD design to estimate the effects of West Virginia's (WV) PROMISE on the quantity (4- and 5-year graduate rates, sum of credits earned) and quality (cumulative GPA) of long-term college indicators. The study examines the effects of receiving West Virginia's PROMISE scholarship for in-state students attending four-year public institutions. The present study uses fuzzy and frontier RD designs because PROMISE receipt is not perfect above the threshold and is based on multiple criteria. Administrative data was obtained from West Virginia Higher Education Policy Commission (WVHEPC). This study demonstrates the significant potential that the frontier RD and the propensity score weighting techniques hold for education research. Tables and figures are appended.
Empirical Benchmarks of Hidden Bias in Educational Research: Implication for Assessing How well Propensity Score Methods Approximate Experiments and Conducting Sensitivity Analysis
Author: Dong, Nianbo
Date: 2014-00-00
Type: Reports - Research
Subjects: Educational Research,Benchmarking,Statistical Analysis,Computation,Comparative Analysis,Experiments,Statistical Bias,Program Evaluation,Curriculum Evaluation

When randomized control trials (RCT) are not feasible, researchers seek other methods to make causal inference, e.g., propensity score methods. One of the underlined assumptions for the propensity score methods to obtain unbiased treatment effect estimates is the ignorability assumption, that is, conditional on the propensity score, treatment assignment is independent of the outcome. The purpose of this study is to use within-study comparisons to assess how well propensity score methods can approximate experiments under various conditions. The authors test three ways of constructing comparison groups: (1) using the sample from the states that are different from the original experiments, with pretest and demographic information; (2) using the sample from the same state and districts (local matching) with the experiments, with demographic information only; and (3) same as (2) but with pretest as well. Propensity score methods (optimal matching, propensity score weighting, and stratification) are used to estimate the treatment effects for three ways of constructing comparison groups, which are compared with the benchmark from the experiment to assess estimate bias. This study will contribute to the literature by providing empirical evidence about how well propensity score methods can approximate experiments under various conditions. This study uses data from four IES funded projects, among which three are large scaled experiments: (1) &quot;Scaling up TRIAD: Teaching Early Mathematics for Understanding with Trajectories and Technologies&quot;; (2) &quot;Evaluating the Effectiveness of Tennessee's Voluntary Pre-Kindergarten Program&quot;; and (3) &quot;Experimental Evaluation of the Tools of the Mind Pre-K Curriculum&quot;, and one is a measurement study, &quot;Learning-Related Cognitive Self-Regulation School Readiness Measures for Preschool Children Study&quot;. The analysis is undergoing. The authors report partial results in this report. Tables and figures are appended.
On Internal Validity in Multiple Baseline Designs
Author: Pustejovsky, James E.
Date: 2014-00-00
Type: Reports - Research
Subjects: Validity,Research Design,Hierarchical Linear Modeling,Case Studies,Models

Single-case designs are a class of research designs for evaluating intervention effects on individual cases. The designs are widely applied in certain fields, including special education, school psychology, clinical psychology, social work, and applied behavior analysis. The multiple baseline design (MBD) is the most frequently used single-case design, and is often described as having desirable internal validity characteristics. This paper examines how the use of certain hierarchical linear models that have been proposed for the MBD affect internal validity, focusing specifically on the two key features described above. First, the author considers a deliberate but non-random method of treatment assignment. I demonstrate that the treatment effect estimate from a conventional multi-level model can be biased under this mechanism, then provide an expression for the magnitude of the bias. Second, he argues that the main analytic models proposed for the MBD fail to capture the benefits of staggered treatment introduction. The author proposes an alternative model that does account for this key feature and, based on the model, he defines an index measuring the strength of control offered by the MBD. A table and figure are appended.
Teacher Retention, Mobility, and Attrition in Kentucky Public Schools from 2008 to 2012. REL 2016-116
Author: Lochmiller, Chad R.
Date: 2016-01-00
Type: Reports - Research
Subjects: Public School Teachers,Teacher Persistence,Faculty Mobility,Public Schools,Teacher Characteristics,Institutional Characteristics,Elementary Secondary Education,Age Differences,Racial Differences,Teaching Experience,Educational Attainment,Student Characteristics,Economically Disadvantaged,Rural Urban Differences

The purpose of this descriptive study was to determine the rates of retention, mobility, and attrition for classroom teachers in Kentucky public schools, as well as how those rates might vary by various teacher and school characteristics. The study looks at retention, defined as teachers returning to their same classroom (&quot;stayers&quot;); mobility, when teachers change schools within the school system (&quot;movers&quot;); and attrition, when teachers leave the system (&quot;leavers&quot;) from one year to the next. The study used data on teachers collected by the Kentucky Center for Education &amp; Workforce Statistics on every teacher employed in PK-12 public schools in academic years 2008/09, 2009/10, 2010/11, and 2011/12. Data on schools were obtained from the National Center for Education Statistics Common Core of Data. The study found that the Kentucky teacher workforce was largely stable across the study period (2008-2012). Most teachers (85.6 percent, on average) stayed in the same school from one year to the next, 6.0 percent moved to a different school, and 8.4 percent left the public school system. The study revealed some variation in rates based on select teacher and school characteristics. In particular, teachers with the fewest years of experience, teachers in urban schools, and teachers in schools where more students are eligible for free or reduced price lunch were retained at the lowest rates. Data sources and methods are appended.
Examining Teacher, School, and Program Moderators in the Context of Teacher Professional Development Studies
Author: Phelps, Geoffrey
Date: 2014-00-00
Type: Reports - Research
Subjects: Influences,Faculty Development,Program Evaluation,Teacher Evaluation,Elementary School Teachers,Middle School Teachers,Pretests Posttests,Teacher Characteristics,Mathematics Education,Computation,Change

A variety of factors have converged in the last decade to focus attention on the need for more rigorous designs of professional development (PD). Teacher development is increasingly viewed as one of the primary levers for improving teaching quality and ultimately student achievement. The authors present data from the Teacher Knowledge Assessment System (TKAS), which is designed to administer the Learning Mathematics for Teaching (LMT) measures. TKAS is being widely adopted in the evaluation of PD programs with over 500 separate program administrations and 16,000 teachers representing every major region in the country. The study findings provided potential guidance for the design of studies using teacher knowledge as an outcome measure. Also, the results provide context relevant guidelines (e.g., test outcome, characteristics of teachers, school or program) that researchers may use individually or in combination to design studies and/or assess the practical impact of teacher PD interventions.