ethanporcaro/eric-fulltext-meta huggingface dataset:
Communicating to Policymakers: What to Do with Ambiguous Results from Three Impact Studies of the Same Education Program
Author: Hartry, Ardice
Date: 2013-00-00
Type: Reports - Research
Subjects: Outcome Measures,Ambiguity (Context),Randomized Controlled Trials,Reading Programs,After School Programs,Comparative Analysis,Program Effectiveness,Elementary School Students,Intermediate Grades,Program Implementation,Program Design,Data Interpretation,Replication (Evaluation)

This paper considers one challenge from recent emphasis on rigorous research for decision-making in education: the interpretation of non-replicated findings. Under Clinton and Bush, the federal government called for and funded highly rigorous research. The &quot;gold-standard&quot; for research was randomized-control trials (RCTs), which supposedly offered clear answers about program efficacy. Such clear answers were meant to be used by stakeholders in making decisions about adopting programs. During 2005-2008, the authors conducted an RCT meant to measure the impact of a reading program in the afterschool setting, with the goal of comparing the effectiveness of that program with regular school offerings. The study covered three school years, with slight variations in both the delivery of the intervention and the design of the research, resulting in three essentially discrete studies conducted within a single school district. The authors hoped by replicating the study in one context, they would be able to draw conclusions about the effectiveness of the reading program. This paper, therefore, addresses the following questions: (1) How may contradictory findings be used in program design and implementation? (2) How may the conversation among practitioners, policymakers and researchers be expanded to acknowledge both consistency and contradictions in robust research studies? and (3) How may we better communicate what contradictory findings suggest about practice and program development to practitioners?
An Efficacy Study of Interleaved Mathematics Practice. Revised
Author: Rohrer, Doug
Date: 2013-00-00
Type: Reports - Research
Subjects: Mathematics Instruction,Teaching Methods,Assignments,Control Groups,Mathematics Teachers,Middle School Students,Grade 7,Middle School Teachers,Mathematics Tests,Instructional Effectiveness,Problem Solving

In a typical mathematics course, the material is divided into many lessons, and each lesson is followed by an assignment consisting of practice problems. Most commonly, each assignment consists solely of problems on the preceding lesson. For example, a lesson on ratios might be followed by an assignment with 12 problems on ratios. In other words, problems of the same kind are arranged in blocks. In this study, &quot;blocked assignments&quot; served as the control. In an alternative approach that is the intervention of interest, problems within the course are rearranged so that most of the problems within each assignment are based on previous lessons and arranged in an &quot;interleaved&quot; order. For example, after a lesson on ratios, an interleaved assignment might include a small block of four ratio problems and one problem from each of eight previous lessons. The remaining eight ratio problems are distributed across future assignments. Researchers conducted the study at a large public middle school in Tampa, Florida. Three middle school mathematics teachers and 140 of their seventh grade students participated. Students received 10 assignments over an 8-week period. All students received the same practice problems, but the problems were rearranged to produce two versions of each assignment. Across all assignments, students received 12 problems on each of four different kinds of problems. The study used a counterbalanced crossover design. For each assignment, researchers provided teachers with a slide presentation with solved examples and solutions to each problem. Teachers presented the examples before distributing the assignment. On the following school day, teachers presented the solutions while encouraging students to make any necessary corrections to their own solutions, and then collected the assignments. Within two days of each assignment's due date, at least one of the authors visited the school and scored the assignments. The Final Test was given to students during their regular classroom period and proctored by both the teacher and one author. Final test scores revealed that interleaved assignments were nearly twice as effective as blocked assignments. This finding suggests that interleaved mathematics assignments might be feasible and effective. This intervention could be implemented at all levels of mathematics instruction, and creators of mathematics texts and other instructional media can provide interleaved practice by simply rearranging practice problems.
Effect Size Measure and Analysis of Single Subject Designs
Author: Society for Research on Educational Effectiveness (SREE)
Date: 2013-00-00
Type: Reports - Research
Subjects: Research Design,Effect Size,Intervention,Statistical Analysis,Meta Analysis,Hypothesis Testing,Regression (Statistics),Markov Processes,Monte Carlo Methods,Comparative Analysis,Bayesian Statistics,Models

One of the vexing problems in the analysis of SSD is in the assessment of the effect of intervention. Serial dependence notwithstanding, the linear model approach that has been advanced involves, in general, the fitting of regression lines (or curves) to the set of observations within each phase of the design and comparing the parameters of these lines (or curves). In the simplest case of an AB design, this involves fitting a regression lines to the observations in phases A and B and comparing the slopes and intercepts. In the event the slope is zero in the two phases, the comparison reduces to the comparison of the means or levels. If there is a trend but the slope is the same in the two phases, then the adjusted means (levels) are compared. A problem arises if the levels and the trends in the two phases are different. In this case the intercepts and slopes are compared across the phases. While statistically such comparisons are not problematic, the interpretations of treatment effects that arise from these comparisons are not intuitive. A &quot;combination&quot; of the slope and intercept parameters that yields a single measure of treatment effect could be more meaningful. Such a measure of treatment effect yields, in turn, an effect size that can be readily computed and interpreted. To this end, the study deals with: (1) Development of a measure of the effect of intervention which leads to an effect size measure; and (2) Development of statistical procedures for assessing intervention effects that take into account the serial dependence in single-subject designs. The statistical procedure described here that includes a definition of treatment effect and consequently an effect size provides a viable method for the analysis of single-subject designs. The advantage of the procedure outlined here is that a single measure of treatment effect and consequently a single effect size measure is available when there is both a level change and trend change in the phases.
Accelerating Vocabulary Development and Reading Comprehension in Grades 3-4-5 through an Inductive Vocabulary Model
Author: Vitale, Michael R.
Date: 2013-00-00
Type: Reports - Research
Subjects: Vocabulary Development,Reading Comprehension,Grade 3,Grade 4,Grade 5,Elementary School Students,Models,Learning Strategies,Intervention,Elementary School Teachers,Instructional Effectiveness,Learner Engagement,Semantics,Achievement Tests,Standardized Tests,Control Groups,Experimental Groups,Comparative Analysis,Fidelity,Program Implementation,Pretests Posttests

A variety of research has pointed to the interdependent linkage among vocabulary knowledge, reading comprehension, and level of literacy. With this point in mind, the present study addressed the question of whether student vocabulary acquisition could be accelerated by using a multi-part, semantic word-family-oriented learning strategy to inductively expand vocabulary taught directly. The study was implemented in a large, highly diverse school system in southeastern Florida. The instructional intervention was implemented on a school-wide basis in grades 3-4-5 during an 18 week period in the school year. Observation of classroom implementation averaging 3.4 visits per teacher found the intervention easy to implement by teachers and the project-developed, story-specific vocabulary guides to be effective. Demonstrating the effect of the intervention on reading as well as on vocabulary was an important finding of the present study because engaging students in the inductive vocabulary intervention implicitly required them to focus attention on comprehension of each story. Tables and figures are appended.
Effects of the Teaching Science as Inquiry (TSI) Aquatic Professional Development Course for Middle- and High-School Teachers
Author: Seraphin, Kanesa Duncan
Date: 2013-00-00
Type: Reports - Research
Subjects: Middle School Students,Secondary School Teachers,Faculty Development,Inquiry,Science Instruction,Marine Education,Pedagogical Content Knowledge,Scientists,Self Efficacy,Questionnaires,Middle School Teachers,Teacher Surveys,Student Surveys,Program Effectiveness,Reflection,Personal Narratives,Intervention,Pretests Posttests,Qualitative Research

These authors are in the process of completing the development and evaluation of a series of professional development (PD) modules consisting of in-person trainings coupled with online learning support. The PD is grounded in the Teaching Science as Inquiry (TSI) pedagogical framework, which is centered on learning through authentic application of knowledge and skills. The purpose of the TSI Aquatic PD module series is to increase teachers' content knowledge in aquatic science disciplines and to improve teachers' (1) understanding of scientific inquiry; (2) pedagogical content knowledge needed to create classrooms that function as a community of scientists; and (e) self-efficacy in using TSI pedagogy. In July 2013 the authors will have conducted five PD cohorts, with 62 teachers and four teacher-leaders completing the program. The course begins with an introductory meeting, followed by the four modules. The four modules total more than 88 contact hours spread throughout the academic year. The authors administered teacher and student questionnaires and assessments at the beginning of the academic year, before the first PD workshop, and at the end of the academic year, after the final PD workshop. Preliminary findings suggest that the TSI Aquatic PD has positive, significant effects on teachers' and students' scientific process knowledge. Qualitative analyses of teacher reflections and narratives also suggest that explicit instruction in metacognitive strategies to teachers and their students has increased the ability of both groups to become more aware of their observations, decisions, and thought processes needed to do and understand science. Tables and figures are appended.
Developing an Aggregate Metric of Teaching Practice for Use in Mediator Analysis
Author: Lazarev, Valeriy
Date: 2013-00-00
Type: Reports - Research
Subjects: Teacher Evaluation,Evaluation Methods,Mediation Theory,Program Effectiveness,Professional Development,Teacher Education Programs,Observation,Measurement,Classroom Observation Techniques,Scoring Rubrics,Regression (Statistics),Program Evaluation

Efficacy studies of educational programs often involve mediator analyses aimed at testing empirically appropriate theories of action. In particular, in the studies of professional teacher development programs, the intervention targets presumably teacher performance while the ultimate outcome is the student achievement measured by a standardized test. The main objective of this study is to develop a methodology for creating an optimal aggregate teacher performance metric from domain scores for use as mediators in the analyses of student outcomes. The authors use one particular observation rubric, PLATO, to answer the question of how an aggregate teacher performance metric can be constructed from domain scores that would be best aligned with a selected measure of student achievement. The preliminary analysis determined that only three out of six domains of PLATO used in the study were significantly associated with the value-added measure of student achievement. Of the three significant domains, two had linear relationship with the value added. For the only domain with non-linear relationship, log transformation appeared to be sufficient to linearize the relationship. A figure is appended.
Bayesian Model Averaging for Propensity Score Analysis
Author: Kaplan, David
Date: 2013-00-00
Type: Reports - Research
Subjects: Bayesian Statistics,Models,Probability,Monte Carlo Methods,Markov Processes,Longitudinal Studies,Surveys,Children,Kindergarten,Grade 1,Attendance

The purpose of this study is to explore Bayesian model averaging in the propensity score context. Previous research on Bayesian propensity score analysis does not take into account model uncertainty. In this regard, an internally consistent Bayesian framework for model building and estimation must also account for model uncertainty. The significance of the current study is that it directly addresses the problem of uncertainty in propensity score models via the method of Bayesian model averaging (BMA). The usefulness of the proposed method is that it provides the investigator a way to incorporate prior knowledge regarding the relationship between the covariates and treatment selection (via the Kaplan and Chen, 2012 approach) while at the same time acknowledging model uncertainty via Bayesian model averaging. This paper provides a fully Bayesian MCMC methodology to obtain propensity score and treatment effect estimates, as well as R code to conduct such an analysis. Research design utilizes a combination of simulation studies and real data analysis. The simulation study examines the choice of parameter and model priors. The real data example examines a model relating full vs. part-day kindergarten attendance on achievement outcomes for first grade student using the ECLS-K. Preliminary findings suggest that the fully MCMC algorithm for Bayesian model averaging within the PSA framework provides accurate expected a posteriori estimates of the treatment effect. An appendix provides additional information on the two-step Bayesian PSA model, Bayesian model averaging, and computational considerations.
Doubling Up: Intensive Math Instruction and Educational Attainment
Author: Cortes, Kalena
Date: 2013-00-00
Type: Reports - Research
Subjects: Mathematics Instruction,Algebra,Mathematics Achievement,High School Freshmen,Intervention,Urban Schools,Low Income Students,Grade 9,Grade 8,Achievement Tests,Standardized Tests,African American Students,Hispanic American Students,Longitudinal Studies,Program Effectiveness,Statistical Significance,Comparative Analysis,Grades (Scholastic),Scores,Cohort Analysis,High School Graduates,College Admission,Educational Attainment

The purpose of this research is to study the long-run impacts (i.e. on educational attainment) of a freshman math intervention called &quot;double-dose algebra&quot;. The intervention was conducted in 2003 and 2004 within the Chicago Public Schools (CPS), a large, poor urban school district. In response to low passing rates in 9th grade algebra, CPS launched the double-dose algebra policy for students entering high school in the fall of 2003. Instead of reinstating the traditional remedial courses from previous years, CPS required enrollment in two periods of algebra coursework for all first-time 9th graders testing below the national median on the math portion of the 8th grade Iowa Tests of Basic Skills (ITBS). Such students enrolled for two math credits, a full-year regular algebra class plus a full-year algebra support class. The authors follow two cohorts of students from the Chicago Public Schools from their freshman high school fall (in 2003 and 2004) through the end of high school and the beginning of college. The main analytic sample contains upwards of 11,000 students, 95% of whom are poor and black or Hispanic. Double-dosing increased the proportion of students earning at least a B in freshman algebra by 9.4 percentage points, a more than 65% increase from a base of 13.8 percentage points. Though passing rates for freshman algebra increased by 4.7 percentage points, the increase is statistically insignificant. Double-dosed students were also no more likely to pass geometry. Results show that math instruction was most successful for students with average math skills but relatively low reading skills, and not successful for the average treated student. This highlights the importance of carefully targeting such interventions to students most likely to benefit from them. Tables and figures are appended.
Changing the Developmental Trajectory in Early Math through a Two-Year Preschool Math Intervention
Author: Starkey, Prentice
Date: 2013-00-00
Type: Reports - Research
Subjects: Preschool Children,Preschool Education,Preschool Teachers,Kindergarten,Mathematics Instruction,Mathematics Curriculum,Instructional Effectiveness,Comparative Analysis,Intervention,Longitudinal Studies,Federal Programs,Disadvantaged Youth,Child Development,Control Groups,Experimental Groups,Scores,Mathematics Tests

There is a national need for effective interventions to improve school readiness and subsequent achievement in mathematics for students from low-income families. The purpose of this study was to implement and evaluate a 2-year preschool math intervention that began at preschool entry when children were 3 years of age and continued through the end of the pre-kindergarten (pre-K) year. Three principal objectives will be addressed in this presentation: (1) to evaluate the efficacy of a math curriculum for 3-year-olds implemented in the pre-pre-kindergarten (pre-pre-K) year of preschool; (2) to compare the impact of a 2-year math intervention (implemented during pre-pre-K and pre-K years) with a 1-year math intervention (PK) or a business-as-usual control condition on children's mathematical knowledge at the end of preschool; and (3) to examine the longitudinal effects of the 2-year and 1-year math interventions on young children's mathematical development in kindergarten. The study was conducted in 63 classrooms in three Head Start programs in Northern California. Children remained in the same classrooms with the same teachers throughout their two years of preschool. The study concluded that the Pre-Pre-K Mathematics intervention had a large impact on 3-year-olds' mathematical knowledge. Likewise, the Pre-K Mathematics intervention had a significant impact on 4-year-olds' mathematical knowledge. The cumulative impact of receiving 1 vs. 2 years of math intervention appeared to be surprisingly similar at the end of pre-K. Gains children made during the 1-year intervention began to fade in K, but gains children made during the 2-year intervention were sustained in K. Possible explanations are better retention after 2 years of intervention or greater consolidation of previously learned informal math knowledge. Figures are appended.
The Utility of the SAT® I and SAT II for Admissions Decisions in California and the Nation. Research Report No. 2002-6
Author: Kobrin, Jennifer L.
Date: 2002-00-00
Type: Reports - Research
Subjects: College Entrance Examinations,Research Reports,Evaluation Utilization,College Admission,Predictive Validity,Student Characteristics,Racial Distribution,Gender Differences,Statistical Analysis,Tables (Data),Socioeconomic Status,Test Validity,Writing Tests,Mathematics Tests,Language Tests,Correlation,Relevance (Education)

This study examines the relative utility and predictive validity of the SAT I and SAT II for various subgroups in both California and the nation. The effect of eliminating the SAT I on the test impact and on the over- and under-prediction of various gender and racial/ethnic subgroups is examined. Two statistical adjustments and tables are appended.
The SAT® as a Predictor of Different Levels of College Performance. Research Report No. 2006-3
Author: Kobrin, Jennifer L.
Date: 2006-00-00
Type: Reports - Research
Subjects: Aptitude Tests,College Entrance Examinations,Predictor Variables,Academic Achievement,College Freshmen,Grade Point Average,Regression (Statistics),Scores,Accuracy,Standardized Tests,Racial Differences,Gender Differences

This study employed logistic regression to predict the probability that a student would be successful in achieving a FGPA at various levels, based on that student's SAT scores and high school grade point average. Tables of accuracy rates are appended.
Analyzing the Factorial Structure of the Classroom Assessment Scoring System-Secondary Using a Bayesian Hierarchical Multivariate Ordinal Model
Author: Yuan, Kun
Date: 2013-00-00
Type: Reports - Research
Subjects: Factor Structure,Bayesian Statistics,Multivariate Analysis,Teacher Student Relationship,Observation,Measures (Individuals),Middle Schools,High Schools,English Teachers,Mathematics Teachers,Middle School Teachers,Secondary School Teachers,Hierarchical Linear Modeling,Factor Analysis

Standardized teaching observation protocols have become increasingly popular in evaluating teaching in recent years. One of such protocols that has gained substantial interest from researchers and practitioners is the Classroom Assessment Scoring System-Secondary (CLASSS). According to the developer, CLASS-S has three domains of teacher-student interactions: Classroom Organization, Emotional Support, and Instructional Support. The goal of this study is to examine the structure of CLASS-S while accounting for the hierarchical multivariate ordinal nature of data collected through this protocol. Data used in this study came from two sources. The first source is the Understanding Teacher Quality (UTQ) study that took place middle schools in three large school systems from the same metropolitan region in the southeastern United States. The second source is the Toward an Understanding of Classroom Context (TUCC) study conducted in middle and high schools in an urban fringe mid-Atlantic school district. The UTQ study includes 458 teachers teaching mathematics (n = 231) or English language arts (n = 227) to sixth, seventh, or eight graders. The TUCC study included 82 teachers in 20 middle and 20 high schools. The authors examined the structure of CLASS-S using a Bayesian hierarchical multivariate ordinal model. Results from this study showed that accounting for the hierarchical multivariate ordinal nature of data does matter for understanding the structure of CLASS-S scores. Figures are appended.
The Relationship of PSAT/NMSQT Scores and AP® Examination Grades. Research Notes. RN-02
Author: Camara, Wayne J.
Date: 1997-11-00
Type: Numerical/Quantitative Data
Subjects: College Entrance Examinations,Scores,Advanced Placement Programs,Correlation,Predictor Variables,Predictive Validity,Success,Verbal Tests,Mathematics Tests,Grade Point Average,Ability Identification,Tables (Data),Course Selection (Students),Probability,Grades (Scholastic),High School Students

There is a strong and consistent relationship between PSAT/NMSQT scores and AP Examination grades for nearly all courses--higher scores on the PSAT/NMSQT indicate a higher probability of success on AP Examinations. PSAT/NMSQT verbal or mathematics scores are often more highly related to AP Examination grades than overall high school GPA, number of courses in the subject area, and even previous high school grades in the subject for some examinations.
The Research behind the New SAT®. Research Summary RS-11
Author: Kobrin, Jennifer L.
Date: 2007-00-00
Type: Reports - Descriptive
Subjects: College Entrance Examinations,Educational Research,Educational Change,Research Projects,Curriculum Research,Pretests Posttests,Simulation,Item Analysis,Predictive Validity

This report provides a brief summary of the research projects that have been conducted to support the development of the new SAT.