ethanporcaro/eric-fulltext-meta huggingface dataset:
How Non-Linearity and Grade-Level Differences Complicate the Validation of Observation Protocols
Author: Lazarev, Valeriy
Date: 2013-00-00
Type: Reports - Research
Subjects: Teacher Evaluation,Observation,Scores,Value Added Models,Teacher Competencies,Mathematics Teachers,Mathematics Instruction,Instructional Program Divisions,Grade 4,Grade 5,Grade 6,Grade 7,Grade 8,Elementary Schools,Middle Schools,Departments,Correlation,Urban Schools,School Districts,Teacher Effectiveness,Statistical Analysis,Video Technology

Teacher evaluation is currently a major policy issue at all levels of the K-12 system driven in large part by current US Department of Education requirements. The main objective of this study is to explore the patterns of relationship between observational scores and value-added measures of teacher performance in math classrooms and the variation in these relationships across grade levels. While the MET analyses used a single composite score consisting of a simple average of the eight component scores of the protocol, in our work we treated each component separately since each measures a separately definable aspect of classroom practice. Specifically, across all the components, the authors pose the following questions: (1) Do the relationships between observation scores of math teachers and their value-added scores tend to be non-linear?; (2) Is there a difference in the relationships associated with developmental changes as indicated by grade levels (four through eight)?; and (3) To what extent the observed changes are associated with increasing departmentalization (in upper elementary and middle school)? The relationship between classroom observation instruments and teacher effectiveness is often presumed to be a simple linear mapping. The authors find that the relationship is complex: it is typically non-linear and the shape of this relationship appears to evolve with the grade level progression. Careful analysis of such relationships will improve understanding of teacher effectiveness and result in more accurate teacher evaluation systems. Better evaluation systems will in turn generate more useful administrative data for use in effectiveness research. Tables and figures are appended.
Tuition Discounting: Institutional Aid Patterns at Public and Private Colleges and Universities. Trends in Higher Education Series
Author: Baum, Sandy
Date: 2010-00-00
Type: Reports - Descriptive
Subjects: Tuition,Student Financial Aid,Public Colleges,Private Colleges,Undergraduate Students,Income,Differences,Institutional Characteristics

Recent data from the College Board's Annual Survey of Colleges reveal significant variation in institutional aid patterns among colleges and universities. The undergraduate tuition discount rate--the ratio of institutional grant aid to published tuition and fee charges--is higher at private institutions than at public institutions, but there are also notable differences within sectors. It is not just the magnitude of the discounts offered that is important but also the purposes of the discounts and the distribution of the grants to students in different circumstances. Since 2000-01, the total discount rate has been relatively stable in the public sector but continues to rise in the private sector. Breaking the discount rate down into aid that meets financial need, aid that goes beyond financial need (or goes to students with no documented need), and the more specialized categories of tuition waivers and athletic awards. This report presents a summary of the report analysis and findings.
Improving the Design of Science Intervention Studies: An Empirical Investigation of Design Parameters for Planning Group Randomized Trials
Author: Westine, Carl
Date: 2013-00-00
Type: Reports - Research
Subjects: Randomized Controlled Trials,Research Design,Correlation,Program Effectiveness,Hierarchical Linear Modeling,Student Characteristics,Statistical Analysis,Pretests Posttests,Elementary Schools,Middle Schools,High Schools,Student Records,Scores,Achievement Tests

The capacity of the field to conduct power analyses for group randomized trials (GRTs) of educational interventions has improved over the past decade (Authors, 2009). However, a power analysis depends on estimates of design parameters. Hence it is critical to build the empirical base of design parameters for GRTs across a variety of outcomes and contexts. This study provides a first step towards building this base of design parameters specifically for science outcomes. Unlike reading and math, science is not typically tested each year. Preliminary findings from this study suggest that although not direct comparisons, unconditional intraclass correlations (ICCs) for science outcomes are smaller than grade 3 math and reading as reported by Bloom, Richburg-Hayes, and Black (2005; 2007) for five urban districts. Similarly, Hedges and Hedberg (2007) found larger ICCs for both reading and math using a nationally representative sample of students nested in schools. R-square values for school level covariates have not been computed yet, but a one year lagged student pretest appears to be highly effective in reducing variance between and within schools, more so than in reading (R-squares less than 0.86) and math (less than 0.63) as reported by Bloom et al. (2005; 2007). The empirical estimates from this study will help improve the accuracy of the power analyses for GRTs of science interventions. Tables are appended.
Writing Assessment in Admission to Higher Education: Review and Framework. College Board Report No. 99-3. GRE Board Research Report No. 96-12R
Author: Breland, Hunter M.
Date: 1999-00-00
Type: Reports - Research
Subjects: Writing Research,Writing Tests,Writing (Composition),Writing Instruction,Testing Programs,Standardized Tests,Test Bias,Individual Differences,Test Reliability,Test Validity,College Entrance Examinations,Graduate Study,Law Schools,Medical Schools,Language Tests,English (Second Language),Second Language Learning,Scoring,Multiple Choice Tests

A comprehensive review was conducted of writing research literature and writing test program activities in a number of testing programs. The review was limited to writing assessments used for admission in higher education. Programs reviewed included ACT, Inc.'s ACT™ program, the California State Universities and Colleges (CSUC) testing program, the College Board's SAT® program, the Graduate Management Admissions Test® (GMAT®) program, the Graduate Record Examinations ® (GRE®) test program, the Law School Admission Test® (LSAT®) program, the Medical College Admission Test® (MCAT®) program, and the Test of English as a Foreign Language™ (TOEFL®) testing program. Particular attention was given in the review to writing constructs, fairness and group differences, test reliability, and predictive validity. A number of recommendations are made for research on writing assessment. Tables Summarizing Previous Studies of Mean Differences, Reliability, and Validity are provided in the appendix.
Evaluation, Integration and Institutionalization of Initiatives to Enhance STEM Student Success
Author: Dickson, Lisa
Date: 2013-00-00
Type: Reports - Research
Subjects: STEM Education,Student Diversity,Academic Persistence,Graduation Rate,Program Effectiveness,Intervention,Calculus,Mathematics Instruction,Discussion (Teaching Technique),Mentors,Risk Assessment,Cooperative Learning,College Freshmen,Randomized Controlled Trials,Control Groups,Experimental Groups,Comparative Analysis

Many researchers, policymakers, and university administrators have called for more students to major in Science, Technology, Engineering and Mathematics (STEM) fields to address the critical need for a diverse and technically skilled workforce that is prepared to address national problems and educate its future STEM leaders. One of the questions facing the nation is how best to increase the number of students obtaining degrees in STEM fields. The University of Maryland, Baltimore County (UMBC) received support from the National Science Foundation (NSF) to comprehensively investigate cost effective and scalable strategies to increase the number of students graduating with STEM degrees. UMBC's Meyerhoff Scholarship Program has been recognized as a national model for increasing diversity among future leaders in science and engineering. The programs, as currently administered, can impact a limited number of students due to the cost. Also, it is unclear as to what individual facets of these programs matter most for student retention and graduation. The project's primary purpose is to identify techniques that significantly impact retention and graduation rates among students who intend to major in STEM disciplines, to verify the estimates of these impacts through careful analysis, and to assess all costs relative to the effectiveness of these interventions. Eligible students who signed the consent forms to participate were randomly assigned into five separate groups: (1) high-status faculty mentoring with ongoing risk assessment; (2) professional staff mentoring with ongoing risk assessment; (3) placement in an active learning discussion section in pre-calculus or calculus classes in the CNMS Active Science Teaching and Learning Environment (CASTLE); (4) assistance in forming and maintaining study groups; and (5) the control group that offers a $50 UMBC Bookstore gift card. Participation took place over the fall and spring semesters with Cohort 1 held during the 2011-2012 AY, Cohort 2 held during the 2012-2013 AY, and Cohort 3 held during the 2013-2104 AY. The project is in its third and final Cohort at the time of the writing of this paper. The remaining time of the five-year study is devoted to data analysis and dissemination.
Validity of the SAT for Predicting Second-Year Grades: 2006 SAT Validity Sample. Statistical Report No. 2011-1
Author: Mattern, Krista D.
Date: 2011-02-00
Type: Numerical/Quantitative Data
Subjects: College Entrance Examinations,Predictive Validity,Test Validity,Scores,Grades (Scholastic),Grade Point Average,Correlation,Institutional Characteristics,Student Characteristics,Public Colleges,Private Colleges,High School Students,College Students,Tables (Data)

This report presents the validity of the SAT for predicting two second-year outcomes: (1) second-year cumulative GPA (2nd Yr Cum GPA), and (2) second-year grade point average (2nd Yr GPA). Similar to the results for first-year grade point average (1st Yr GPA), the SAT is strongly correlated with second year outcomes. For many significant subgroups, such as ethnic minority students and female students, the SAT was in fact a better predictor of 2nd Yr Cum GPA and 2nd Yr GPA than were high school grades alone. However, for all students, SAT score in combination with high school grades was the best predictor of these second year outcomes since both measures provide incrementally validity over each other. For example, even within HSGPA levels, there is still a strong positive relationship between SAT and 2nd Yr Cum GPA and 2nd Yr. An appendix lists the institutions providing second-year data on the 2006 freshman cohort.
Improving Student Outcomes with mCLASS: Math, a Technology-Enhanced CBM and Diagnostic Interview Assessment
Author: Wang, Ye
Date: 2013-00-00
Type: Reports - Research
Subjects: Educational Technology,Technology Uses in Education,Mathematics Instruction,Intervention,Student Evaluation,Educational Assessment,Progress Monitoring,Interviews,Mathematics Skills,Computer Assisted Testing,Mathematics Achievement,Teaching Methods,Achievement Gains,Standardized Tests,Achievement Tests,Mathematics Tests,Scores,Grade 3,Elementary School Students,Grade 2,Comparative Analysis,Statistical Analysis

The No Child Left Behind Act resulted in increased school-level implementation of assessment-based school interventions that aim to improve student performance. Diagnostic assessments are included among these interventions, designed to help teachers use evidence about student performance to modify and differentiate instruction and improve student outcomes. The mCLASS: Math software (Ginsburg, Cannon, Eisenband, &amp; Pappas, 2006) is comprised of screening/progress monitoring curriculum-based measures (CBMs) and Diagnostic Interviews to help teachers identify students' skill levels. mCLASS: Math enables teachers to target instruction to each student's needs and monitor each student's progress toward mastery. Educators are expected to use the constantly updated diagnostic information to improve ongoing instruction and increase student achievement. Schools have found that the use of mCLASS: Math helps support curriculum, instruction, and assessment (Ginsburg et al, 2006). This study provides evidence about the effectiveness of mCLASS: Math in improving student outcomes on a statewide math achievement test, examining whether or not using the mCLASS: Math CBM impacts students' achievement test scores. Among the students who were administered the CBM, we investigate whether or not being administered the mCLASS: Math Diagnostic Interviews further influences subsequent test scores. This study was conducted in a large mid-western state which provided data from 175 districts, 606 schools, and 1856 teachers for this study. All the schools and teachers had elected to implement mCLASS: Math; however, there are instances in which individuals students may be exempted from the assessment. CBM measures quickly identify students who are at risk for poor mathematical performance while Diagnostic Interviews identify students' mathematical strengths and weaknesses which can be used to inform instruction. Overall, the administration of CBM measures provides broad, useful information about students' skill levels and identifies those students in need of further assessments and development. Tables are appended.
How Teacher-Scaffolded Summer Reading Improves Different Components of Reading Comprehension: Lessons Learned from Two Experimental Studies
Author: Kim, James S.
Date: 2013-00-00
Type: Reports - Research
Subjects: Scaffolding (Teaching Technique),Reading Instruction,Reading Comprehension,Reading Achievement,Summer Programs,Reading Programs,Reading Tests,Standardized Tests,Achievement Tests,Elementary School Students,Pretests Posttests,Program Evaluation,Program Effectiveness,Surveys

The problem of summer reading loss among low-income children has been amply documented by researchers. Over time, income-based disparities in reading achievement appear to grow more rapidly during the summer months than the academic school year. This paper describes two experiments in which teachers scaffold summer book reading with two different approaches to improving comprehension through end of school year lessons. Because teachers implemented different comprehension lessons in the two studies, the instructional activities varied across the two studies. Both studies used the same norm-referenced reading comprehension test (Iowa Test of Basic Skills), enabling a comparison of impacts on a similar measure across two studies. The purpose of this study is to capitalize on the lessons from the two experiments. Results from the 2006 experiment, which focused on comprehension strategy instruction, improved children's performance on comprehension tasks that required inference and interpretation (d = 0.15). Results from the 2011 experiment suggest that content-oriented comprehension instruction improved children's performance on tasks that required analysis and generalization (d = 0.11) compared to strategy-oriented instruction. In study 1, strategy lessons appeared to benefit Grade 4 children's ability to make inferences and interpretation. In study 2, content lessons appeared to benefit Grade 3 children's ability to make analyses and generalizations. Tables are appended.
Validity of the SAT for Predicting First-Year Grades: 2008 SAT Validity Sample. Statistical Report No. 2011-5
Author: Patterson, Brian F.
Date: 2011-02-00
Type: Numerical/Quantitative Data
Subjects: College Entrance Examinations,Predictive Validity,Test Validity,Scores,Grades (Scholastic),Grade Point Average,Correlation,Institutional Characteristics,Student Characteristics,Public Colleges,Private Colleges,High School Students,Tables (Data),College Freshmen

The findings for the 2008 sample are largely consistent with the previous reports. SAT scores were found to be correlated with FYGPA (r = 0.54), with a magnitude similar to HSGPA (r = 0.56). The best set of predictors of FYGPA remains SAT scores and HSGPA (r = 0.63), as the addition of the SAT sections to the correlation of HSGPA alone with FYGPA leads to a substantial improvement in prediction (?r = 0.07). This finding was consistent across all subgroups of the sample, by both institutional characteristics and demographics (?r = 0.06). All correlations presented here have been corrected for restriction of range, but the same basic patterns hold for the raw correlations. The following are appended: (1) Institutions Providing First-Year Outcomes Data for the 2008 Cohort; (2) Raw Correlations of SAT and HSGPA with FYGPA by Institutional Characteristics ; and (3) Raw Correlation of SAT Scores and HSGPA with FYGPA by Subgroups.
Examining Variation in Effects of Student Mobility Using Cross-Classified, Multiple Membership Modeling
Author: Rose, Bess A.
Date: 2013-00-00
Type: Reports - Research
Subjects: Program Effectiveness,Intervention,Student Mobility,Student Records,Correlation,Reading Achievement,Grade 6,Statistical Analysis

Research on the effectiveness of educational interventions usually is based on samples of students who remain in the same school over time. In contrast, most students transfer schools at least once during their K-12 school career, not including normative transfers such as those from elementary to middle school (Rumberger, 2002). Even when looking at just the two years prior to the 1998 NAEP, one-third of fourth graders, 19 percent of eighth graders, and 10 percent of twelfth graders had changed schools at least once (Rumberger, 2002). Mobility is higher among low-income and minority populations (Rumberger, 2002). While many studies have investigated the relationship of student mobility with achievement (Alexander, Entwisle, &amp; Dauber, 1996; Reynolds, Chen, &amp; Herbers, 2009; Rumberger &amp; Larson, 1998; Tucker, Marx, &amp; Long, 1998), the degree to which this relationship might vary among schools has not been fully investigated; in other words, are some schools more effective with mobile students than others? Data for the current study were obtained from a prior study of student mobility in a mid-Atlantic state that took place in 2001-2003. (A full description is available in Rogers, 2004.) This study examined complete school history data from a statewide sample of students in order to investigate the relationship between mobility and reading achievement in the sixth year of schooling. Cross-classified, multiple membership models were used to accurately account for students' membership in multiple schools during Year 6 as well as prior years. The relationship between mobility and reading scores was found to be non-significant on average, but examination of the variance components revealed that the impact of student mobility on reading achievement varied significantly among schools. Furthermore, the covariance estimate suggests that mobility gaps are especially large in schools with higher overall levels of achievement. This suggests that further research is necessary that more closely examines the contextual effects of mobility. Tables are appended.
The Validity of the Academic Rigor Index (ARI) for Predicting FYGPA. Research Report 2012-5
Author: Mattern, Krista D.
Date: 2012-00-00
Type: Numerical/Quantitative Data
Subjects: College Readiness,Grade Point Average,College Entrance Examinations,Scores,High School Students,College Students,Academic Standards,Difficulty Level,Academic Achievement,Predictive Validity,Measures (Individuals),Statistical Distributions,Prediction,Family Income,Correlation

A recurrent trend in higher education research has been to identify additional predictors of college success beyond the traditional measures of high school grade point average (HSGPA) and standardized test scores, given that a large percentage of unaccounted variance in college performance remains. A recent study by Wyatt, Wiley, Camara, and Proestler (2012) expanded the definition of college readiness beyond test scores and HSGPA to include a measure of the academic rigor or challenge associated with a student's course work in high school, referred to as the academic rigor index (ARI). This study represents the first examination of the validity of ARI in predicting first-year grade point average (FYGPA). The correlation between ARI and FYGPA indicated a moderate effect overall and by gender, ethnicity, and household income subgroups; however, ARI did not add incremental validity above SAT scores and HSGPA. Additionally, when added to SAT scores and HSGPA, ARI had no impact on differential prediction by relevant subgroups. Given the current movement toward a more holistic assessment of college applicants, a standardized measure of the academic rigor of a student's course load in high school suggests a promising additional measure to the assessment of a student's level of college readiness. Table A-1 is appended.
Implementation Challenges and Results
Author: Walters, Kirk
Date: 2013-00-00
Type: Reports - Research
Subjects: Summer Science Programs,Program Implementation,Barriers,Online Courses,Algebra,Student Surveys,Teacher Surveys,Observation,Content Analysis,Educational Practices

This paper describes the implementation of the online and f2f summer algebra courses that were delivered in summers 2011 and 2012. These data will be used to frame the impact results presented in an earlier paper. In particular, the paper will provide a detailed picture of how the online course was structured and the types of supports provided to students; compare the algebra content and course rigor between the online and f2f classes; and examine students' perceptions of the online course between summers 2011 and 2012, which were starkly different in terms of the software glitches that interfered with the delivery of the course in 2012. The paper will be guided by the following research questions: (1) How were the key components of the online algebra credit recovery course implemented during the summers of 2011 and 2012, including specific technology challenges?; (2) How did the online and f2f Algebra IB courses compare in terms of content rigor and grading expectations?; and (3) How did the instructional experience compare for students taking online and f2f summer credit recovery courses, and between students who took the online course in 2011 and 2012? The authors will draw from several different sources of implementation data that were collected in both conditions in both summer 2011 and 2012 to answer the paper's research questions. These include in-person observations of the online and f2f classrooms, archival data generated from the online course, student and teacher surveys, online mentor logs, and course materials (syllabi, annotated tables of contents). The preliminary findings from year 1 (summer 2011) indicated that students were engaged, cooperative and attentive in both the online and f2f algebra classes, with no statistically significant differences between conditions. The online mentors reported spending the majority of their time (62%) on administrative tasks, 28% of their time teaching mathematics; and 9% of their time communicating with the online teacher or Aventa technical support.