ethanporcaro/eric-fulltext-meta huggingface dataset:
Test Development and Technical Information on the Writing Section of the SAT Reasoning Test™. Research Notes RN-25
Author: Kobrin, Jennifer L.
Date: 2006-03-00
Type: Reports - Research
Subjects: Test Construction,Writing Tests,Cognitive Tests,College Entrance Examinations,Statistical Analysis,Test Reliability,Test Validity,Predictive Validity,Multiple Choice Tests,Essays,Scoring Formulas,Psychometrics,Scientific and Technical Information

Based on statistics from the first few administrations of the SAT writing section, the test is performing as expected. The reliability of the writing section is very similar to that of other writing assessments. Based on preliminary validity research, the writing section is expected to add modestly to the prediction of college performance when critical reading and mathematics scores are considered.
Substituting SAT® II: Subject Tests for SAT I: Reasoning Test: Impact on Admitted Class Composition and Quality. Research Report No. 2001-3. ETS RR-01-07
Author: Bridgeman, Brent
Date: 2001-00-00
Type: Reports - Research
Subjects: College Entrance Examinations,Grade Point Average,Racial Composition,Models,Comparative Analysis,Educational Quality,Selection Criteria,Achievement Tests,Aptitude Tests,Student Records,Data Analysis,College Freshmen

Using data from a sample of 10 colleges at which most students had taken both SAT I: Reasoning Test and SAT II: Subject Tests, the authors simulated the effects of making selection decisions using SAT II scores in place of SAT I scores. Specifically, they treated the students in each college as forming the applicant pool for a more select college, and then selected the top two-thirds (and top one-third) of the students using high school grade-point average combined with either SAT I scores or the average of SAT II scores. Success rates, in terms of freshman grade-point averages, were virtually identical for students selected by the different models. The percent of African American, Asian American, and White students selected varied only slightly across models. Appreciably more Mexican American and Other Latino students were selected with the model that used SAT II scores in place of SAT I scores because these students submitted Subject Test scores for the Spanish test on which they had high scores.
Setting Cut Scores for College Placement. Research Report No. 2005-9
Author: Morgan, Deanna L.
Date: 2005-00-00
Type: Reports - Descriptive
Subjects: Cutting Scores,Student Placement,Academic Standards,Scoring Formulas,Decision Making,Program Validation,Standard Setting,Evaluation Methods,College Administration,Evaluation Criteria,Regression (Statistics)

Due to the high stakes that my be attached to placement decisions, it is imperative that the placement process be as solid and defensible as possible. An integral part of the placement process is the identification and use of cut scores, the point(s) on the score scale that classify students into adjacent categories for placement decisions. This paper is geared toward helping college administrators make valid decisions regarding setting cut scores, focusing particularly on selecting a method, but also discussing issues such as defining performance levels and validating the process.
Reporting Randomized Controlled Trials in Education
Author: Mayo-Wilson, Evan
Date: 2014-00-00
Type: Reports - Research
Subjects: Educational Research,Randomized Controlled Trials,Intervention,Standards,Context Effect,Validity,Research Reports,Delphi Technique,Researchers

Randomized controlled trials (RCTs) are increasingly used to evaluate programs and interventions in order to inform education policy and practice. High quality reports of these RCTs are needed for interested readers to understand the rigor of the study, the interventions tested, and the context in which the evaluation took place (Mayo-Wilson et al., 2013). The Consolidated Standards of Reporting Trials (CONSORT) Statement--and its extensions--are the preeminent guidelines for reporting RCTs; they are based on empirical evidence and expert consensus about biases related to validity (Schulz, Altman, Moher, for the CONSORT Group, 2010). While CONSORT guidelines are well-known in the social and behavioral sciences, there is less evidence of widespread uptake and implementation in education compared with biomedical disciplines, and deficiencies persist in the reporting of intervention trials in education (Torgerson, Torgerson, Birks, &amp; Porthouse, 2005). Following recommended techniques for guideline development and dissemination (Moher et al., 2010), this presentation focuses on a structured research programme to develop a reporting guideline for intervention RCTs in education and related disciplines. Namely, this presentation will overview the reporting quality of RCTs in education, and discuss reporting guidance for reporting education intervention RCTs.
SAT® Writing: An Overview of Research and Psychometrics to Date. Research Notes. RN-32
Author: Mattern, Krista
Date: 2007-01-00
Type: Reports - Research
Subjects: College Entrance Examinations,Writing Tests,Educational Research,Psychometrics,Educational Quality,Test Reliability,Alignment (Education),Relevance (Education),Racial Distribution,Scores,Essay Tests,Test Validity,Correlation,Test Coaching

The focus of this report is to summarize the research that has been conducted thus far on the new SAT Writing section. The evidence provided reveals that the new writing section has satisfactory psychometric quality in that its reliability is acceptable; it is significantly related to first-year college GPA and college English grades; it has been perceived to impact high school curriculum; and it does not result in larger score disparities among racial/ethnic groups.
Using Generalizability Theory to Examine Sources of Variance in Observed Behaviors within High School Classrooms
Author: Abry, Tashia
Date: 2014-00-00
Type: Reports - Research
Subjects: Generalizability Theory,Teacher Behavior,Student Behavior,High School Students,Secondary School Teachers,Reliability,Outcome Measures,Randomized Controlled Trials,Positive Behavior Supports,High Schools,Observation

Generalizability theory (GT) offers a useful framework for estimating the reliability of a measure while accounting for multiple sources of error variance. The purpose of this study was to use GT to examine multiple sources of variance in and the reliability of school-level teacher and high school student behaviors as observed using the tool, Assessing School Settings: Interactions of Students and Teachers (ASSIST). The study took place in the context of a high school randomized controlled trial (Maryland Safe and Supportive Schools [MDS3]--58 high school randomized controlled trials funded by the U.S. Department of Education and the William T. Grant Foundation) of Positive Behavioral Interventions and Supports (PBIS) (Sugai &amp; Horner 2006). PBIS is a school-based decision-making framework designed to enhance students' social and academic functioning. Baseline data (collected prior to intervention) from six Maryland high schools provided the analytic sample for the study. Two raters worked in tandem to observe 25 classrooms/occasions in each of the six high schools using the ASSIST, resulting in a partially crossed design in which observations were nested within schools and crossed by rater. School interactions were assessed using the ASSIST, an observational measure of teachers' behavior management strategies and student behavior. Generalizability studies (G-studies) and decision studies (D-studies) were conducted using GENOVA software. Variations in the number of raters used to conduct observations and the number of observations occurring within a school had direct implications for the reliability of ASSIST tally items. Results indicate that raters can be trained to observe selected behaviors with high levels of agreement, and that studies using the ASSIST may be well-positioned to capture school-level differences for the behaviors of teacher proactive behavior expectations, teacher reactive behavior management, student disruption, and student profanity. This study demonstrates GT as a useful tool when evaluating and planning for the use of observational measures in large-scale program evaluations. Tables and figures are appended.
Summary of Evaluation Results of Advanced Placement Program® (AP®) Vertical Teams for English. Research Notes. RN-17
Author: College Entrance Examination Board
Date: 2002-09-00
Type: Reports - Research
Subjects: Advanced Placement Programs,Program Implementation,Teacher Surveys,Administrator Surveys,Program Effectiveness,Program Evaluation,Teamwork,Partnerships in Education,Educational Practices,Educational Principles,Educational Objectives,Faculty Development,Communities of Practice,English Teacher Education,Middle School Teachers

Teachers and administrators were surveyed to determine the extent to which AP Vertical Teams for English were implemented in schools. Results indicate that even though the goals of the AP Vertical Teams were only partially realized, there was some positive impact on student exposure to Pre-AP instruction and AP courses and student performance on the AP Examinations. Although the results are only modest, they are promising. It appears that if the AP Vertical Team goals were more fully implemented, the impact is likely to be more extensive. Suggestions for improving the implementation of AP Vertical Teams are provided.
Which Interventions Have the Greatest Effect on Student Learning in Sub-Saharan Africa? &quot;A Meta-Analysis of Rigorous Impact Evaluations&quot;
Author: Conn, Katharine
Date: 2014-00-00
Type: Reports - Research
Subjects: Foreign Countries,Intervention,Program Effectiveness,Outcome Measures,Meta Analysis,Learning,Outcomes of Education,Program Evaluation,Elementary Secondary Education,Effect Size

In the last three decades, there has been a large increase in the number of rigorous experimental and quasi-experimental evaluations of education programs in developing countries. These impact evaluations have taken place all over the globe, including a large number in Sub-Saharan Africa (SSA). The fact that the developing world is socially and economically diverse and that different interventions and programs are often culturally specific suggests that the broad results from previous syntheses may not be applicable to SSA, which is itself a quite diverse region. In this paper, the author conducts a large systematic review of the impact evaluation literature throughout the region, East, Central, Southern, and Western Africa, as well as the Indian Ocean Islands. This includes both experimental and quasi-experimental studies from all fields which evaluate the impact of education programs on student learning (fields include economics, health, psychology, and education). The author employed a random effects meta-analysis model. This meta-analysis found that interventions in pedagogical methods (particularly those that emphasize procedural or conceptual learning) to be very promising. Tables and figures are appended.
SAT® Subject Area Readiness Indicators: Reading, Writing, and STEM
Author: Wyatt, Jeffrey N.
Date: 2012-08-31
Type: Reports - Research
Subjects: College Entrance Examinations,Educational Indicators,College Readiness,Reading Tests,Writing Tests,STEM Education,Benchmarking,Career Readiness,Maximum Likelihood Statistics,Grade Point Average,Predictor Variables,Content Area Reading,Content Area Writing

In 2011, the College Board developed the SAT College and Career Readiness Benchmark to assist educators and policymakers in their efforts to better evaluate the college readiness of their students. This benchmark was designed to identify the point on the SAT score scale that is indicative of students' having a high likelihood of success in college, with &quot;success&quot; being defined as a 65% probability of obtaining a first-year GPA (FYGPA) of 2.67 (B-) or higher. This criterion was informed by a panel of expert educators and policymakers convened by the College Board in 2007 (Kobrin, Patterson, Wiley, &amp; Mattern, 2012).
SAT® II: Subject Tests in Foreign Languages--Using the Tests for Admission and Placement. Research Summary RS-07
Author: College Entrance Examination Board
Date: 2002-06-00
Type: Reports - Descriptive
Subjects: College Entrance Examinations,Second Languages,Language Tests,Student Placement,Academic Ability,Test Items,Difficulty Level,Test Content,Native Speakers,Academic Achievement

As an admissions test, the SAT II: Subject Tests in foreign languages allow students to demonstrate academic competence in the selected language area. As a placement tool, the SAT II: Subject Tests in foreign languages serve the same function as the SAT II: Subject Tests do in other academic areas such as world history, chemistry, or math. SAT II: Subject Tests also provide college faculty with a superior placement tool without using faculty time to develop, administer, or score the tests. Issues regarding the tests include test difficulty, test content, selection of test titles, and assessing students who are native speakers of the language.
Sex Differences in the Performance of High-Scoring Examinees on the SAT®-M. College Board Report No. 90-3. ETS RR No. 90-27
Author: Gallagher, Ann M.
Date: 1990-00-00
Type: Reports - Research
Subjects: College Entrance Examinations,Gender Differences,Scores,Mathematics Tests,Mathematics Skills,Test Items,Problem Solving,Mathematical Logic,Time,Mathematics Achievement,Verbal Tests,Verbal Ability,Comparative Analysis,Coding,High School Students,High School Seniors,Test Bias,Statistical Analysis

Performance of high-scoring males and females on the mathematics section of three forms of the College Board's Scholastic Aptitude Test (SAT-M) was examined to determine how item content, solution strategy, and speededness differentially affect performance. The mathematical and verbal sections of the SAT were also compared for similarities in the performance patterns of high scorers. Items on three forms of the SAT-M were classified on the basis of content and potential solution strategies. Male and female performance in the top 5 percent was examined using the Mantel-Haenszel Differential Item Functioning (MH-DIF) procedure. A main effect was found for itemsolution strategy, but not for item content. Males tended to outperform females on items requiring logical estimation or insight even when subjects were matched on years of mathematics course taking and average mathematics grades. Conventional measures indicated that the test was not differentially speeded. However, omit rates indicated that females omitted a greater proportion of items requiring estimation. Different patterns by gender were found on the mathematical and verbal sections of the test. Item Classification of SAT-M Sex Differences Study is appended.
Scoring the Essay on the SAT® Writing Section. Research Summary RS-10
Author: Camara, Wayne J.
Date: 2003-07-00
Type: Reports - Descriptive
Subjects: Scoring,Essay Tests,Holistic Approach,Scoring Formulas,Writing Tests,Scoring Rubrics,Evaluation Methods,College Entrance Examinations

The essay on the writing section of the SAT will be scored using a holistic approach. In holistic scoring, a piece of writing is considered as a total piece of work--the whole of which is greater than the sum of its parts.
Student Group Differences in Predicting College Grades: Sex, Language, and Ethnic Groups. College Board Report No. 93-1. ETS RR No. 94-27
Author: Ramist, Leonard
Date: 1994-00-00
Type: Reports - Research
Subjects: College Entrance Examinations,Scores,Grade Point Average,College Freshmen,Grades (Scholastic),Academic Ability,Predictor Variables,High School Students,Comparative Analysis,Course Selection (Students),Gender Differences,Language Usage,English,Racial Differences,Ethnic Groups,Institutional Characteristics,English (Second Language)

Part 1 of this study investigated possible causes of the observed decline in correlations between SAT scores and freshman grade-point average (FGPA). Working with a database of 38 colleges, the study found that the comparability of course grades received by entering freshmen declined in the 1980s. Three new measures of grade comparability-variety of courses taken, variation in average student aptitude among courses, and appropriateness of average course grade in relation to student aptitude level-proved to be excellent indicators of both the level of and the change in SAT validity for predicting FGPA. Using course grade as a criterion instead of FGPA reduced the decline in both SAT and high school GPA (HSGPA) validity for predicting course grades by 40 percent. Contrary to the assumption that high school record (HSR) is a better predictor than the SAT, compared with HSR the SAT had higher or equal average validities for predicting course grades in almost all categories of courses. Part 2 examined course selection, grading patterns, grade comparability, SAT predictive effectiveness and mean over- and underprediction across different courses for groups based on gender, English as best or not best language, and ethnicity. All results were analyzed by college selectivity level and size. The following are appended: (1) Colleges Participating in the Study; and (2) Course Categories.
Sex Differences in SAT® Scores. College Board Report No. 88-9. ETS RR No. 88-58
Author: Burton, Nancy W.
Date: 1988-00-00
Type: Reports - Research
Subjects: College Entrance Examinations,Gender Differences,Scores,Verbal Ability,Standardized Tests,Mathematics Achievement,Background,Student Characteristics,Student Experience,High School Students,Ethnic Groups,Racial Differences,Majors (Students),Socioeconomic Status,Regression (Statistics)

After adjusting for differences in background, women's average SAT-verbal scores were found to be higher than, or nearly equal to, men's. Although women's average SAT-mathematical scores after adjustment are still lower than men's, they are 25 points higher when adjusted for background. This report's analysis establishes that the background differences between men and women are significantly related to verbal and mathematical score differences. The research looked at samples of students in the high school classes of 1975, 1980, and 1985 who had taken the SAT.
Students with Discrepant High School GPA and SAT® I Scores. Research Notes. RN-15
Author: Kobrin, Jennifer L.
Date: 2002-01-00
Type: Reports - Research
Subjects: College Entrance Examinations,Grade Point Average,Prediction,Scores,Correlation,High School Students,Critical Reading,Calculators,Predictive Validity,Regression (Statistics),Student Characteristics

The SAT® and high school grades are the most accurate predictors of first-year college performance. Together, these two measures have a high multiple correlation (r = 0.7) with first-year college grades when the correlation coefficient is corrected for restriction in range, criterion unreliability, and different course grading standards (Bridgeman, McCamley-Jenkins, and Ervin, 2000; Ramist, Lewis, and McCamley-Jenkins, 1993). The current study examined predictions of first-year college performance for students whose SAT score and high school grades were discrepant and compared these results to predictions for students whose SAT score and high school grades were not discrepant. The current analyses replicated those performed in an earlier study conducted in 1990 by Baydar but utilized a larger sample of schools and recentered SAT scores. In addition, the current study used SAT scores that emphasized critical reading, grid-in math items, and calculator use, and de-emphasized antonym items (Bridgeman, McCamley- Jenkins, and Ervin, 2000). Results of the study suggest that students with a high HSGPA (high school grade point average) in the presence of low SAT scores will not do any better in college than students with lower HSGPA scores but higher SAT scores. Therefore, the SAT may be a more accurate predictor than the HSGPA for these students.
Swimming against the Tide: The Poor in American Higher Education. Research Report No. 2001-1
Author: Terenzini, Patrick T.
Date: 2001-00-00
Type: Reports - Research
Subjects: Higher Education,Low Income Groups,Socioeconomic Influences,High School Students,Student Financial Aid,Expectation,Knowledge Level,Parents,Access to Information,Enrollment,Fiscal Capacity,College Choice,Student Characteristics,Outcomes of Education,Labor Market

Despite an enormous investment in equalizing educational opportunities for all Americans, substantial evidence indicates that significant inequities remain, particularly for low-socioeconomic-status (SES) students. The report draws on an extensive review of the current research literature and contributes new analyses of national databases to fill in some of the holes in the existing literature. Among the findings are: (1) by the ninth grade, most students have developed occupational and educational expectations that are strongly related to SES; (2) parents' knowledge of financial aid, financial planning for college, and students' access to college and financial aid information are clearly associated with SES; and (3) nearly one-half of the lowest-SES-quartile high school graduates do not enroll the following fall in any postsecondary institution, a nonenrollment rate nearly five times higher than that of high-SES students. Two appendices are included: (1) Definitions of Wealth: A Review of the Literature; and (2) Sources of Data and Methodology.
Score Change for 2007 PSAT/NMSQT® Test-Takers: An Analysis of Score Changes for PSAT/NMSQT Test-Takers Who Also Took the 2008 PSAT/NMSQT Test or a Spring 2008 SAT® Test. Research Notes RN-41
Author: Proctor, Thomas P.
Date: 2010-08-00
Type: Reports - Research
Subjects: Scores,Achievement Gains,Bayesian Statistics,College Entrance Examinations,Statistical Analysis,Predictive Measurement,Gender Differences,Racial Distribution,Tables (Data),Statistical Distributions,High School Students

The purpose of this paper is to provide information about how students' scores change when they retake the PSAT/NMSQT as juniors or take the SAT in the spring after they take the PSAT/NMSQT as juniors. Two research questions guided this study and motivated the approach for analysis of the data: How do scores change for students who took the PSAT/NMSQT both as a sophomore and junior or the PSAT/NMSQT and SAT as juniors? How will future students taking the PSAT/NMSQT be expected to perform when they take the SAT or retake the PSAT/NMSQT? Tables and figures are appended.