ethanporcaro/eric-fulltext-meta huggingface dataset:
The Promises and Challenges of Implementing Evidence-Centered Design in Large Scale Assessment
Author: Huff, Kristen
Date: 2009-04-00
Type: Reference Materials - General
Subjects: Measurement,Evidence Based Practice,Test Construction,Advanced Placement Programs,Achievement Tests

Presented at the Annual Meeting of National Council on Measurement in Education (NCME) in San Diego, CA in April 2009. This presentation provides an overview of ECD. In addition, the presentation describes the benefits of, as well as the challenges that were faced, in implementing ECD in the Advanced Placement Program.
&quot;Almost&quot; 31 Flavors of Multi-Level Modeling in SAS
Author: Patterson, Brian F.
Date: 2008-12-09
Type: Reference Materials - General
Subjects: Hierarchical Linear Modeling,Statistics,Computer Software,College Entrance Examinations,Achievement Tests,Advanced Placement Programs,Research Problems

Presented at the quarterly meeting of the New York Area SAS Users-Group in December 2008. In virtually any research context, there is likely to be opportunity to apply mixed-models--of which repeated-measures, multilevel, hierarchical and random- and fixed-effect models are specific types-- that improve upon their traditional linear or generalized linear model counterparts. This presentation focused primarily on multi-level modeling and on drawing parallels between the statistical specification of these models and their implementation in SAS. Some time was devoted to the discussion of problems of convergence, some diagnostics and other impediments to the modeling process.
The Cost of Unprepared Students
Author: Vasavada, Natasha
Date: 2011-10-25
Type: Reference Materials - General
Subjects: College Preparation,Course Selection (Students),Common Core State Standards,Educational Attainment,Success,Career Readiness,College Readiness,College Entrance Examinations,Educational Quality,Grade Point Average,Racial Differences,Ethnic Groups,Academic Standards,Alignment (Education)

Presented at the College Board National Forum, October 2011. In this session on academic preparedness and the Common Core, presenters summarized research on academic rigor and current course taking patterns for students and discussed how the new common core state standards will impact curriculum and expectations for student coursework.
Using the Many-Facet Rasch Model to Evaluate Standard-Setting Judgments: Setting Performance Standards for Advanced Placement® Examinations
Author: Kaliski, Pamela
Date: 2012-04-15
Type: Non-Print Media
Subjects: Advanced Placement Programs,Achievement Tests,Item Response Theory,Models,Standard Setting,Science Tests,Test Items,Difficulty Level,Standards,Validity

The Many-Facet Rasch (MFR) Model is traditionally used to evaluate the quality of ratings on constructed response assessments; however, it can also be used to evaluate the quality of judgments from panel-based standard setting procedures. The current study illustrates the use of the MFR Model by examining the quality of ratings obtained from a standard setting employing the Multiple Yes-No (MYN) modified Angoff standard setting procedure to set the cuts on the Advanced Placement® (AP®) Environmental Science exam. The locations of the panelists, judged item difficulties, rounds, and performance standards using the MFR Model were examined. Results revealed evidence of acceptable quality of the ratings obtained from the panelists. Thus, the current study provides an important contribution of validity evidence for the AP Environmental Science standard setting, as well as an illustration of the utility of the MFR Model to evaluate standard setting judgments.
AP® Biology Exam
Author: Reshetar, Rosemary
Date: 2012-09-13
Type: Reference Materials - General
Subjects: Advanced Placement,Biology,Science Tests,Science Education,Educational Strategies,Evidence Based Practice,Scientific Methodology,Student Educational Objectives,Review (Reexamination),Science Process Skills,Knowledge Level,Test Items,Scoring,Measurement,Models,High School Students

On 9/13/12, the Workshop on Developing Assessments to Meet the Goals of the 2012 Framework for K-12 Science Education was held at the National Academies of Science. The workshop was organized and led by the NRC Committee on Developing Assessments of Science Proficiency in K-12 (co-chaired by James Pellegrino and Mark Wilson) and targeted to state assessment leaders. Rosemary represented the College Board as one of three panelists in a large-scale assessment session focused on assessments that assess science practices in conjunction with core ideas and cross cutting concepts, similar to those depicted in the Framework. Focus was on how these strategies can be used to measure learning as envisioned in the Framework. The revised AP Biology Course and Exam which went into effect this fall were presented as an example of how College Board utilized the Evidence Centered Design process for the course and exam review. Of particular interest was the integration of content knowledge with science practices into the learning objectives of the course, and ultimately into the exam to be offered in May.
Multilevel Issues in the Application of Propensity Score Matching: Simulations and Results for Evaluation of the Advanced Placement Program
Author: Patterson, Brian F.
Date: 2012-04-17
Type: Non-Print Media
Subjects: Probability,Scores,Statistical Bias,High School Students,College Students,Intervention,Simulation,Program Evaluation,Advanced Placement,Advanced Placement Programs,Grades (Scholastic),Multivariate Analysis,Predictor Variables,Academic Achievement

The main goal of this study was to illustrate and provide some direction for dealing with the complexities of propensity score matching within different multilevel contexts. Special attention is given to how procedures typically applied in a non-hierarchical setting may be modified to properly reduce the expected bias in the estimated treatment effect of a high school-level intervention on college-level outcomes. In particular, students self-selected into a high school level intervention and the outcome of interest was observed within the context of the college or university in which students subsequently enrolled. A simulation was prepared to delineate the relevant issues and to demonstrate the effects of ignoring a hierarchical nesting of subjects. An empirical example of an evaluation of a large-scale, national educational program is given.
Comparing State SAT Scores Using a Mixture Modeling Approach
Author: Kim, YoungKoung Rachel
Date: 2009-04-15
Type: Reference Materials - General
Subjects: Conference Papers,Scores,Comparative Analysis,College Entrance Examinations,Identification,Item Response Theory,Models,Goodness of Fit,Statistical Analysis,Classification,Class Size,Educational Indicators,Institutional Characteristics

Presented at the national conference for AERA (American Educational Research Association) in April 2009. The large variability of SAT taker population across states makes state-by-state comparisons of the SAT scores challenging. Using a mixture modeling approach, therefore, the current study presents a method of identifying subpopulations in terms of SAT scores while taking into account the percentage of SAT takers so that state SAT scores within a subpopulation can be properly compared.
Realizing the Dream: Using Data to Increase Achievement
Author: Lee, John
Date: 2008-07-30
Type: Reference Materials - General
Subjects: African American Students,Student Participation,Performance,Advanced Placement Programs,Data,Information Utilization,Academic Achievement,Success,College Entrance Examinations,Scores,High School Students

Presented at the Knowledge Is Power Program (KIPP) School Summit in San Antonio, TX in July 2008. This presentation explores the participation and performance of African American students in the SAT and AP programs and use data to inform ways to better support and encourage achievment and college success.
A Case for Not Going SAT-Optional: Students with Discrepant SAT and HSGPA Performance
Author: Mattern, Krista D.
Date: 2010-05-03
Type: Reference Materials - General
Subjects: College Entrance Examinations,Grade Point Average,High School Students,Scores,Standardized Tests,College Admission,Student Characteristics,Racial Differences,Ethnic Groups,Predictive Validity,College Freshmen,Student Records,Questionnaires,Gender Differences,Family Income

Presented at the national conference for the American Educational Research Association (AERA) in 2010. This presentation describes an alternative way of presenting the unique information provided by the SAT over HSGPA, namely examining students with discrepant SAT-HSGPA performance.
An Exploration of L1, L2, and Bilingual Students' Writing Features on the SAT Essay
Author: Kobrin, Jennifer L.
Date: 2012-04-14
Type: Non-Print Media
Subjects: Bilingualism,English (Second Language),Second Language Learning,Scores,Regression (Statistics),Writing Tests,Prior Learning,College Entrance Examinations,Cultural Background,Language Proficiency,Role,Essays,Language Styles,Native Language,Language Dominance,Prompting,Writing Evaluation,Language Usage,Persuasive Discourse,Test Construction,Coding,Cultural Differences,Writing Instruction,High School Students

It is well-documented that students' prior knowledge, cultural background, and language proficiency play a role in how they read, interpret, and respond to writing tasks (Barkaoui, 2007; Connor &amp; Kramer, 1995; Hinkel, 2002). Essays written by students from different language backgrounds often differ in their linguistic, stylistic, and rhetorical characteristics and these features may affect the scores that students receive. This study explored the features of essays written on the SAT by students for whom English was not their best language (L2 students) and bilingual students who reported both English and another language as their best language, compared to students for whom English was solely their best language (L1 students). A sample of essay responses on 14 different prompts administered between October 2005 and January 2006 were coded on a variety of features including number of words, use of first-person voice, use of a five-paragraph structure, and types of examples offered (e.g., scholarly or personal experience). Multilevel logistic regression analyses indicated that L2 students had greater odds of using first-person and using personal experience in their essay responses; and had lesser odds of taking a mixed argument approach in responding to the essay prompt. There was substantial variability in the frequency of essay features by language group across prompt, suggesting that different prompts elicit responses with different features. The implications for test development and student instruction are discussed.
Use of the SAT in Advisement and Placement: What the Research Tells Us
Author: Shaw, Emily J.
Date: 2009-02-23
Type: Reference Materials - General
Subjects: College Entrance Examinations,Test Validity,Standardized Tests,Student Placement,College Bound Students,High School Students,College Admission,Academic Persistence,College Students,Grade Point Average,Correlation,Predictor Variables,Scores,Grades (Scholastic),Mathematics Instruction,English Instruction,At Risk Students

Presented at the College Board Western Regional Office (WRO) Forum in San Diego in February 2009. This presentation explores SAT test validity and how it can be used to inform both advisement and placement at the national or institution level. Institutional validity can be measured using the Admitted Class Evaluation Service (ACES) which schools can request.
Landscape of Higher Education: Human Capital
Author: Edwards, Kelcey
Date: 2012-10-26
Type: Non-Print Media
Subjects: Higher Education,Educational Trends,Human Capital,Enrollment Trends,Academic Persistence,College Transfer Students,Outcomes of Education,Education Work Relationship,College Entrance Examinations,Achievement Tests,Student Costs,Advanced Placement Programs

There are many changes occurring in higher education, and often it is difficult to see trends nationally. In this presentation, experts working with national data present trends in higher education from a forward-looking perspective. Senior researchers from the College Board provide trends using College Board SAT and AP data, as well as new research around Net Price Calculators. The director of the National Center for Higher Education Management Systems Information Center provides information from national data systems that can help higher education policymakers and analysts. The research director of the National Student Clearinghouse offers trends in enrollment and persistence, with a special focus on transfer students. [The following slide presentations are presented within this slide deck: (1) Landscape of Higher Education: Net Price (Greg Perfetto); (2) The Changing Landscape of How We Understand Postsecondary Pathways--With a Focus on Transfer Students and Persistence Anywhere (Don Hossler); and (3) The College Completion Agenda: From &quot;Education for Education's Sake&quot; to Return on Investment and Gainful Employment (Patrick J. Kelly).
Efforts to Define College Readiness in the United States
Author: Patelis, Thanos
Date: 2009-05-00
Type: Reference Materials - General
Subjects: College Readiness,Definitions,Nonprofit Organizations,Educational Indicators,Research and Development,Validity,Predictor Variables,Academic Achievement,College Entrance Examinations,Aptitude Tests,Benchmarking,Regression (Statistics)

Presented at the 11th International Conference on Education (ICE) in 2009. This presentation provides an overview of the College Board and its efforts to develop college readiness indicators. The information provided includes a conceptual overview of the indicators and the research and development components in the development and validation of the indicators.
Representing Targets of Measurement within ECD
Author: Ewing, Maureen
Date: 2009-04-00
Type: Reference Materials - General
Subjects: Evidence Based Practice,Advanced Placement Programs,Achievement Tests,Test Construction,Measurement,Test Content,Skills,Course Content

Presented at the Annual Meeting of National Council on Measurement in Education (NCME) in San Diego, CA in April 2009. This presentation describes the methodology that was used with subject-matter experts (SMEs) to articulate the content and skills important in the domain, and then the iterative processes that were used to articulate the claims and evidence to represent the targets of instruction for AP courses, and by extension, the targets of measurement for the AP exams.
Validating Performance Level Descriptors (PLDs) for the AP® Environmental Science Exam
Author: Reshetar, Rosemary
Date: 2012-07-05
Type: Non-Print Media
Subjects: Advanced Placement Programs,Science Tests,Achievement Tests,Classification,Pilot Projects,Test Validity,Expertise,Statistical Analysis

This presentation summarizes a pilot study conducted after the May 2011 administration of the AP Environmental Science Exam. The study used analytical methods based on scaled anchoring as input to a Performance Level Descriptor validation process that solicited systematic input from subject matter experts.
Pathways to Success: Participation and Performance for Latino Students in the 2007 SAT® Cohort
Author: Geller, Martha
Date: 2008-05-00
Type: Reference Materials - General
Subjects: College Entrance Examinations,Hispanic American Students,Student Participation,Performance,High School Students,Student Characteristics,Scores,Academic Aspiration,College Preparation

Presented at the Annual Prepárate Conference in Chicago, IL in May 2008. This presentation explores the participation and performance of Latino students in the SAT program and attempt to find a way to find how to better support and encourage students in striving for success.
Data-Based Decision Making: The Road to AP Equity
Author: Edwards, Kelcey
Date: 2012-07-20
Type: Non-Print Media
Subjects: Data,Decision Making,Advanced Placement Programs,Equal Education,Student Participation,Performance,Racial Differences,Ethics,Data Interpretation,High School Students,High School Graduates

Presented at the Advanced Placement Annual Conference (APAC) in Lake Buena Vista, FL in July 2012. This presentation reviews concepts central to achieving equitable AP access and success for all willing and academically prepared students. We analyze trends in participation and performance by race/ethnicity from the AP Report to the Nation and consider the ethics of data in order to understand challenges to interpretation as well as behavioral consequences of various metrics. The intended audience is AP Coordinators, school administrators, and those who recognize the importance of building equity from the ground up.
AP Report to the Nation: A Closer Look at the Nation and Florida
Author: Sawtell, Ellen A.
Date: 2012-07-20
Type: Non-Print Media
Subjects: Advanced Placement Programs,STEM Education,Student Participation,Racial Differences,Ethnic Groups,Hispanic American Students,Trend Analysis,Equal Education,Student Needs,At Risk Students,Public Schools,School Districts,High School Students,High School Graduates,Scores,Minority Group Students,African American Students,American Indian Students,Alaska Natives,Low Income Groups,Spanish Speaking,Language Usage,Asian American Students,White Students,Multiracial Persons,English (Second Language)

In February 2012, the College Board published The 8th Annual AP Report to the Nation. This session provides a deeper dive into key information for the United States with an emphasis on Florida, and participants hear how one school in Florida utilizes AP Potential™ to help build their AP Program. Participants also learn about AP participation and performance in the STEM field. Analyses by race/ethnicity is provided. Participants also gain a clear understanding of common questions centered around the growing Hispanic/Latino population in AP. The goal of this session is to provide participants with the knowledge of how the AP population has changed over the last 10 years. At the end of this session, administrators, counselors and other school personnel are better able to help identify students not currently being served by AP and understand how the available data can assist in building their AP program.
The Effect of Anchor Test Construction on Scale Drift
Author: Antal, Judit
Date: 2010-04-03
Type: Reference Materials - General
Subjects: Test Items,Equated Scores,Test Construction,Simulation,College Administration,College Entrance Examinations,Data Analysis,Evaluation Criteria

Presented at the Annual Meeting of National Council on Measurement in Education (NCME) in 2010. The focus of the research is to investigate the effect of applying the Sinharay &amp; Holland (2007) midi-test idea for building anchor tests to an on-going testing program with a series of versions of the test and comparing these results to the more commonly applied miniature anchor test assembly method. Overall, the results of our study support the proposal by Sinharay and Holland under very strict item and simulee generation method as the amount and degree of equating error observed did not appear to be significantly impacted by the change in the anchor test characteristics.
Understanding Students with Discrepant Critical Reading and Writing Scores
Author: Shaw, Emily
Date: 2009-04-14
Type: Non-Print Media
Subjects: Critical Reading,Reading Achievement,Writing Achievement,Scores,Demography,Grade Point Average,College Freshmen,Academic Persistence,School Holding Power,Performance Factors,Replication (Evaluation),Student Characteristics,Reading Tests,Writing Tests,Aptitude Tests,Predictive Validity,Correlation,Predictor Variables,Outcomes of Education,College Entrance Examinations

Presented at the national conference for AERA (American Educational Research Association) . This study examined whether there are distinct differences in the demographic characteristics, HSGPA, first-year college performance, and second-year college retention rates among students who have discrepant CR-W performance.
The Politics, Psychometrics and Promise of the New SAT Writing Section
Author: Wiley, Andrew
Date: 2009-04-00
Type: Non-Print Media
Subjects: Aptitude Tests,Writing Tests,Test Construction,Test Format,Test Items,Psychometrics,Essay Tests,Alignment (Education),High School Graduates,Program Implementation,Predictive Validity

Presented at the national conference for the American Educational Research Association (AERA) in 2009. This discussed the development and implementation of the new SAT writing section.
It's Major! College Major Selection &amp; Success
Author: Byers, Jenny
Date: 2011-10-26
Type: Non-Print Media
Subjects: Majors (Students),Decision Making,High School Students,Intervention,Best Practices,STEM Education,Academic Persistence,Advanced Placement Programs,College Bound Students,Academic Advising,College Students,College Entrance Examinations,Questionnaires,Databases,Standardized Tests,Student Characteristics,Intention

Presented at the College Board National Forum, October 26, 2011. Choosing a college major is challenging enough, without stopping to consider the impact it has on a student's college experience and career choice. To provide support during this major decision, participants in this session will develop strategies to facilitate students in making an appropriate major choice. Based on research and recommendations from high school and college educators, participants will discuss interventions and best practices. Participants will focus on factors related to major choice and retention (with an emphasis on STEM majors), and will investigate some noteworthy findings related to students with undeclared majors.
The Validity of the SAT for Predicting Cumulative Grade Point Average by College Major
Author: Shaw, Emily J.
Date: 2011-04-10
Type: Non-Print Media
Subjects: College Entrance Examinations,Predictive Validity,Grade Point Average,College Students,Majors (Students),Student Characteristics,Grading,School Culture,Gender Differences,Racial Differences,Ethnic Groups,Parent Background,Educational Attainment,Scores,High School Students,Academic Persistence

Presented at the Annual Meeting of the American Educational Research Association (AERA) in New Orleans, LA in April 2011. The current study examined the differential validity of the SAT for predicting cumulative GPA through the second-year of college by college major, as well as the differential prediction of cumulative GPA by college major among student subgroups. The relationship between the SAT and cumulative GPA varied somewhat by major, as well as by major and subgroup, likely due to differences in the nature of the college coursework, grading practices, student self-selection, and differing academic cultures (e.g., male dominated or highly competitive) across majors.
Exploring Approaches to Constructing Comparable Groups within Hierarchies
Author: Finkelstein, Doreen
Date: 2009-04-15
Type: Non-Print Media
Subjects: Comparative Analysis,Statistical Analysis,Computation,Probability,Regression (Statistics)

Presented at the Annual Meeting of the American Educational Research Association (AERA) in April 2009. Compares results of different approaches to propensity-score matching with hierarchical data.