ethanporcaro/eric-fulltext-meta huggingface dataset:
Evidence-Centered Design as a Foundation for ALD Development
Author: Plake, Barbara S.
Date: 2009-04-00
Type: Non-Print Media
Subjects: Achievement,Classification,Evidence,Test Construction,Test Items,Measurement,Design,Scores,Test Interpretation

[Slides] presented at the Annual Meeting of National Council on Measurement in Education (NCME) in San Diego, CA in April 2009. This presentation discusses a methodology for directly connecting evidence-centered assessment design (ECD) to score interpretation and use through the development of Achievement level descriptors.
Score Resolution in Essay Grading: A View from a Signal Detection Model of Rater Behavior
Author: DeCarlo, Lawrence T.
Date: 2008-03-00
Type: Non-Print Media
Subjects: Essays,Grading,Evaluation Criteria,Scoring Rubrics,Evaluators,Interrater Reliability,Scores,Simulation,Aptitude Tests,Test Results,Accuracy,College Entrance Examinations

[Slides] presented at the American Educational Research Association (AERA) Conference in New York in March 2008. This presentation explores what cues are used as a deciding factor in essay scoring by the essay grader.
Claims, Evidence and Achievement Level Descriptors as a Foundation for Item Design and Test Specifications
Author: Hendrickson, Amy
Date: 2009-04-00
Type: Non-Print Media
Subjects: Test Items,Test Construction,Evidence,Achievement,Student Evaluation,Measurement,Evaluation Methods,Scores,Inferences,Data Collection,Models,Validity,Achievement Rating

[Slides] presented at the Annual Meeting of National Council on Measurement in Education (NCME) in San Diego, CA in April 2009. This presentation describes how the vehicles for gathering student evidence--task models and test specifications--are developed.
The Role of Language Arts in a Successful STEM Education Program
Author: DeBoer, George
Date: 2010-10-00
Type: Non-Print Media
Subjects: Role,Language Arts,STEM Education,Standards,Teaching Methods,Science Instruction,Instructional Effectiveness,Academic Discourse,Literacy,Correlation,Modeling (Psychology),Core Curriculum,Elementary Secondary Education,Common Core State Standards

[Slides] presented at the College Board National Forum in Washington, D.C., October 2010. This presentation examines the implementation of Common Core Standards for English Language Arts in STEM classrooms and the implications for effective teaching of STEM subjects.
Investigating Grade Inflation and Non-Equivalence
Author: Godfrey, Kelly E.
Date: 2009-00-00
Type: Non-Print Media
Subjects: Grade Inflation,High School Students,Aptitude Tests,Scores,Grading,Evaluation Criteria,Advanced Placement,Longitudinal Studies,Multiple Regression Analysis,Academic Achievement,Secondary School Teachers,Grade Point Average,High School Graduates,Verbal Ability,Biology,Mathematics,Calculus,English Literature,Language Arts,History,Standardized Tests

[Slides] presented at the American Educational Research Association (AERA) in 2009. This study investigated the presence of a general rise in high school grades without a corresponding rise in SAT scores, as well as different grading criteria between schools, as shown by differential relationships between AP course grades and exam scores. These findings were put into context of conducting large-scale longitudinal educational research and its implications.
Evaluating a Strategy for Setting Cut Scores on a Computer Adaptive Test
Author: Morgan, Deanna L.
Date: 2011-00-00
Type: Non-Print Media
Subjects: Cutting Scores,Computer Assisted Testing,Adaptive Testing,Common Core State Standards,Feedback (Response),Measurement

[Slides] presented at the annual conference of the National Council on Measurement in Education, 2011, New Orleans.
College and Career Readiness: An Initial Validation Argument
Author: Camara, Wayne
Date: 2011-06-00
Type: Non-Print Media
Subjects: College Readiness,Career Readiness,Benchmarking,Student Evaluation,State Standards,Mathematics,English,Language Arts,College Credits,Evidence,Educational Testing,Psychological Testing,Test Validity,Construct Validity,Regression (Statistics),College Entrance Examinations,Academic Standards,Difficulty Level,Sciences,Social Sciences,Second Language Learning,Correlation

This presentation was presented at the 2011 National Conference on Student Assessment (CCSSO). The focus of this presentation is how to validate the common core state standards (CCSS) in math and ELA and the subsequent assessments that will be developed by state consortia. The CCSS specify the skills students need to be ready for post-secondary career training or college credit courses, yet much of the evidence has been based on professional judgments and reviews of standards and assessment frameworks specified by states and international assessments. A comprehensive validation strategy requires multiple sources of convergent evidence. This presentations reviews the sources of such evidence, the weight that should be attributed to such evidence and some preliminary thinking about how to proceed in developing a validation framework. The College Board's college readiness benchmarks, and how they have been empirically developed, are also discussed.
Using Think Aloud Interviews in Evidence-Centered Assessment Design for the AP World History Exam
Author: Kaliski, Pamela
Date: 2011-04-00
Type: Non-Print Media
Subjects: World History,Advanced Placement Programs,Achievement Tests,Protocol Analysis,Interviews,Evidence Based Practice,Test Construction,Models,Test Validity,Thinking Skills,Difficulty Level,Test Items

Developing a cognitive model of task performance is an important and often overlooked phase in assessment design; failing to establish such a model can threaten the validity of the inferences made from the scores produced by an assessment (e.g., Leighton, 2004). Conducting think aloud interviews (TAIs), where students think aloud while completing and interacting with a task, is an ideal approach for developing a cognitive model of task performance. The objective of this research was to pilot the use of think aloud interviews (TAIs) to inform item development and strengthen the validity argument for a new AP World History exam that is currently being developed. The results indicate that we are seeing evidence in students' verbal reports of intended historical thinking skills. Also, there are several different features (e.g., degree of familiarity) that contribute to the perceived difficulty of these items. [Slides] presented at the Annual Meeting of the American Educational Research Association (AERA) and National Council on Measurement in Education (NCME) in New Orleans, LA in April 2011.
Key Indicators of College Success: Predicting College Enrollment, Persistence, and Graduation
Author: Godfrey, Kelly E.
Date: 2010-04-00
Type: Reference Materials - General
Subjects: Educational Indicators,College Students,Predictor Variables,Academic Achievement,Success,High School Students,College Bound Students,Academic Persistence,Hierarchical Linear Modeling,Graduation,Student Characteristics,Definitions,Statistical Data,Statistical Analysis

[Slides] presented at AERA in Denver, CO in April 2010. In today's education climate, an enormous amount of pressure has been placed on states, school districts, and programs to produce graduates who are prepared to successfully enter, persist through, and graduate from the nation's universities and colleges. However, much of this research has been limited to one school or has focused on publicly available variables, thus limiting the analyses' generalizability and relevance. This study investigates the power of various student-level background and academic variables as well as school-level social and academic characteristics in the prediction of college enrollment, persistence, and graduation, demonstrating that characteristics at both levels play a role in the likelihood of reaching these goals. By creating this prediction model, researchers hope to further the field's understanding of the impact that a student's high school experience has on his or her likelihood of successfully navigating through college immediately following high school graduation.
Aligning Items and Achievement Levels: A Study Comparing Expert Judgments
Author: Kaliski, Pamela
Date: 2011-04-00
Type: Non-Print Media
Subjects: Alignment (Education),Academic Achievement,Expertise,Evaluative Thinking,Achievement Tests,Multiple Choice Tests,Test Construction,Instructional Design,Pilot Projects,Test Items,Item Analysis,Comparative Analysis,Difficulty Level,Summative Evaluation,Educational Assessment,High School Students,Student Evaluation,Social Sciences,History,Evaluation Methods,Data Collection,Advanced Placement,Taxonomy

For educational achievement tests that employ multiple-choice (MC) items and aim to reliably classify students into performance categories, it is critical to design MC items that are capable of discriminating student performance according to the stated achievement levels. This is accomplished, in part, by clearly understanding how item design features align with each of the achievement level descriptions. Difficulty drivers are defined as specific item features that are related to the difficulty of an item. The purpose of this research study was to pilot a methodology for working with subject matter experts to identify what item features drive item difficulty, and to determine specific features that drive item difficulty on a summative large scale assessment used in high school social science and history classrooms. Results of the study indicated that although this methodology was useful for gathering such information, modifications could be made in the future to better elicit the desired information about difficulty drivers. [Slides presented at the Annual Meeting of the American Educational Research Association (AERA) and National Council on Measurement in Education (NCME) in New Orleans, LA in April 2011.]
Design and Evaluation of Mixed-Format Large Scale Assessments for the Advanced Placement Program® (AP®)
Author: Reshetar, Rosemary
Date: 2010-04-00
Type: Speeches/Meeting Papers
Subjects: Psychometrics,Test Construction,Multiple Choice Tests,Test Items,Test Validity,Test Bias,Test Reliability,Scores,Responses,Advanced Placement Programs,Measurement,High School Students,College Bound Students,Scoring

This paper discusses issues related to the design and psychometric work for mixed-format tests --tests containing both multiple-choice (MC) and constructed-response (CR) items. The issues of validity, fairness, reliability and score consistency can be addressed but for mixed-format tests there are many decisions to be made and no examination or examination program faces exactly the same choices. This paper raised some issues and used the AP experience to illustrate the types of questions that need to be addressed and outlined the strong test development processes that need to be in place to address the issues.
Accelerate Learning with SMART Goals for Students and Staff
Author: Davis, Donna
Date: 2010-04-20
Type: Non-Print Media
Subjects: Acceleration (Education),Goal Orientation,Expectation,Task Analysis,Mathematics Education,Common Core State Standards,Core Curriculum,Curriculum Implementation,Professional Development,Models,Best Practices,Teacher Expectations of Students,Educational Assessment,Public Schools,School Effectiveness,Mathematics Tests,Grade 8,Educational Indicators,Mathematics Achievement

[Slides] presented at the annual conference of the National Council of Supervisors of Math.
Exploring the Variability in the Validity of SAT Scores and High School GPA for Predicting First-Year College Grades at Different Colleges and Universities
Author: Kobrin, Jennifer L.
Date: 2010-00-00
Type: Reference Materials - General
Subjects: Scores,Validity,Prediction,College Freshmen,College Entrance Examinations,High School Students,Grade Point Average,Institutional Characteristics,Correlation,College Applicants,Statistical Analysis,Mathematical Models

There is substantial variability in the degree to which the SAT and high school grade point average (HSGPA) predict first-year college performance at different institutions. This paper demonstrates the usefulness of multilevel modeling as a tool to uncover institutional characteristics that are associated with this variability. In a model that includes both HSGPA and SAT scores as predictors of FYGPA, the results suggest that higher mean SAT score at an institution is associated with slightly smaller validity coefficients for both HSGPA and SAT scores. The validity coefficients for HSGPA were larger at institutions with a larger percentage of white freshmen; and the predictive validity of the SAT was found to be higher at institutions with a larger percentage of students submitting SAT scores, and at smaller institutions. [Slides presented at the Annual Meeting of the American Educational Research Association (AERA) in 2010.]
Consequences of the Addition of the SAT Writing Section: A Survey of Writing Changes in the Nation's K-12 Educational System
Author: Noeth, Richard J.
Date: 2009-00-00
Type: Reference Materials - General
Subjects: College Entrance Examinations,Writing Tests,Elementary Secondary Education,Writing Instruction,Educational Change,National Surveys,Teacher Surveys,Administrator Surveys

[Slides] presented at the national conference for AERA (American Educational Research Association). This study examined whether the addition of the Writing section caused the implementation of changes in writing instruction across the nation. It used survey data from teachers and administrators across the country to measure the impact on the teaching and learning of writing.
Examining the Accuracy of Self-Reported High School Grade Point Average
Author: Shaw, Emily J.
Date: 2010-04-00
Type: Reference Materials - General
Subjects: Accuracy,Grade Point Average,High School Students,Correlation,Questionnaires,Student Attitudes,College Entrance Examinations,Comparative Analysis,Student Records,Databases,Higher Education,Outcomes of Education,Statistical Analysis,Scores,Critical Reading,Mathematics Tests,Writing Tests,Race,Ethnicity,Parent Background,Educational Attainment,Student Characteristics,Family Income,Disclosure

[Slides] presented at AERA in Denver, CO in April 2010. This study examined the relationship between students' self-reported high school grade point average (HSGPA) from the SAT Questionnaire and their HSGPA provided by the colleges and universities they attend. The purpose of this research was to offer updated information on the relatedness of self-reported (by the student) and school-reported (by the college/university from the high school transcript) HSGPA, compare these results to prior studies and provide recommendations on the use of self-reported HSGPA. Results from this study indicated that even though the correlation between the self-reported and school-reported HSGPA is slightly lower than in prior studies (r = 0.74), there is still a very strong relationship between the two measures.
Modeling the Predictive Validity of SAT Mathematics Items Using Item Characteristics
Author: Kobrin, Jennifer L.
Date: 2011-00-00
Type: Non-Print Media
Subjects: College Entrance Examinations,Mathematics Tests,Test Items,Predictive Validity,Models,Multiple Regression Analysis,Test Content,Test Format,Difficulty Level,Classification,Correlation,Grade Point Average,College Freshmen,Grades (Scholastic),College Mathematics,Majors (Students),STEM Education,Outcome Measures,Multiple Choice Tests

There is much debate on the merits and pitfalls of standardized tests for college admission, with questions regarding the format (multiple-choice versus constructed response), cognitive complexity, and content of these assessments (achievement versus aptitude) at the forefront of the discussion. This study addressed these questions by investigating the relationship between SAT Mathematics (SAT-M) item characteristics and the item's ability to predict college outcomes. Using multiple regression, SAT-M item characteristics (content area, format, cognitive complexity, and abstract/concrete classification) were used to predict three outcome measures: the correlation of item score with first-year college grade point average (FYGPA), the correlation of item score with mathematics course grades, and the percentage of students who answered the item correctly and chose to major in a mathematics or science (STEM) field. Separate models were run including and excluding item difficulty and discrimination as covariates. The results revealed that many of the item characteristics were related to the outcome measures, and that item difficulty and discrimination had a mediating effect on several of the predictor variables, particularly on the effects of non-routine/insightful items, and multiple-choice items. [Slides presented at AERA 2011.]
The National SAT Validity Study: Sharing Results from Recent College Success Research
Author: Shaw, Emily J.
Date: 2010-04-00
Type: Non-Print Media
Subjects: College Entrance Examinations,Test Validity,Educational Research,Predictive Validity,College Admission,Grade Point Average

[Slides] presented at the annual conference of the Southern Association for College Admission Counseling, April 2010. This presentation summarizes recent research from the national SAT Validity Study and includes information on the Admitted Class Evaluation Service (ACES) system and how ACES can help institutions conduct their own validity research.
The Redesigned SAT® Pilot Predictive Validity Study: A First Look. Research Report 2016-1
Author: Shaw, Emily J.
Date: 2016-00-00
Type: Reports - Research
Subjects: College Entrance Examinations,Test Construction,College Readiness,College Admission,Student Placement,Scores,Grade Point Average,College Freshmen,Grades (Scholastic),Test Validity,Prediction,Correlation,High School Students,Student Characteristics,Predictive Validity,Gender Differences,Racial Differences,Ethnic Groups,Language Usage,Family Income,Parent Background,Educational Attainment,Questionnaires

In February of 2013, the College Board announced it would undertake a redesign of the SAT® in order to develop an assessment that better reflects the work that students will do in college, focusing on the core knowledge and skills that evidence has shown to be critical in preparation for college and career. The redesigned test will be introduced in March 2016 and will include a number of important changes. As with the redesign of all assessments, it is important to examine and understand how the changes to the content and format of the test impact the inferences made from the test's scores for their intended uses. One primary use of the SAT is for admission and placement decisions and, therefore, it was important to examine the relationship between the scores from the redesigned test with college outcomes such as first-year grade point average (FYGPA) and college course grades. In order to conduct such an analysis a pilot study was initiated because the test is not yet operational. Fifteen four-year institutions were recruited to administer a pilot form of the redesigned SAT to between 75 and 250 first-year, first-time students very early in the fall semester of 2014. Measures were taken to ensure that the redesigned SAT was administered to students under standardized conditions and that students were motivated to perform well on the test. In June 2015, participating institutions provided the College Board with first-year performance data for those students participating in the fall 2014 administration of the redesigned SAT so that relationships between SAT scores and college performance could be analyzed. Results of study analyses show that the redesigned SAT is as predictive of college success as the current SAT, that redesigned SAT scores improve the ability to predict college performance above high school GPA alone, and that there is a strong, positive relationship between redesigned SAT scores and grades in matching college course domains, suggesting that the redesigned SAT is sensitive to instruction in English language arts, math, science, and history/social studies.
Changing Methods and Mindsets: Lessons from Innovate NYC
Author: Hodas, Steven
Date: 2016-01-00
Type: Reports - Descriptive
Subjects: Urban Schools,Educational Innovation,School Administration,School Districts,Purchasing,Decision Making,Educational Cooperation,Educational Technology,Entrepreneurship,Administrative Organization,Technology Uses in Education,Elementary Secondary Education

As schools and classrooms explore technology-based, student-centered, personalized approaches to teaching and learning, their efforts to innovate can be hamstrung by archaic district operating systems that do not allow them to take advantage of new technologies, to work with smaller startup companies, or to quickly make and implement decisions. When he was at the helm of the New York City Department of Education (NYCDOE), Chancellor Joel Klein attempted to remedy this problem by creating the iZone, an office explicitly charged with fostering innovation from within the school district. From 2012 to 2014, the author designed and served as executive director of Innovate NYC Schools, the iZone project that focused on new structures for district procurement and decision making. As the first urban school system to make design thinking a routine part of its practice with educators, alternatives were created to established procurement processes and build collaborations between the school district and early-stage education technology companies. Regular support was also provided to entrepreneurs, investors, and innovators who sought to do business with NYCDOE. Using the philosophy and examples from NYCDOE, this paper lays out steps school systems can take to foster innovative practices throughout the organization and develop new instructional tools and practices through nimble procurement procedures.
Physics Masters One Year after Degree: Results from the Follow-Up Survey of Master's Recipients, Classes of 2012, 2013, &amp; 2014 Combined. Focus On
Author: Pold, Jack
Date: 2015-12-00
Type: Reports - Evaluative
Subjects: Physics,Masters Degrees,Outcomes of Education,Education Work Relationship,Citizenship,Graduate Study,Doctoral Programs,Employment Level,Science Education,Student Characteristics,Gender Differences,Age Differences,Salaries,Private Sector,Job Satisfaction,Surveys

Exiting physics master's recipients from the classes of 2012, 2013, and 2014 combined pursued a variety of different post-degree outcomes. Initial outcomes differed greatly depending on citizenship. The majority of US citizens were in the workforce following the receipt of their master's degrees. In contrast, the majority of non-US citizens continued their education by enrolling in another graduate program. Regardless of citizenship, most masters continuing with graduate study did so in physics. In this report, an exiting master is defined as an individual who received his or her degree from a US physics department and left that department with no degree higher than a master's degree. Exiting masters can be earned at departments where the master's is the highest degree offered as well as at departments that offer a PhD. Each year there are many students who earn a masters en route and continue working toward a physics PhD at the same institution. They are not included in this analysis.