ethanporcaro/eric-fulltext-meta huggingface dataset:
Developing Form Assembly Specifications for Exams with Multiple Choice and Constructed Response Items: Balancing Reliability and Validity Concerns
Author: Hendrickson, Amy
Date: 2010-05-01
Type: Speeches/Meeting Papers
Subjects: Multiple Choice Tests,Test Format,Test Construction,Test Validity,Test Reliability,Test Items,Psychometrics,Culture Fair Tests,Test Content,Weighted Scores,Advanced Placement Programs,Statistical Analysis,Equated Scores

The psychometric considerations and challenges associated with including constructed response items on tests are discussed along with how these issues affect the form assembly specifications for mixed-format exams. Reliability and validity, security and fairness, pretesting, content and skills coverage, test length and timing, weights, statistical specifications, and equating concerns are all discussed. Practical suggestions for dealing with these concerns in the form assembly specifications are presented. Advanced Placement Program® (AP®) Exams are used as examples throughout, as most all of these large-scale standardized exams contain both multiple-choice and constructed response items.
Defining the College-Bound Core Curriculum in High Schools
Author: Finkelstein, Doreen
Date: 2009-04-16
Type: Non-Print Media
Subjects: Longitudinal Studies,Secondary School Curriculum,Core Curriculum,College Bound Students,College Preparation,Course Selection (Students),Educational Quality,Grades (Scholastic),Academic Achievement,Common Core State Standards,Required Courses,High School Students,High School Seniors,College Attendance,State Colleges,Bachelors Degrees

Presented at the Annual Meeting of the American Educational Research Association (AERA) in April 2009. Using a comprehensive longitudinal data set as an example, poses questions and raises issues around defining a high school curriculum that is related to indicators of college success.
The Formation of the College Board and the Context of Today
Author: Patelis, Thanos
Date: 2011-04-10
Type: Non-Print Media
Subjects: Governing Boards,Nonprofit Organizations,Educational History,Organizational Development,Consortia,Common Core State Standards,Aptitude Tests,College Entrance Examinations,Educational Assessment,Educational Trends

In an effort to ensure fair preparation and selection of high school applicants to college, a committee of university and secondary school leaders after significant discussion and dialogue organized in 1900 the College Entrance Examination Board, known today as simply the College Board. The presentation will highlight the topics of this discussion and dialogue that led to the formation of this organization highlighting the issues and context that facilitated its formation. The presentation will also attempt to show how education is faced with very similar issues with the Common Core Standards and the formation of the consortia to develop assessments. The intention of the presentation is not to offer a full history and evolution of the College Board, but to offer a perspective on how the context seen in history seems to repeat itself. The presenter will comment on what can be learned from the formation for the College Board in the context of today's issues.
College-Bound Students' Interest in Engineering: Pathways and Characteristics of the Pre-College Pipeline
Author: Edwards, Kelcey
Date: 2011-04-00
Type: Non-Print Media
Subjects: College Bound Students,College Preparation,Occupational Aspiration,Engineering,Student Interests,High School Graduates,Career Development,Majors (Students),Student Characteristics,Scores,Academic Achievement,Mathematics Achievement,Science Achievement,Self Efficacy,Gender Differences,Undergraduate Students,College Attendance,Educational Attainment,Bachelors Degrees,Course Selection (Students),Racial Differences,Ethnic Groups,High School Seniors,Mathematics Instruction,Science Instruction,College Entrance Examinations,High School Students

Presented at the Annual Meeting of the American Educational Research Association (AERA) in New Orleans, LA in April 2011. The first study evaluated the level of interest in engineering as students approached the end of high school and explored the degree to which the academic preparation of these students shaped the engineering pipeline over a tenyear period. The second analyzed a recent graduating class in order to understand various pathways that students took toward or away from engineering as they progressed through high school. Several paths emerged, including students who demonstrated stable interest across time, students who became interested in engineering, and students who turned toward other majors by the end of high school. Groups differed with regard to demographic characteristics, test scores, academic background, and math and science self-efficacy.
A Look beyond Cognitive Predictors of Academic Success: Understanding the Relationship between Academic Self-Beliefs and Outcomes
Author: Mattern, Krista D.
Date: 2010-05-02
Type: Non-Print Media
Subjects: Academic Achievement,Academic Ability,Self Concept,Correlation,Outcomes of Education,Predictor Variables,Higher Education,Questionnaires,Student Attitudes,College Entrance Examinations,College Freshmen,Mathematics Achievement,Grade Point Average,Writing Ability,Mathematics Skills,High School Students,Academic Aspiration

Presented at the national conference for the American Educational Research Association (AERA) in 2010. This presentation examines the relationship between academic self-beliefs and outcomes.
A Case for Transforming the Criterion of a Predictive Validity Study
Author: Patterson, Brian F.
Date: 2011-04-10
Type: Non-Print Media
Subjects: Predictive Validity,Evaluation Criteria,Regression (Statistics),College Freshmen,Cohort Analysis,Admission Criteria,College Admission,Grade Point Average,Evaluation Problems,Least Squares Statistics,College Entrance Examinations

This study presents a case for applying a transformation (Box and Cox, 1964) of the criterion used in predictive validity studies. The goals of the transformation were to better meet the assumptions of the linear regression model and to reduce the residual variance of fitted (i.e., predicted) values. Using data for the 2008 cohort of first-time, first-year students attending 129 different colleges and universities, this study demonstrated how the transformation may be incorporated into the process of validating an admission test for predicting students' first-year grade point average (FYGPA). In this setting, we regularly found substantial negative skewness in both the criterion and its residuals and the use of a Box-Cox transformation of the criterion yielded residuals that much better approximated the assumption of normally distributed residuals. The result is that with relatively little additional effort, researchers may perform predictive validity studies for college admission measures with one of the most meaningful criteria (i.e., FYGPA) and they can better meet the assumptions of the linear regression model.
Overcoming Non-Response: Lessons Learned Migrating to a Web-Based Platform and Offering a Multi-Ethnic Response Option
Author: Sawtell, Ellen A.
Date: 2007-06-06
Type: Non-Print Media
Subjects: Response Rates (Questionnaires),Computer Assisted Testing,Testing Problems,Questionnaires,College Entrance Examinations,Student Characteristics,Ethnicity,Multiracial Persons

Presented at the Association for Institutional Research (AIR) in Kansas City in June 2007. This presentation explores what has been learned in the effort to maintain the ability to collect demographic information on SAT test takers when registration moved to the web. The information which is collected for research purposes include student's ethnicity information which is more varied than in the past due to an increase in multi-ethnic families and test takers.
Overcoming Non-Response: Lessons Learned Migrating to a Web-Based Platform
Author: Sawtell, Ellen A.
Date: 2005-05-09
Type: Non-Print Media
Subjects: Response Rates (Questionnaires),Computer Assisted Testing,Testing Problems,College Entrance Examinations,Questionnaires,Intervention

Presented at the American Educational Research Association (AERA) Conference in San Diego in May 2005. This presentation explores the issues and problems with declining response rates to the questionnaire once test registration moved from paper format to a web platform. A decline would affect research over time so interventions were put in place to increase future response rates.
Using the Admitted Class Evaluation Service (ACES) to Conduct Institution-Specific Admission or Placement Validity Studies
Author: Shaw, Emily J.
Date: 2011-09-25
Type: Non-Print Media
Subjects: College Admission,Student Placement,Test Validity,Graphs,Prediction,Online Systems,Academic Achievement,Admission Criteria,Futures (of Society),Evaluation Methods,Correlation,College Entrance Examinations,Web Sites,College Freshmen,Grade Point Average,At Risk Students,Statistical Analysis

Presented at the 23rd Annual Historically Black Colleges &amp; Universities (HBCU) Conference in Atlanta, GA, in September 2011. Admitted Class Evaluation Service (ACES) is the College Board's free online service that predicts how admitted students will perform at a college or university generally, and how successful students will be in specific courses. ACES studies provide the information needed to confirm or improve current admission and placement policies and can identify the optimum combination of measures to predict a student's future performance at your institution. Each study presents an analysis of findings and gives you general background information to help you examine the study in greater detail. This session provided an overview of ACES, including how to design useful admission and placement validity studies for your institution, advice on submitting the corresponding data file(s), and how to interpret the text, graphs, and key findings from ACES admission and placement validity reports. The following are appended: (1) Prediction equations; (2) Statistical summaries of study variables; and (3) List of IDs for students possibly at risk for not completing their degrees at Sample One University
Incorporating SAT® Writing into Admission and Placement Decisions
Author: Shaw, Emily
Date: 2010-10-28
Type: Reference Materials - General
Subjects: Test Validity,College Entrance Examinations,Writing Achievement,Writing Tests,College Admission,English,Student Placement,Correlation,Scores,Prediction,Statistical Analysis,Critical Reading,Grades (Scholastic),Evaluation Methods,College Applicants

Presented at the College Board National Forum in Washington, D.C., October 2010. This presentation examines the recent national validity evidence that supports the use of SAT Writing in college admissions and English placement. Additionally it includes information on the College Board's free online Admitted Class Evaluation Service (ACES) system, for institutions interested in conducting their own admission and placement validity research on SAT Writing.
Analysis of the Stability of Teacher-Level Growth Scores from the Student Growth Percentile Model. REL 2016-104
Author: Lash, Andrea
Date: 2016-01-00
Type: Reports - Research
Subjects: Growth Models,Teacher Evaluation,Scores,Teacher Effectiveness,School Districts,Reliability,Academic Achievement,Elementary School Teachers,Middle School Teachers,Criterion Referenced Tests,Mathematics Tests,Reading Tests

This study, undertaken at the request of the Nevada Department of Education, examined the stability over years of teacher-level growth scores from the Student Growth Percentile (SGP) model, which many states and districts have selected as a measure of effectiveness in their teacher evaluation systems. The authors conducted a generalizability study using three years of data in mathematics and reading for nearly 370 elementary and middle school teachers from Washoe County School District in Reno, Nevada's second-largest district. The study found that in mathematics, half of the variation among teachers' annual growth score (median SGPs) was attributable to differences among teachers, while half was random or unstable. In reading, 0.41 of the variance in annual scores was attributable to differences among teachers, while 0.59 was due to random or unstable sources. More stable measures of effectiveness can be constructed by averaging multiple years of growth scores for a teacher, and the report provides stability estimates for averages of two, three, and four years of annual scores. The results from this study can also be used to examine the accuracy of judgments of teachers' effectiveness that are based on these scores. Study results suggest that as states examine properties of their estimates of teacher effectiveness and consider their use in teacher accountability, they may want to be cautious in using such scores for teacher evaluation. The following are appended: (1) Related literature; (2) Creating student achievement variables; (3) Design and methods; and (4) Calculating misclassifications of effectiveness.
Measuring School Leaders' Effectiveness: Findings from a Multiyear Pilot of Pennsylvania's Framework for Leadership. Stated Briefly. REL 2016-111
Author: McCullough, Moira
Date: 2016-01-00
Type: Reports - Research
Subjects: Leadership Effectiveness,Principals,Assistant Principals,Administrator Evaluation,Value Added Models,Measures (Individuals),Statistical Analysis,Reliability,Validity,Correlation,Academic Achievement,Elementary Schools,Middle Schools,High Schools

This &quot;Stated Briefly&quot; report is a companion piece that summarizes the results of another report of the same name. This study examines the accuracy of performance ratings from the Framework for Leadership (FFL), Pennsylvania's tool for evaluating the leadership practices of principals and assistant principals. The study analyzed four key properties of the FFL: score variation, internal consistency, year-to-year stability, and concurrent validity. Score variation was characterized by the percentages of school leaders earning scores in different portions of the rating scale. To measure the internal consistency of the FFL, Cronbach's alpha was calculated for the full FFL and for each of its four categories of leadership practices. Analyses of score stability used data on FFL scores of school years across two years to calculate Pearson's correlation coefficient. Concurrent validity was assessed through a regression model for the relationship between school leaders' estimated contributions to student achievement growth and their FFL scores. This report is based primarily on the 2013/14 pilot in which 517 principals and 123 assistant principals were rated by their supervisors; an interim report examined data from the 2012/13 pilot year. The study finds that the FFL is a reliable measure, with good internal consistency and a moderate level of year-to-year stability in scores. The study also finds evidence of the FFL's concurrent validity: principals with higher scores on the FFL, on average, make larger estimated contributions to student achievement growth. Higher total FFL scores and scores in two of the four FFL domains are significantly or marginally significantly associated with both value-added in all subjects combined and value-added in math specifically. This evidence of the validity of the FFL sets it apart from other principal evaluation tools: No other measures of principals' professional practice have been shown to be related to principals' effects on student achievement. However, in both pilot years, variation in scores was limited, with most school leaders scoring in the upper third of the rating scale. As the FFL is implemented statewide, continued examination of evidence on its statistical properties, especially the variation in scores, is important. The following are appended: (1) Structure of the Framework for Leadership; and (2) Study data and methods.
A Tale of Two Teachers: An Analytical Look at the Co-Teaching Theory Using a Case Study Model
Author: Grant, Marquis
Date: 2014-01-00
Type: Reports - Research
Subjects: Team Teaching,Case Studies,Teacher Attitudes,Regular and Special Education Relationship,Disabilities,Accountability,Models,Teaching Methods,Educational Strategies,Best Practices,Small Group Instruction,Individualized Instruction,Qualitative Research,Rating Scales,Individualized Education Programs,Teacher Surveys,Interviews

Co-teaching involves a highly collaborative, mutually accountable relationship between a regular education and special education teacher in an inclusive environment. Effective co-teaching involves both teachers working together in the regular classroom setting in an effort to make learning accessible for all students regardless of ability or disabilities. Most of the analysis of co-teaching has determined that both special education students and regular education students benefit from the provision of differentiated instruction and the presence of two teachers in the classroom. This study seeks to gain an accurate analysis of co-teaching from the perspectives of regular education and special education teachers.
Rational Emotive Behavior Based on Academic Procrastination Prevention: Training Programme of Effectiveness
Author: Düsmez, Ihsan
Date: 2016-04-00
Type: Journal Articles
Subjects: Foreign Countries,Psychotherapy,Time Management,Prevention,Intervention,Program Effectiveness,Experimental Groups,Control Groups,Pretests Posttests,Primary Education,Elementary School Students,Grade 2,Grade 3,Measures (Individuals),Nonparametric Statistics,Comparative Analysis,Statistical Significance,Scores,Statistical Analysis,Likert Scales

The research is an experimental study which has experimental and control groups, and based on pre-test, post-test, monitoring test model. Research group consists of second and third grade students of Primary School Education and Psychological Counseling undergraduate programmes in Giresun University Faculty of Educational Sciences. The research group was formed among the students whose academic procrastination tendency is middle or high, by using academic procrastination scale. 60 students, whose academic procrastination tendencies are medium and high level, were identified. 24 students, who volunteered to participate to the study, were appointed randomly to the experimental and control groups. Academic procrastination scale was used to collect data. Scales were given to experiment and control groups as pre-test. A prevention of academic procrastination program which developed by the researcher based on Rational Emotive Behavioral Approach was applied to the experimental group. Control group was not given any training. After the training the of experiment group, post-test measurements were taken. And also 30 days after the experimental training, follow-up measurements were taken. Data were analyzed using Analyzis of Nonparametric. In the result of the applied program, a significant difference was found in favor of experiment group in academic procrastination. The results were interpreted in the light of the literature considering effects of Rational Emotive Behavioral Approach on academic procrastination.
Using the Partial Credit Model to Evaluate the Student Engagement in Mathematics Scale
Author: Leis, Micela
Date: 2015-00-00
Type: Journal Articles
Subjects: Learner Engagement,Measures (Individuals),Psychometrics,Test Items,Models,Elementary School Students,Elementary School Mathematics,Grade 5,Factor Analysis

The Student Engagement in Mathematics Scale (SEMS) is a self-report measure that was created to assess three dimensions of student engagement (social, emotional, and cognitive) in mathematics based on a single day of class. In the current study, the SEMS was administered to a sample of 360 fifth graders from a large Mid-Atlantic district. The Rasch partial credit model (PCM) was used to analyze the psychometric properties of each sub-dimension of the SEMS. Misfitting items were removed from the final analysis. In general, items represented a range of engagement levels. Results show that the SEMS is an effective measure for researchers and practitioners to assess upper elementary school students' perception of their engagement in math. The paper concludes with several recommendations for researchers considering using the SEMS. An appendix presents the Student Engagement in Mathematics Scale (SEMS).
The Impact of Discourse Features of Science Test Items on ELL Performance
Author: Kachchaf, Rachel
Date: 2014-04-00
Type: Speeches/Meeting Papers
Subjects: English Language Learners,Science Tests,Test Items,Discourse Analysis,Mixed Methods Research,Multiple Choice Tests,Standardized Tests,Correlation,Test Bias,Item Analysis,Evaluation Methods,Coding,Grade 5

Most research on linguistic features of test items negatively impacting English language learners' (ELLs') performance has focused on lexical and syntactic features, rather than discourse features that operate at the level of the whole item. This mixed-methods study identified two discourse features in 162 multiple-choice items on a standardized science assessment. After analyzing the frequency of features, we correlated the presence of these features with values of Differential Item Functioning (DIF) favoring non-ELLs over ELLs. Next, we analyzed 52 interviews with ELLs to examine the interactions between students and items with and without these features. Our results indicate that these two discourse features negatively impacted ELLs' item performance and item comprehension, indicating they should be avoided when developing test items.
Do Linguistic Features of Science Test Items Prevent English Language Learners from Demonstrating Their Knowledge?
Author: Noble, Tracy
Date: 2014-04-00
Type: Speeches/Meeting Papers
Subjects: Science Tests,English Language Learners,Linguistics,Test Items,Test Bias,Grade 5,Multiple Choice Tests,Interviews,Biological Sciences,Correlation,Performance,Earth Science,Space Sciences,Physical Sciences,Elementary School Students,Mixed Methods Research

Little research has examined individual linguistic features that influence English language learners (ELLs) test performance. Furthermore, research has yet to explore the relationship between the science strand of test items and the types of linguistic features the items include. Utilizing Differential Item Functioning, this study examines ELL performance on 162 Grade 5 large-scale science multiple-choice test items and its relationship to two discourse features, as well as the distribution of these features across three science strands. We also interviewed 52 ELLs to examine their interaction with these two features. Results indicate that these two features were most frequent on items under the Life Science strand. Additionally, both of these features were significantly correlated with DIF disfavoring ELLs indicating that the presence of these features potentially hinder ELLs' abilities to understand test items containing these features. Interviews confirmed that these two features in combination interfered with ELLs' abilities to make sense of the items, which often resulted in students answering incorrectly, even when they demonstrated knowledge of the content. Because the features are most frequent on Life Science items, ELLs' content knowledge on these topics may be severely underestimated for this strand.
The Educational Measurement Profession in 2007: What Is the State of Our Union?
Author: Camara, Wayne J.
Date: 2008-03-00
Type: Reference Materials - General
Subjects: Educational Assessment,Professional Personnel,Measurement,Psychometrics,Surveys,Demography,Work Environment,Sex,Race,Ethnicity,Geographic Regions,Age,Educational Attainment,Tenure,Career Change,Experience,Likert Scales,Time on Task,Compensation (Remuneration),Income,Consultants,Correlation,Statistical Data

[Slides] presented at the Annual National Council on Measurement in Education (NCME) in New York in March 2008. This presentation explores the demographic information as well as how and where educational measurement professionals are in the field.