ethanporcaro/eric-fulltext-meta huggingface dataset:
Who Needs the Undercommons? Refuge and Resistance in Public High Schools
Author: Greer, G. H.
Date: 2018-00-00
Type: Journal Articles
Subjects: Public Schools,High Schools,Resistance (Psychology),Inclusion,Activism,Educational Policy,Student Behavior,Social Justice

This paper is a theoretical discussion of &quot;The Undercommons: Fugitive Planning and Black Study&quot; (Harney &amp; Moten, 2013) as a contribution to critical education in public schools. The undercommons serves here as an epistemic device, or a way of seeing and knowing, in relation to public education. The function of this device is to establish an appreciative view of student survival and activist behaviours and to centre educational policy as a potential mechanism of student exclusion. I propose that the practice of inclusion in schools coexists with unacknowledged operations of exclusion. The undercommons is employed as a lens to make such mechanisms of disenfranchisement apparent. I advocate here for an extension of inclusive education which, in addition to targeted supports for particular demographic groups, must concern itself with more general practices of disenfranchisement.
Instructors&apos; Perceptions of Networked Learning and Analytics
Author: Comber, Scott
Date: 2018-00-00
Type: Journal Articles
Subjects: Teacher Attitudes,Social Networks,Network Analysis,Graduate Study,Graduate School Faculty,Foreign Countries,Educational Research,Data Collection,Online Courses,Integrated Learning Systems,Learner Engagement

This study seeks to understand instructors' perceptions of social network analysis (SNA) and network visualizations as learning analytics (LA) tools for generating useful insights about student online interactions in their class. Qualitative and quantitative data were collected from three graduate courses taught at a Canadian university at the end of the academic term and came from two sources: (1) class-wide forum discussion messages, and (2) interviews with instructors regarding their perceptions of student networks and interactions. This study is unique as it focuses on instructors' self-assessments of online student interactions and compares this with the SNA visualization. The difference between instructors' perceptions of social network interactions and actual interactions underlines the potential that LA can provide for instructors. The results confirmed that SNA and network visualizations have the potential of making the &quot;invisible&quot; visible to instructors, thus enhancing their ability to engage students more effectively.
What Do We Know about Secondary Mathematics Teacher Preparation in the United States?
Author: Winsor, Matthew S.
Date: 2018-00-00
Type: Journal Articles
Subjects: Secondary Education,Mathematics Instruction,Preservice Teachers,Mathematics Teachers,Teaching Methods,Secondary School Mathematics,Preservice Teacher Education

The structures of secondary mathematics teacher preparation (SMTP) programs in the United States are not well documented. This study sought to identify baseline data for SMTP programs. Survey data was collected from 86 different institutions regarding their SMTP programs. Results indicated that most SMTP programs are housed in mathematics departments, with programs often producing less than 10 teachers per year. Additionally, four-year SMTP programs were most common, with nearly all requiring a teaching methods course as well as clinical experiences. Tenure or tenure-track faculty did most student teaching supervision. Half of the participating institutions reported having a capstone course.
Writing Processes in Short Written Responses to Questions Probing Prior Knowledge. Research Report. ETS RR-18-39
Author: Deane, Paul
Date: 2018-12-00
Type: Journal Articles
Subjects: Writing Processes,Responses,Prior Learning,Task Analysis,Scores,Productivity,Editing,Word Lists,High School Students,Knowledge Level,Writing (Composition)

The purpose of the report is to explore some of the mechanisms involved in the writing process. In particular, we examine students' process data (keystroke log analysis) to uncover how students approach a knowledge-telling task using 2 different task types. In the first task, students were asked to list as many words as possible related to a particular topic (word listing). In a second task, students were asked to write to a specific prompt that was designed to elicit their background knowledge of a topic using connected text (knowledge elicitation). Using a matrix incomplete block design, 1,592 high school students completed the 2 writing tasks in addition to a multiple-choice test of their background knowledge in 2 of 5 possible topics in the domain of U.S. history. An array of process data including students' typing and associated timing features was used to predict the writing scores on the 2 different types of tasks. The analyses revealed several distinct patterns that were associated with processing at the task knowledge productivity level, the editing effort level, and the keyboarding effort level. The robustness of the features was reflected in a set of hierarchical regressions that demonstrated that the process features were predictive of the writing score even when students' knowledge scores on the associated multiple-choice test were considered. In sum, the results indicate that process data in the form of log file analysis are useful for both understanding the writing process and exploring potential differences between students with high and low knowledge. [For the corresponding grantee submission, see ED586324.]
Touch Screen Technology: Implementing a Technologically-Enhanced Profiling System for Student Sport Coaches
Author: Browne, Tom
Date: 2018-00-00
Type: Journal Articles
Subjects: Case Studies,Athletic Coaches,Innovation,Competence,Internship Programs,Information Technology,Profiles,Program Descriptions,Student Attitudes,Learning Processes,Technology Integration,Team Sports,Athletics,Data Collection,Physical Education,Undergraduate Students

This exploratory case study evaluates the implementation and use by student coaches of an innovative coaching profiling system, Touch Screen Technology (TST), to assess coaching behaviours. The study was designed to evaluate the potential of this technologically-enhanced assessment system as a profiling option for gathering, storing, retrieving, and presenting data about coaching competences. The case study documents and evaluates trainee coaches' (N=100) experiences of using TST during an internship, identifying the advantages and challenges of implementation, and evaluating the potential for coach education and development. Evidence was triangulated from questionnaires, journals, and interviews (N=20). The responses from the coaches were overwhelmingly positive. The touch screen technology, specified assessment criteria, and graphical profiling helped to integrate assessment into the program along with increased awareness and understanding of the assessment process. The students' subsequent reflections on the components of the coaching process positively impacted their performance. The study concludes that TST has the potential to assist in enhancing the learning process and bridging the gap between education and practice. Attention is drawn to the challenges of implementation.
A Rubric Development Study for the Assessment of Modeling Skills
Author: Tekin-Dede, Ayse
Date: 2018-00-00
Type: Journal Articles
Subjects: Scoring Rubrics,Mathematics Instruction,Grade 6,Grade 11,Mathematical Concepts,Middle School Students,High School Students,Mathematical Models,Problem Solving

The purpose of this study was to introduce a rubric named &quot;Rubric for the Assessment of Modeling Skills&quot; for assessing cognitive modeling skills and explaining the rubric's development process. The dimensions of the rubric are understanding the problem, simplifying, mathematizing, working mathematically, interpreting and validating. Twenty-three 6th grade and twenty 11th grade students took part in the study. During the ten weeks of the data collection process, we analyzed the participants' solutions on the modeling tasks and constructed the levels of the dimensions. The rubric offers a detailed assessment and scoring for different solutions that may arise in any modeling task implementation.
The Impact of Math Teachers&apos; Circle Participation: Case Studies
Author: Donaldson, Brianna
Date: 2018-00-00
Type: Journal Articles
Subjects: Teacher Effectiveness,Mathematics Teachers,Mathematics Instruction,Teacher Participation,Mathematics,Professional Personnel,Problem Solving,Faculty Development,Communities of Practice,Models,Comparative Analysis

The Math Teachers' Circle (MTC) model is a widely implemented form of professional development that invites teachers to become practitioners of mathematics, thereby developing their fluency with mathematical practices. This article presents comparative case studies investigating the self-reported experiences of several MTC-participating teachers representing a range of mathematical backgrounds. Teachers' reflections were examined through the lenses of three themes: views of mathematics, approaches to pedagogy, and professional engagement and leadership. Cross-case comparisons suggest several ways MTCs may be able to improve their support of teachers' professional growth, for example through discussing mathematical mindsets and incorporating additional leadership opportunities.
The Impact of Aberrant Responses and Detection in Forced-Choice Noncognitive Assessment. Research Report. ETS RR-18-32
Author: Kim, Sooyeon
Date: 2018-12-00
Type: Journal Articles
Subjects: Measurement Techniques,Response Style (Tests),Accuracy,Computation,Test Format,Personality Assessment,Scoring,Markov Processes

The purpose of this study is to assess the impact of aberrant responses on the estimation accuracy in forced-choice format assessments. To that end, a wide range of aberrant response behaviors (e.g., fake, random, or mechanical responses) affecting upward of 20%--30% of the responses was manipulated under the multi-unidimensional pairwise preference (MUPP) model with forced-choice format. Under each aberrance condition, we also computed the lz statistic (a common person-fit index) to determine how well this index can accurately detect aberrant responses without erroneously misidentifying honest examinees. Some aberrant behaviors (e.g., limited responses) were more problematic than others (e.g., omitting) in terms of true trait level recovery. The lz statistic associated with each dimension of the MUPP form led to higher detection rates than did the overall lz statistic. When the dimensional lz approach was used, however, many honest examinees were misclassified as outliers. The detection rates of lz differed as a function of aberrant response types.
A Simulation-Based Method for Finding the Optimal Number of Options for Multiple-Choice Items on a Test. Research Report. ETS RR-18-22
Author: Guo, Hongwen
Date: 2018-12-00
Type: Journal Articles
Subjects: Multiple Choice Tests,Test Items,Simulation,Test Construction,Difficulty Level,Test Reliability,Scores,Psychometrics,Guessing (Tests)

For a multiple-choice test under development or redesign, it is important to choose the optimal number of options per item so that the test possesses the desired psychometric properties. On the basis of available data for a multiple-choice assessment with 8 options, we evaluated the effects of changing the number of options on test properties (difficulty, reliability, and score comparability) using simulation. Using 2 criteria (low frequency and poor discrimination) to remove nonfunctioning options and 2 schemes (random and educated guessing) to model hypothetical response behavior for the removed options, we found that decreasing the number of options (from 8) created an easier test form but that a test form with reduced options could be more reliable if low-discriminating options were removed and an educated guessing strategy were assumed. We present a rationale for the optimal number of options for this test being approximately 5, which would result in a shorter test while preserving its psychometric quality. Simulation methods discussed in this report could be applied to any test to compare the effects of changing the number of options.
The Pseudo-Equivalent Groups Approach as an Alternative to Common-Item Equating. Research Report. ETS RR-18-02
Author: Kim, Sooyeon
Date: 2018-12-00
Type: Journal Articles
Subjects: Test Items,Item Banks,Difficulty Level,Comparative Analysis,Raw Scores,Testing,Test Wiseness

The purpose of this study was to evaluate the effectiveness of linking test scores by using test takers' background data to form pseudo-equivalent groups (PEG) of test takers. Using 4 operational test forms that each included 100 items and were taken by more than 30,000 test takers, we created 2 half-length research forms that had either 20 (strong anchor) or 10 (weak anchor) items in common. Because the 2 research forms were assembled from a single form that had been administered in a large-scale operational testing setting, we obtained the direct equating function between the 2 research forms through the single-group design and treated it as a criterion or true equating function between the 2 research forms. We equated the 2 research forms in a common-item design using the poststratification equipercentile (PSE) and chained equipercentile (CHEQ) methods, and then compared the common-item results to the results derived from the PEG linking. Because the new and reference groups differed substantially in ability, by study design, the CHEQ method produced more accurate results than did the PSE method in both the strong and weak anchor conditions. CHEQ using 10 common items was as effective as PSE using 20 common items. PSE using 10 common items produced the least accurate results among the five methods. The PEG linking produced more accurate results compared to the PSE method using the weak anchor.
Agreement of Teachers on Evaluating Assessments of Learning Progressions in English Language Arts and Mathematics. Research Report. ETS RR-18-11
Author: van Rijn, Peter
Date: 2018-12-00
Type: Journal Articles
Subjects: Teacher Attitudes,Student Evaluation,Summative Evaluation,Language Arts,Mathematics,Interrater Reliability,Test Construction,Learning Processes,Test Items,Difficulty Level,Mathematics Teachers,Secondary School Teachers,English Teachers

In this study, we explored the extent to which teachers agree on the ordering and separation of levels of two different learning progressions (LPs) in English language arts (ELA) and mathematics. In a panel meeting akin to a standard-setting procedure, we asked teachers to link the items and responses of summative educational assessments to LP levels. We evaluated four types of agreement among teachers, with test developers, and with trained raters. Although the results were quite mixed, agreement on the LP levels of mathematics items was generally better than that for ELA tasks. The agreement on the LP levels of ELA sample responses was generally better than that for mathematics. Implications of the results are discussed.
A Preliminary Investigation of the Factors Related to the Design and Scoring of Video-Based Oral Communication Performance Tasks in Higher Education. Research Report. ETS RR-18-09
Author: Roohr, Katrina Crotts
Date: 2018-12-00
Type: Journal Articles
Subjects: Speech Communication,Video Technology,Test Construction,Scoring,Performance Based Assessment,Speech Skills,Speech Tests,Interrater Reliability,Test Reliability,Undergraduate Students

Oral communication has been identified as an important skill by higher education institutions and by the workforce community. Despite its importance, minimal research has been conducted around the development of tasks to measure oral communication skills and behaviors. The purpose of this preliminary study is to evaluate the different factors related to the design and scoring of oral communication tasks. For this study, oral communication refers specifically to video-based oral presentations using a webcam. The study administered 2 informative and 2 persuasive speech tasks of varying length (2 or 4 minutes) to 8 college students. Performance was scored holistically on overall performance, and on content and delivery dimensions by 2 raters. Results show that (a) interrater reliability estimates may be impacted by task type and length, (b) there is no variation in the three scoring criteria, (c) a minimum of 2 tasks is necessary to achieve satisfactory reliability on an oral communication measure, and (d) supporting materials may impact student performance. Results of this preliminary study provide guidance on how to further this foundational research. Future research and next steps are discussed.
Bridging Validity and Evaluation to Match International Large-Scale Assessment Claims and Country Aims. Research Report. ETS RR-18-27
Author: Oliveri, María Elena
Date: 2018-12-00
Type: Journal Articles
Subjects: International Assessment,Test Validity,Test Use,Alignment (Education),Participation,Educational Objectives,Foreign Countries,Evaluation,Models

Fifty years after the first international large-scale assessment (ILSA), participation in these studies continues to grow, with more than 50% of the world's countries participating. Concomitant with growth in ILSAs is an expansion in the diversity of participant countries with respect to languages, cultures, and educational perspectives and goals. As educational aims might differ for new participants and goals among historic participants can be expected to shift over time, it is useful to understand the degree to which countries' expectations of ILSAs--as a means for understanding their educational system--align with the explicitly and implicitly stated purposes of these studies. In this presentation, we shift the conversation away from countries reporting ILSA shock and dissatisfaction with participation to fostering a productive conversation about the value and utility of participation. We propose a framework that combines notions from meta-evaluation to systematically test the evaluation tools-- ILSAs and validity theory (in relation to test use and alignment with stakeholder needs) to help countries understand why they participate in ILSAs and the potential value in taking part. We develop this conceptual framework with the aim that countries can (a) systematically consider their educational goals and the degree to which ILSA participation can reasonably help countries monitor progress toward them; (b) use an argument model to analyze claims by ILSA programs against the background of a country's specific context; and (c) more clearly understand intended and unintended consequences of ILSA participation. The framework offers a tool to systematically think through a complex web of implicit and explicit purposes, goals, and actors related to ILSAs and educational systems. To demonstrate our proposed framework, we review national education agendas in several countries with differing educational traditions (e.g., the United States, Mexico, and Norway) against published ILSA frameworks. Using our proposed method would offer a set of general guidelines that national funders can use to chart a path forward in terms of future ILSA participation. It can also equip participating countries with the knowledge to engage in reasoned conversations with testing organizations regarding unmet needs from testing programs.
A Comparison of Score Aggregation Methods for Unidimensional Tests on Different Dimensions. Research Report. ETS RR-18-01
Author: Fu, Jianbin
Date: 2018-12-00
Type: Journal Articles
Subjects: Comparative Analysis,Scores,Correlation,Standardized Tests,Test Length,Simulation,Item Response Theory,Raw Scores,Factor Analysis,Models,Test Items

In this study, we propose aggregating test scores with unidimensional within-test structure and multidimensional across-test structure based on a 2-level, 1-factor model. In particular, we compare 6 score aggregation methods: average of standardized test raw scores (M1), regression factor score estimate of the 1-factor model based on the correlation matrix of test raw scores (M2), overall ability from a unidimensional generalized partial credit model (GPCM) based on the items from all tests (M3), average of ability estimates from individual tests based on GPCM (M4), regression factor score of the 1-factor model based on the correlation matrix of ability estimates from individual tests based on GPCM (M5), and general ability from the testlet model (M6). The 4 design factors considered in the simulation study are ability correlation between tests (0.3, 0.5, 0.7, 0.8, and 0.9), test length (10, 20, 30, and 60 items), number of tests (2 and 4), and factor loading distribution (equal and unequal). The comparisons are also conducted on a real test data set with 2 tests. On the basis of the results, M1 and M4 are recommended for 2 tests, and M2, M5, and M6 are recommended for 3 or more tests. Several issues regarding attaining aggregate score reliability for intended uses and score aggregation types distinguished by test dimensionality are discussed, and practical suggestions for score aggregation are provided.
A Psychological Approach to Human Capital. Research Report. ETS RR-18-30
Author: Kell, Harrison J.
Date: 2018-12-00
Type: Journal Articles
Subjects: Human Capital,Cognitive Ability,Personality Traits,Vocational Interests,Psychological Patterns,Cognitive Processes,Help Seeking,Academic Achievement,Predictor Variables,Affective Behavior,Social Psychology,Models

The concept of human capital originated in economics over 200 years ago. The current paper takes a novel approach to human capital by interpreting it from a psychological perspective. We divide human capital into 2 domains: traditional and nontraditional. The traditional domain consists of the constructs that have been historically classified as human capital: the cognitive skills and knowledge associated with educational success and measured by high-stakes tests. The nontraditional domain consists of constructs that have not historically been associated with human capital: personality traits, vocational interests, and psychosocial and academic-related factors. Both traditional and nontraditional human capitals are important predictors of school and work success. Science, however, is concerned with developing explanations in addition to making accurate predictions, and we move beyond a descriptive taxonomy of human capital constructs by providing a psychological process-based account of human capital grounded in the cognitive-affective processing system (CAPS) developed by Mischel and Shoda. To practically illustrate the power of this approach for explaining how human capital is manifested in individuals' actions, we offer a CAPS-based model of student help-seeking behavior--a type of behavior that is an important facilitator of positive educational outcomes.
Administrators&apos; Uses of Teacher Observation Protocol in Different Rating Contexts. Research Report. ETS RR-18-18
Author: Qi, Yi
Date: 2018-12-00
Type: Journal Articles
Subjects: Teacher Evaluation,Classroom Observation Techniques,Observation,Evaluation Methods,Administrators,Evaluators,Teacher Effectiveness,Scoring,Cognitive Processes,Professional Training,Protocol Analysis,Educational Strategies

Teacher observations are being used for high-stakes purposes in states across the country, and administrators often serve as raters in teacher evaluation systems. This paper examines how the cognitive aspects of administrators' use of an observation instrument, a modified version of Charlotte Danielson's Framework for Teaching, interact with the complex and dynamic rating contexts in applied settings. Findings suggest that administrators' rating strategies and rating approaches vary as the characteristics of the rating contexts differ. Even shortly after training (and more so as time passed), raters used reasoning strategies not supported by their training to make scoring decisions. We discuss the implications of the findings for the training of raters and the development of evaluation systems in high-stakes contexts.