ethanporcaro/eric-fulltext-meta huggingface dataset:
Comparing the Effect of Contextualized versus Generic Automated Feedback on Students&apos; Scientific Argumentation. Research Report. ETS RR-22-03
Author: Olivera-Aguilar, Margarita
Date: 2022-12-00
Type: Journal Articles
Subjects: Computer Assisted Testing,Formative Evaluation,Automation,Scoring,Feedback (Response),Science Process Skills,Persuasive Discourse,Climate,Scores,Achievement Gains,Science Achievement,Writing (Composition),Scoring Rubrics

This study uses a computerized formative assessment system that provides automated scoring and feedback to help students write scientific arguments in a climate change curriculum. We compared the effect of contextualized versus generic automated feedback on students' explanations of scientific claims and attributions of uncertainty to those claims. Classes were randomly assigned to the contextualized feedback condition (227 students from 11 classes) or to the generic feedback condition (138 students from 9 classes). The results indicate that the formative assessment helped students improve their scores in both explanation and uncertainty scores, but larger score gains were found in the uncertainty attribution scores. Although the contextualized feedback was associated with higher final scores, this effect was moderated by the number of revisions made, the initial score, and gender. We discuss how the results might be related to students' familiarity with writing scientific explanations versus uncertainty attributions at school.
Evaluating the New &quot;TOEFL ITP&quot;® Speaking Test: Insights from Field Test Takers. &quot;TOEFL&quot;® Research Report. TOEFL-RR-99. ETS Research Report No. RR-22-10
Author: Lee, Shinhye
Date: 2022-12-00
Type: Journal Articles
Subjects: English (Second Language),Second Language Learning,Language Tests,Test Validity,Oral Language,Foreign Countries,Language Proficiency,Listening Comprehension,Reading Comprehension,Scores,Likert Scales,Difficulty Level

In response to the calls for making key stakeholders' perspectives relevant in the test validation process, the study discussed in this report sought test-taker feedback as part of collecting validity evidence and supporting the ongoing field testing efforts of the new &quot;TOEFL ITP&quot;® Speaking section. Specifically, I aimed to investigate the extent to which test takers' perceptions of the newly proposed ITP test tasks are in agreement with the intended characteristics and qualities of the tasks. In addition, I opted to gather insights into whether the speaking tasks are perceived as acceptable by its prospective test takers and also to identify any unwarranted challenges posed for them in completing the tasks. A two-part questionnaire was thus administered during field testing of the new speaking section, and resulting data were analyzed both quantitatively and qualitatively. Findings emerging from the questionnaire data suggest that test-taker perceptions can be used to provide support to corroborate the intended (or hypothesized) properties of the tasks, while pointing to several areas for further monitoring and improvement.
Longitudinal Stability and Change of the Dark Triad: A Call for Research in Postsecondary, Occupational, and Community Settings. Research Report. ETS RR-22-18
Author: Williams, Kevin M.
Date: 2022-12-00
Type: Journal Articles
Subjects: Personality Traits,Psychopathology,Personality Problems,Construct Validity,Interpersonal Competence,Educational Research,Longitudinal Studies,Participant Characteristics,Behavior Problems,Negative Attitudes,Personality Measures,Training,Work Environment,Personality Change

Researchers, theorists, and practitioners have expressed a renewed interest in the longitudinal dynamics of personality characteristics in adulthood, including organic life span trajectories and their amenability to volitional change. However, this research has apparently not yet expanded to include the Dark Triad (psychopathy, narcissism, Machiavellianism), despite approximately 2 decades of research that has thoroughly examined other important issues related to construct validity and interpersonal behavior. We argue that researchers in postsecondary, occupational, and community-based settings are in a unique position to study the important phenomenon of Dark Triad malleability, as they are less hindered by obstacles in clinical and forensic contexts that have generated largely inconclusive results. In this article, we discuss several examples of methods for evaluating, quantifying, and interpreting Dark Triad malleability, examples of relevant extant training programs, possibilities for developing new programs, and factors that may moderate training efficacy, including Dark Triad levels themselves. Beyond addressing a fundamental question regarding the nature of these traits, the Dark Triad's destructive tendencies suggest that efforts to reduce them would provide myriad societal benefits and could propel Dark Triad research in an important new direction.
Influence of Selected-Response Format Variants on Test Characteristics and Test-Taking Effort: An Empirical Study. Research Report. ETS RR-22-01
Author: Guo, Hongwen
Date: 2022-12-00
Type: Journal Articles
Subjects: Test Format,Test Wiseness,Test Items,Item Response Theory,Difficulty Level,Scores,Effect Size,College Students,Foreign Countries,Reaction Time

Different variants of the selected-response (SR) item type have been developed for various reasons (i.e., simulating realistic situations, examining critical-thinking and/or problem-solving skills). Generally, the variants of SR item format are more complex than the traditional multiple-choice (MC) items, which may be more challenging to test takers and thus may discourage their test engagement on low-stakes assessments. Low test-taking effort has been shown to distort test scores and thereby diminish score validity. We used data collected from a large-scale assessment to investigate how variants of the SR item format may impact test properties and test engagement. Results show that the studied variants of SR item format were generally harder and more time consuming compared to the traditional MC item format, but they did not show negative impact on test-taking effort. However, item position had a dominating influence on nonresponse rates and rapid-guessing rates in a cumulative fashion, even though the effect sizes were relatively small in the studied data.
Examining Teachers&apos; Participation in Professional Development in Terms of Their Demographic Characteristics
Author: Eroglu, Mehmet
Date: 2023-00-00
Type: Journal Articles
Subjects: Faculty Development,Teacher Characteristics,Teacher Participation,Elementary School Teachers,Secondary School Teachers,Foreign Countries,Gender Differences,Marital Status,Teaching Experience

The study aims to examine teachers' participation in professional development (PD) in terms of gender, marital status, seniority, and content area. The causal-comparative model was used in the study. The sample of the study consists of 542 content area teachers working in K-12 schools in Turkey. Personal information form and a professional development activities scale developed by Kwakman (1999), updated by Dijkstra (2009), and adapted to Turkish by Eroglu and Özbek (2020) was used to collect the data. Descriptive statistics, t-test, and Anova were used to analyze the data. Results of the study indicated that teachers' participation in PD activities and especially collaboration activities is lower. Teachers' participation in PD activities differ significantly in terms of their gender, marital status, seniority and branches. It is thought that the low participation of teachers in PD activities is due to the limited effective professional development opportunities and the lack of support for PD in Turkey.
Investigating the Relationship between Career and Technical Education High School Course-Taking and Early Job Outcomes. Research Report. ETS RR-22-19
Author: Olivera-Aguilar, Margarita
Date: 2022-12-00
Type: Journal Articles
Subjects: Vocational Education,High Schools,Course Selection (Students),Job Performance,Outcomes of Education,Predictor Variables,Income,Job Satisfaction,Longitudinal Studies,Socioeconomic Status,Expectation,Self Efficacy,Gender Differences,Racial Differences,Thinking Skills,Personality Traits

This study examined how high school course-taking patterns (i.e., career and technical education [CTE] vs. academic vs. no concentration), personal characteristics embedded in a social cognitive theory framework (e.g., self-efficacy, academic expectations), and contextual variables (e.g., parental expectations, socioeconomic status [SES]) interact with each other in the prediction of students' income and job satisfaction 8 years after graduating from high school. Using a nationally representative data set (the Educational Longitudinal Study of 2002), we found significant differences by sex and course-taking pattern in the prediction of income: Among men, CTE concentrators had the highest income, whereas among women, academic concentrators reported the greatest earnings. We observed similar levels of job satisfaction among academic and CTE concentrators. We also found that SES significantly moderated the effect of English self-efficacy and academic expectations in the prediction of income and general effort in the prediction of job satisfaction. Our findings highlight how a social cognitive framework can be used to investigate the links between high school course-taking, personal and contextual factors, and job outcomes. They additionally suggest the need to consider a broader set of outcomes for evaluating the benefits of CTE participation.
Noncredit Career and Technical Community College Programs as a Bridge to Employers: Report on NYC Region Study. Research Report. ETS RR-22-09
Author: Haviland, Sara
Date: 2022-12-00
Type: Journal Articles
Subjects: Noncredit Courses,Community Colleges,Vocational Education,College Programs,Education Work Relationship,Labor Force Development,Labor Force,Credentials,Competence,Job Placement,Alignment (Education),Employer Attitudes

Noncredit community college programs provide an important route for workforce development. They offer affordable and accessible short-term training options for individuals seeking access to middle-skills jobs. Absent the burdens of accreditation standards, they can respond nimbly to local labor market needs. However, they can also be varied and confusing, and despite the high volume of students that they serve, they are an underexamined area in higher education. This study examines noncredit programs in the New York City labor market to determine how schools align noncredit offerings to the labor market, focusing on credential design, competencies, and market processes. It pursues a push--pull design through a combination of document review and interviews with school leaders and employers and introduces quality taxonomy for understanding employer engagement in individual programs. Implications for students, programs, schools, and employers are explored.
Team of Caring Educators Delivering a Specialized Math Curriculum for Racialized Youth during COVID-19
Author: Abudiab, Sally
Date: 2023-00-00
Type: Journal Articles
Subjects: After School Programs,COVID-19,Pandemics,Partnerships in Education,Teacher Attitudes,Parent Attitudes,Community Programs,Foreign Countries,Mathematics Education,Low Income Students,Minority Group Students,Achievement Gap,Mathematics Curriculum,Program Implementation,Culturally Relevant Education,Student Attitudes,Elementary School Students

This study examined the efficacy of an afternoon and weekend academic program called the &quot;Community School Initiative&quot; (CSI) during the COVID-19 pandemic. CSI was offered to racialized students and families from the Jane and Finch community in Toronto which is one of the most under-resourced neighbourhoods in Canada. It involved a partnership between private social enterprise &quot;Spirit of Math&quot; and non-profit organization &quot;Youth Association for Academics, Athletics, and Character Education&quot; (YAAACE). Seven teachers participated in a focus group and 33 students and parents completed a survey to express their experiences attending the CSI. Findings were identified using thematic analysis from a Critical Race Theory paradigm. Key characteristics of effective structural community-programming were identified. The results highlight the importance of access to opportunities in a culturally reflective manner to ensure student success through continuity of care particularly on evenings and weekends involving a team of caring educators.
Scoring Essays on an iPad Versus a Desktop Computer: An Exploratory Study. Research Report. ETS RR-22-08
Author: Ling, Guangming
Date: 2022-12-00
Type: Journal Articles
Subjects: Tablet Computers,Computers,Essays,Scoring,Accuracy,Time,Barriers,Scores,Behavior

Recognizing the appealing features of a tablet (e.g., an iPad), including size, mobility, touch screen display, and virtual keyboard, more educational professionals are moving away from larger laptop and desktop computers and turning to the iPad for their daily work, such as reading and writing. Following the results of a recent survey of individuals who serve as ETS raters, more than 40% reported that they would prefer to use an iPad or other type of tablet to score essays. However, iPad-based essay scoring could affect scoring accuracy and scoring time because the smaller screen and other features of an iPad may also affect raters' reading comprehension and score assigning processes. To address this issue, we invited 10 experienced raters to score holistically 40 essays for a graduate admission test using a desktop computer and an iPad following a counterbalanced design. We compared the raters' scores against the criterion scores and analyzed scoring times, scoring behaviors, and raters' answers to a structured interview after the scoring experiment. The results reveal no obvious differences between the two devices in the scoring accuracy or average scoring time per essay, which suggests that scoring on an iPad may not reduce scoring quality or scoring productivity for essays that are holistically scored as compared to scoring the essays on a desktop computer. We also found a few iPad-specific issues that raters reported, including issues associated with the invisible scrolling bar and the extra scrolling needed to reach the score-assignment panel, difficulty navigating between the prompt and the essay response, and oversensitivity of the touch screen.
Technology-Enhanced Items and Model-Data Misfit. Research Report. ETS RR-22-11
Author: Eckerly, Carol
Date: 2022-12-00
Type: Journal Articles
Subjects: Computer Assisted Testing,Test Items,Item Response Theory,Scoring,Goodness of Fit,Psychometrics,Problem Solving,Models,Adaptive Testing

Testing programs have explored the use of technology-enhanced items alongside traditional item types (e.g., multiple-choice and constructed-response items) as measurement evidence of latent constructs modeled with item response theory (IRT). In this report, we discuss considerations in applying IRT models to a particular type of adaptive testlet referred to as a branching item. Under the branching format, all test takers are assigned to a common question, and the assignment of the next question relies on the response to the first question through deterministic rules. In addition, the items at both stages are scored together as one polytomous item. Real and simulated examples are provided to discuss challenges in applying IRT models to branching items. We find that model-data misfit is likely to occur when branching items are scored as polytomous items and modeled with the generalized partial credit model and that the relationship between the discrimination of the routing component and the discriminations of the subsequent components seemed to drive the misfit. We conclude with lessons learned and provide suggested guidelines and considerations for operationalizing the use of branching items in future assessments.
Investigating Constructed-Response Scoring over Time: The Effects of Study Design on Trend Rescore Statistics. Research Report. ETS RR-22-15
Author: Donoghue, John R.
Date: 2022-12-00
Type: Journal Articles
Subjects: Item Response Theory,Test Construction,Scoring,Testing,Scores,Sampling,Statistical Distributions,Interrater Reliability,Probability,Computation

When constructed-response items are administered for a second time, it is necessary to evaluate whether the current Time B administration's raters have drifted from the scoring of the original administration at Time A. To study this, Time A papers are sampled and rescored by Time B scorers. Commonly the scores are compared using the proportion of exact agreement across times and/or t-statistics comparing Time A means to Time B means. It is common to treat these rescores with procedures that assume a multinomial sampling model, which is incorrect. The correct, product-multinomial model reflects the stratification of Time A scores. Using direct computation, the research report demonstrates that both proportion of exact agreement and the t-statistic can deviate substantially from expected behavior, providing misleading results. Reweighting the rescore table gives each statistic the correct expected value but does not guarantee that the usual sampling distributions hold. It is also noted that the results apply to a wider class of situations in which a set of papers is scored by one group of raters or scoring engine and then a sample is selected to be evaluated by a different group of raters or scoring engine.
Are They Prepared? Examining Teacher Candidates&apos;, Cooperating Teachers&apos;, and University Supervisors&apos; Perspectives of a Redesigned Field Experience
Author: Myers, Joy
Date: 2023-00-00
Type: Journal Articles
Subjects: Cooperating Teachers,Student Teacher Supervisors,Student Attitudes,Teacher Attitudes,Field Experience Programs,Program Development,Practicums,Student Teachers,Interpersonal Relationship,Professionalism,Career Readiness,Elementary Education

A team of teacher educators conducted a phenomenological study to explore the perceptions of teacher candidates (TCs), their cooperating teachers (CTs), and university supervisors (USs) during and after they participated in a redesigned seven-week practicum that led directly into seven weeks of student teaching. Data were collected via two surveys with open-ended prompts and revealed three distinct themes across the three groups of participants: relationships, preparedness, and professionalism. Opportunities, presented by the redesigned field experience, were seized upon but in varying ways dependent on each group of participants (TC, CT, and US). The findings from this study are significant because unlike other studies that just examine the perspectives of one group of the triad, the goal of this research was to understand the perspectives of the entire triad (TC, CT, and US). Recommendations for teacher education programs interested in redesigning their field experiences are shared.
Augmented Skills of Educators Teaching Generation Z
Author: Whitehead, Evangelin
Date: 2023-00-00
Type: Journal Articles
Subjects: Generational Differences,Educational Technology,Technology Uses in Education,Blended Learning,Technological Literacy,Teacher Competencies,Student Characteristics,Teacher Characteristics,Constructivism (Learning),Synchronous Communication,Asynchronous Communication,Emotional Intelligence,Stress Management,Interpersonal Competence,Self Control,Teacher Role,Counseling,Research Skills,Teacher Researchers,Inquiry,Job Skills,Inclusion

Learners from different generations have enrolled and graduated from educational institutions for many decades, but educators have been using the same strategies despite what generation they teach in their classrooms. A new generation of students has occupied today's classrooms who are called 'Generation Z' or Gen Z for short, and they are colloquially called 'Zoomers'. They have unique characteristics and expectations, and they were raised with technology. They are digitally connected all the time, and it is necessary to check the quality of education given to them. Educational quality is a crucial topic worldwide and it is the need of the hour to discuss the quality of educators. To a large extent, teachers are considered the key factor in determining, and improving the quality of our educational systems. Presently, our teachers are expected to provide a quality hybrid delivery of teaching that best fits Gen Z learners. In the present century, teachers face greater challenges than ever before. The augmented skills of educators with modern, innovative, and creative strategies are the most in demand in this digital world. As new skills and technologies take over all the fields, educators also need to up-skill and re-skill themselves. This white paper discusses the augmented skills of the educators who teach Generation Z and the future generations to come.
The Effectiveness of the Brain-Based Learning Style Cycle
Author: Sahin, Seyma
Date: 2023-00-00
Type: Journal Articles
Subjects: Brain,Cognitive Style,Metacognition,Attitude Measures,Self Concept Measures,Self Efficacy,Cooperative Learning,Student Attitudes,Models,Learning Strategies,Active Learning,College Students,Foreign Countries

The purpose of this research was to evaluate the effectiveness of the brain-based learning style cycle. In the research, a pre-test and post-test quasi-experimental design without a control group was used. The research study group consisted of sophomore (2nd year) students studying at Duzce University Faculty of Education, Psychological Counseling and Guidance Department in the fall semester of the 2020-2021 academic year. There were a total of 111 students, 84 females and 27 males. The course was structured according to the Brain-Based Learning Style Cycle. Research data were collected using the &quot;Attitude Scale Towards Cooperative Learning&quot;, the &quot;Teacher Self-Efficacy Scale&quot;, the &quot;Metacognition Thinking Skill Scale&quot; and the &quot;Student Letters&quot;. At the end of the research, it was concluded that the students had positive attitudes towards the model. It was determined that the model created a significant difference in students' attitudes towards cooperative learning, teacher self-efficacy beliefs, and metacognitive thinking skills.
Primary School Teachers&apos; Music Course Competencies on the Axis of the Teachers&apos; Perspectives and the Curriculum
Author: Ünver, Ruveyda Sultan
Date: 2022-00-00
Type: Journal Articles
Subjects: Elementary School Teachers,Music Education,Teacher Competencies,Teacher Attitudes,Teacher Effectiveness,Curriculum Implementation,Grade 1,Grade 2,Grade 3,Grade 4

In this study, it was aimed to determine the proficiency of primary school teachers in music courses. The case study model, which is among the qualitative research methods, was used in this research. In the process of collecting data; the expected teacher competencies in the primary school 1,2,3 and 4th grade music textbooks and the competencies gained in the music teaching course of the Higher Education Council 2018 primary school teaching undergraduate program were specified and compared. Within the scope of this research, interviews were conducted with eight music education experts who previously delivered music lessons in the primary school teaching department and the results of the interviews were analyzed. As a result of the research, it has been determined that 22 different competencies are expected for a primary school teacher to be able to conduct music lessons at all levels of primary schools, but the competencies by the undergraduate program do not provide the six of expected competencies. As a result of expert opinions, it was concluded that most of the expected competencies cannot be gained in a onesemester course period and accordingly a qualified education is not possible. The data obtained from the research were exhibited under the relevant headings and in the light of these results, suggestions were made regarding the current situation and the future of music education.