ethanporcaro/eric-fulltext-meta huggingface dataset:
Impact of Categorization and Scaling on Classification Agreement and Prediction Accuracy Statistics. Research Report. ETS RR-21-26
Author: Wang, Wei
Date: 2021-12-00
Type: Journal Articles
Subjects: Classification,Scaling,Prediction,Accuracy,Statistics,Measurement,Correlation,Scoring,Computer Assisted Testing,Automation,Essays

Agreement statistics and measures of prediction accuracy are often used to assess the quality of two measures of a construct. Agreement statistics are appropriate for measures that are supposed to be interchangeable, whereas prediction accuracy statistics are appropriate for situations where one variable is the target and the other variables are predictors. Using bivariate normality assumptions, we analytically examine the impact of categorization of a continuous variable and mean/sigma scaling on different measures of agreement and different measures of prediction accuracy. We vary the degree of relationship (squared correlation) between two continuous measures of a construct and the degree to which these measures are reduced to fewer and fewer categories (categorization). The main findings include that (a) categorization influences all the statistics investigated, (b) the correlation between the continuous variables affects the values of the statistics, and (c) scaling a prediction of a target variable to have the same mean and variability as the target increases agreement (according to Cohen's kappa and quadratic weighted kappa) but does so at the expense of prediction accuracy. The implications of these results for scoring of essays by humans or machines are also discussed.
Methods for Measuring Speededness: Chronology, Classification, and Ensuing Research and Development. Research Report. ETS RR-21-22
Author: Cintron, Dakota W.
Date: 2021-12-00
Type: Journal Articles
Subjects: Classification,Research and Development,Timed Tests,Guessing (Tests),Item Response Theory,Test Wiseness,Pacing,Reaction Time,Measurement,Test Items

The extent to which a test's time limit alters a test taker's performance is known as speededness. The manifestation of speededness, or speeded behavior on a test, can be in the form of random guessing, leaving a substantial proportion of test items unanswered, or rushed test-taking behavior in general. Speeded responses do not depend solely on a test taker's ability and are therefore not appropriate for traditional item response theory. The literature on measuring the extent of speededness on a test is extensive and dates back over a halfcentury. Yet, simple rules of thumb for measuring speededness, dating back until at least Swineford in 1949, are still in operation--for example, 80% of the candidates reach the last item. The purpose of this research report is to provide a chronology and classification of methods for measuring speededness and to discuss ensuing research and development in measuring speededness.
The Development of Innovative Media Education Styles in the Era of Information and Communication Technologies
Author: Butyrina, Mariya
Date: 2022-00-00
Type: Journal Articles
Subjects: Information Technology,Educational Technology,Media Literacy,College Students,Student Attitudes,Foreign Countries,Communication Skills,Student Motivation

The new era of the 21st century is characterized by the rapid pace of digitalization of society and the development of information and communication technologies (ICTs). ICTs are transforming the basics of educational activities from the physical environment to the virtual one. There is a similarity between technology and media in content and strategic context. The media actively influence the public opinion, and information and communication technologies are used to increase the impact on academic performance. Therefore, there is a need for a critical analysis of information reality in order to develop the competence in future generation. The article provides the study of the process of development of innovative media education styles, which are effective in educational activities for the formation of a competent future generation capable of critical analysis of the information. The study of the formation of innovative media education styles was based on the Synyavsky's communicative and organizational skills measurement methods in order to diagnose the main aspects of educational activities in the innovative context, Milman's personal motivation technique, survey to determine the competency criterion of media education. A pedagogical experiment was conducted as part of the study. The results of the study became the ground for determining the content of innovative media education styles as an alternative to modern forms of education. Innovative media education styles are formed due to the influence of ICTs on educational activities. The obtained data were processed in SPSS 18.0.1.
Model Adequacy Checking for Applying Harmonic Regression to Assessment Quality Control. Research Report. ETS RR-21-13
Author: Qian, Jiahe
Date: 2021-12-00
Type: Journal Articles
Subjects: Models,Regression (Statistics),Language Tests,Quality Control,Educational Assessment,Prediction,Error of Measurement,Goodness of Fit,Sampling

In recent years, harmonic regression models have been applied to implement quality control for educational assessment data consisting of multiple administrations and displaying seasonality. As with other types of regression models, it is imperative that model adequacy checking and model fit be appropriately conducted. However, there has been no literature on how to perform a comprehensive model adequacy evaluation when applying harmonic regression models to sequential data with seasonality in the educational assessment field. This paper is intended to fill this gap with an illustration of real data from an English language assessment. Two types of cross-validation, leave-one-out and out-of-sample, were designed to measure prediction errors and check model validation. Three types of R-squared (R[superscript 2], R[superscript 2][subscript adj], and R[superscript 2][subscript pred]) and various residual diagnostics were applied to check model adequacy and model fitting.
The Impact of Media Culture on Future Professionals&apos; Training
Author: Tatarnikova, Anzhelika
Date: 2022-00-00
Type: Journal Articles
Subjects: Social Media,Popular Culture,Mass Media,Student Interests,Cognitive Style,Cultural Influences,Foreign Countries,Professional Education,Success,Technological Literacy,College Students,Outcomes of Education,Education Work Relationship,Internet,Computer Mediated Communication,Career Readiness

The relevance of the research is substantiated by the fact that the information society can be addressed as a stage in the modern civilization formation, which is characterized by the increasing role of knowledge, information, as well as information processing technologies. The number of people employed in the information sphere is growing steadily. The information products and information services are gaining additional shares in the market. It is specifically the prerequisites for a future specialist to have high media and cultural competence to tackle current challenges. The research aims to ascertain experimentally the pedagogical conditions for media culture formation and its impact on future professionals' training. The study was conducted using such tools as questionnaires and testing. Diagnosis of cognitive interest (according to N. Kuzmina's scale), the scale for assessing students' operational skills (according to M. Chobitko), Pearson's criterion (chi-square). In the experimental group, only 1.9% of students at the control stage of the experiment corresponded to the low level of media culture according to the cognitive criterion, while in the control group there were 18.4% f students. However, in the experimental group at the control stage of the experiment more than 61% of respondents yielded a high level of media culture according to the cognitive criterion. That said, in the experimental group only 1.9% respondents at the control stage of the experiment corresponded to a low level of media culture according to the cognitive criterion. The obtained results give grounds to argue that media culture formation has a positive effect on the future professionals' training. This is due to the fact that media culture is underlying the general professional competence of the XXI century specialist. The enhancement of the students training through the use of new information and Internet technologies has all chances to be one of the prospects for further research in the relative to the issue addressed.
Enhancing Cosmetology Students&apos; Communication Skills in Malaysian Upper Secondary Vocational Education Program
Author: Tee, Tze Kiong
Date: 2022-03-00
Type: Journal Articles
Subjects: Cosmetology,Communication Skills,Foreign Countries,Vocational Education,Vocational High Schools,Skill Development,Secondary School Students,Student Attitudes

The quality of beauty treatment services will be evaluated by the clients subjectively whether they are satisfied or otherwise with the services provided. However, the reports and complaints related to customer dissatisfaction when receiving beauty treatment services at beauty centers are increasing. Unfortunately, the suggestions for improvement from the previous studies are still insufficient. Therefore, this study was conducted to examine the importance of proficiency in communication skills and the need for training modules in the teaching and learning process (T&amp;L). It is required to improve communication skills among cosmetology students for Upper Secondary Vocational Education Program (USVEP) in beauty arts course. The data were collected through the interview method on six USVEP teachers and a beauty salon entrepreneur. The findings of the study found that the proficiency in communication skills among the cosmetology students is an important requirement for employers in selecting their employees, increasing customer loyalty and the customer's frequency to visit beauty salons, and maintaining the excellent performance of cosmetology students in the workplace. In addition, there is also a high demand for the development of communication skills training modules in helping cosmetology students to constantly updated with new knowledge and skills needed from time to time. Therefore, adequate training is required to improve the competency in communication skills among the cosmetology students.
The Use of Binary Online Lessons in the Context of Forming Critical Thinking in Future Journalists
Author: Poplavska, Nataliya
Date: 2022-00-00
Type: Journal Articles
Subjects: Critical Thinking,Journalism,Teaching Methods,Information Technology,Distance Education,Interdisciplinary Approach,Teamwork,News Reporting,Professional Education,Media Literacy,Photography,Internet,Teacher Responsibility,Cooperative Learning,Foreign Countries,Online Courses

The emphasis is placed on changes in the educational field of Ukraine and the importance of forming the competencies of future journalists. Features of the application of modern information technologies in distance learning are outlined. A review of scientific sources of Ukrainian and foreign scientists on the stated issues. The relevance of the introduction of integrated technologies in modern journalism education, which contributes to improving the training of future professionals, the formation of his competencies. Definitions of the concepts &quot;integrated learning&quot;, &quot;interdisciplinary approach&quot;, &quot;team learning&quot; are defined. The focus is on the need to use an integrated approach - binary classes to increase the level of cognitive activity and activity of higher education. The focus is on forming multi-qualification of the modern journalist working in convergent newsrooms. Means of practical training of students that significantly affect the formation of professional competencies are identified. The importance of media literacy and critical thinking for the training of future journalists has been updated. According to the results of an online survey of students, it was found that interactive teaching methods increase the level of critical thinking and form skills of verification of information as a program competence of future media professionals. Scientific approaches to understanding the concept of &quot;binary class&quot;, the organization and methods of its implementation, the features of distance learning during the pandemic. The binary online lesson &quot;Critical Thinking: Verification of Online Content&quot; for students-journalists in the disciplines of journalism (photo and online journalism) on the Zoom platform is described. The skills and abilities necessary for the future mediator, which create a positive professional image and public authority, are generalized and classified. Summarized the application of new learning technologies for self-realization of students, the atmosphere of cooperation, increased responsibility of teachers for the results of their work. Vectors of further research of integrated interdisciplinary classes in the educational process are outlined.
Equitable STEM Instruction and Assessment: Accessibility and Fairness Considerations for Special Populations. Research Report. ETS RR-21-11
Author: Guzman-Orth, Danielle
Date: 2021-12-00
Type: Journal Articles
Subjects: Equal Education,STEM Education,Evaluation Methods,Accessibility (for Disabled),English Language Learners,Students with Disabilities,Teaching Methods

The landscape for STEM instruction is rapidly shifting in the United States. Attention toward STEM instruction and assessment opportunities is increasing. All students must have opportunities to gain access to the STEM content and show what they know and are able to do. We caution that attention to fairness and accessibility is critical for students from special populations, particularly English learners and students with disabilities. Opportunities for equitable access to STEM instruction and assessment are diminished without accessibility. In this report, we use an assets-based perspective to discuss and reframe common misconceptions and challenges as opportunities. We argue that attention to accessibility at the onset of STEM instruction and assessment is the pivotal foundation for fair opportunities in STEM. We highlight key opportunities and conclude with recommendations for improved fairness and access in STEM.
Exploring Diversity in Graduate and Professional School Applications. Research Report. ETS RR-21-17
Author: Holtzman, Steven
Date: 2021-12-00
Type: Journal Articles
Subjects: Student Diversity,Graduate Study,Professional Education,Scores,Gender Differences,Race,Socioeconomic Status,Admission Criteria,College Admission,Reputation,Majors (Students),College Entrance Examinations,College Choice,Family Influence,School Culture,Student Mobility,Geographic Location

Gathering a diverse student body is important for institutions of higher education (IHEs) at the graduate/professional level. However, it is impossible to select a diverse student body from a homogenous group of candidates. Thus the aim of this study is to discover the extent to which diversity goals in admissions are precluded by the lack of diversity in the applicant pool. To explore this, the proportion of score reports sent to the 150 largest graduate/professional schools and sent for each major was compared, from each gender, race, and socioeconomic status (SES) grouping, to proportions from the overall applicant pool of graduate/professional students. Additionally, differences in the distance graduate/professional school applicants are willing to consider traveling by gender, race/ethnicity, and SES were investigated. Results show that differences exist in the gender, race, and SES distributions for score reports sent to different schools and for different majors as well as in the distance an applicant is willing to consider traveling for graduate/professional school. The patterns found for gender, racial, and socioeconomic diversity provide possibilities for researchers to work further together with graduate/professional schools to tackle the important challenge of increasing diversity in graduate/professional education.
Perspective-Taking and Perspectival Expansions: A Reflection and an Invitation
Author: Baer, Andrea
Date: 2022-06-00
Type: Journal Articles
Subjects: Perspective Taking,Information Literacy,Reflective Teaching,Reflection,Educational Practices

Over the past two+ years, many of us have been recalibrating our views on teaching and learning, our approaches to information literacy education, and our orientations to everyday life in and outside of work. As I imagine how I want my own engagement in teaching and learning to continue unfolding, I've also been reflecting on what I value about &quot;Communication in Information Literacy's&quot; (&quot;CIL&quot;) Perspectives section and what I hope for it as the journal, information literacy, and education continue to evolve. In this short essay, I consider different ways of thinking about the term &quot;perspectives&quot;; reflect on Perspectives as a space for exploring questions, issues, and experiences from new vantage points; and invite members of the information literacy community to share through &quot;CIL's&quot; Perspectives their unique ways of seeing.
Examining the Accuracy of a Conversation-Based Assessment in Interpreting English Learners&apos; Written Responses. Research Report. ETS RR-21-03
Author: Lopez, Alexis A.
Date: 2021-12-00
Type: Journal Articles
Subjects: Accuracy,English Language Learners,Language Proficiency,Language Tests,Communicative Competence (Languages),Test Format,Mathematics Tests,Middle School Students,Task Analysis,Computer Assisted Testing,Peer Relationship,Teacher Student Relationship,Written Language,Responses,Test Items,Item Analysis,English (Second Language),Second Language Learning

Substantial progress has been made toward applying technology enhanced conversation-based assessments (CBAs) to measure the English-language proficiency of English learners (ELs). CBAs are conversation-based systems that use conversations among computer-animated agents and a test taker. We expanded the design and capability of prior conversation-based instructional and assessment systems and developed a CBA designed to measure the English language skills and the mathematics knowledge of middle school ELs. The prototype CBA simulates an authentic and engaging mathematics classroom where the test taker interacts with two virtual agents to solve math problems. We embedded feedback and supports that are triggered by how the CBA interprets students' written responses. In this study, we administered the CBA to middle school ELs (N = 82) residing in the United States. We examined the extent to which the CBA system was able to consistently interpret the students' responses (722 responses for the 82 students). The study findings helped us to understand the factors that affect the accuracy of the CBA system's interpretations and shed light on how to improve CBA systems that incorporate scaffolding.
Meet Students Where They Are: Centering Wikipedia in the Classroom
Author: Park, Diana E.
Date: 2022-06-00
Type: Journal Articles
Subjects: Encyclopedias,Electronic Publishing,Collaborative Writing,Information Literacy,Information Sources,Literacy Education,Reliability,Access to Information,Internet,College Students,Teaching Methods,Course Content,Class Activities,Bias,Languages,Mathematics,Censorship,Privacy,Course Evaluation

There is a common classroom refrain, &quot;Don't use &quot;Wikipedia&quot;; it's unreliable.&quot; Unfortunately, this simple dismissal of the world's largest repository of information fails to engage students in a critical conversation about how knowledge within &quot;Wikipedia&quot; is constructed and shared. &quot;Wikipedia&quot; is available in almost 300 languages, it is the top result in most Google searches, and it provides free, well-sourced, information to millions of people every day. However, despite these positives, there is uneven geographic, historical, and cultural representation; there are well-known information gaps related to women, gender, and sexual identity; and the majority of &quot;Wikipedia&quot; editors are white, Western, men. Engaging students in complex conversations about this information source is one way to improve students' information literacy skills. In 2019 we decided to meet students where they are by developing a two-credit course, &quot;Wikipedia and Information Equity,&quot; at Oregon State University that centers and critically examines &quot;Wikipedia&quot; as an information source and as a community of editors co-creating public knowledge. This article shares our experience teaching this two-credit course three times, with the ultimate goal of providing a template and starting point from which other instructors can develop similar courses and curricula about information equity through the lens of &quot;Wikipedia.&quot;
Teaching and Assessment of Metacognition in the Information Literacy Classroom
Author: McCoy, Erin J.
Date: 2022-06-00
Type: Journal Articles
Subjects: Metacognition,Information Literacy,Evaluation Methods,Teaching Methods,Academic Libraries,Librarians,Librarian Teacher Cooperation,College Students

Information literacy and metacognition have long histories of addressing the same concerns: how people think about and evaluate what they have learned. By exploring research from the library science and cognitive psychology fields, this article highlights how these two concepts are related and how that relationship can be made more explicit in the way librarians talk about and teach information literacy.
Beyond Nuclear Families: Development of Inclusive Student Socioeconomic Status Survey Questions. Research Report. ETS RR-21-18
Author: Whorton, Ryan
Date: 2021-12-00
Type: Journal Articles
Subjects: Socioeconomic Status,Grade 4,Grade 8,Grade 12,Family Structure,Student Attitudes,Background,Educational Attainment,Caregivers,Test Construction,Social Change,Employment Level,Individual Characteristics,Student Surveys

Social changes have resulted in an increase of students living in households that do not include both a mother and a father, reducing the efficacy of common survey questionnaire approaches to measuring student socioeconomic status (SES). This paper presents two studies conducted to develop and test a new,more inclusive set of student SES items appropriate for students from a range of household types. In the first study, we held group interviews with 57 students in Grades 4, 8, and 12 who lived in four nontraditional household types. The study goal was, first, to understand how students thought about their household members and learn what they knew about the educational background and employment status of their caregivers and, second, to develop draft items based on these findings. In the second study, we held 51 individual cognitive interviews with a similar sample to evaluate draft item clarity and function.We found that although students may live with a broad range of family members and other adults, they understood the term caregiver to refer to a person who provides resources and support. Students found it easier to answer items when the items included the titles of their caregivers. Our results demonstrate that a customizable approach to measuring student SES allows more students to report information about their caregivers than the current standard of asking about mothers and fathers. We provide recommendations for student SES measurement and potential next steps for research on this topic.