ethanporcaro/eric-fulltext-meta huggingface dataset:
A Comparison of Spoken and Written Language Use in Traditional and Technology-Mediated Learning Environments. TOEFL® Research Report. RR-94. ETS RR-21-16
Author: Kyle, Kristopher
Date: 2021-12-00
Type: Journal Articles
Subjects: Comparative Analysis,Second Language Learning,English (Second Language),Language Tests,Computer Assisted Testing,Academic Language,Speech Communication,Writing Tests,Computational Linguistics,Integrated Learning Systems,Language Usage,Test Validity,Computer Assisted Instruction,Scores,Teaching Methods,Conventional Instruction,Higher Education,College Admission,Admission Criteria

A key piece of a validity argument for a language assessment tool is clear overlap between assessment tasks and the target language use (TLU) domain (i.e., the domain description inference). The TOEFL 2000 Spoken and Written Academic Language (T2K-SWAL) corpus, which represents a variety of academic registers and disciplines in traditional learning environments (e.g., lectures, office hours, textbooks, course packs), has served as an important foundation for the TOEFL iBT®test's domain description inference for more than 15 years. There are, however, signs that the characteristics of the registers that students encounter may be changing. Increasingly, typical university courses include technology-mediated learning environments (TMLEs), such as those represented by course management software and other online educational tools. To ensure that the characteristics of TOEFL iBT test tasks continue to align with the TLU domain, it is important to analyze the registers that are typically encountered in TMLEs. In this study, we address this issue by collecting a relatively large (4.5 million words) corpus of spoken and written TMLE registers across the six primary disciplines represented in T2KSWAL. This corpus was subsequently tagged for a wide variety of linguistic features, and a multidimensional analysis was conducted to compare and contrast written and spoken language in TMLE and T2K-SWAL. The results indicate that although some similarities exist across spoken and written texts in traditional learning environments and TMLEs, language use also differs across learning environments (and modes) with regard to key linguistic dimensions.
Assessing Mode Effects of At-Home Testing without a Randomized Trial. Research Report. ETS RR-21-10
Author: Kim, Sooyeon
Date: 2021-12-00
Type: Journal Articles
Subjects: Testing,Distance Education,Comparative Analysis,Test Items,Scores,Supervision,Testing Problems,Environmental Influences,Performance,Difficulty Level,Psychometrics

In this investigation, we used real data to assess potential differential effects associated with taking a test in a test center (TC) versus testing at home using remote proctoring (RP). We used a pseudo-equivalent groups (PEG) approach to examine group equivalence at the item level and the total score level. If our assumption holds that the PEG approach removes between-group ability differences (as measured by the test) reasonably well, then a plausible explanation for any systematic differences in performance between TC and RP groups that remain after applying the PEG approach would be the operation of test mode effects. At the item level, we compared item difficulties estimated using the PEG approach (i.e., adjusting only for ability differences between groups) to those estimated via delta equating (i.e., adjusting for any systematic differences between groups). All tests used in this investigation showed small, nonsystematic differences, providing evidence of trivial effects associated with at-home testing. At the total score level, we linked the RP group scores to the TC group scores after adjusting for group differences using demographic covariates. We then compared the resulting RP group conversion to the original TC group conversion (the criterion in this study). The magnitude of differences between the RP conversion and the TC conversion was small, leading to the same pass/fail decision for most RP examinees. The present analyses seem to suggest little to no mode effects for the tests used in this investigation.
The Analysis of Problems and Needs of Educational Information Technology of Thailand National Sports University
Author: Chankuna, Dittachai
Date: 2022-00-00
Type: Journal Articles
Subjects: Foreign Countries,Information Technology,Educational Technology,Technology Uses in Education,Instructional Design,Higher Education,Educational Needs,Teaching Methods

The objectives of this study were to: (1) investigate the problems surrounding the use of information technology in education at Thailand National Sports University Chon Buri Campus; and (2) determine the university's information technology needs. The quantitative method was used to investigate the problems and needs associated with the use of information technology in education at Thailand National Sports University Chon Buri campus. A questionnaire was used to collect data from 92 individuals who worked at Thailand National Sports University's Chon Buri Campus as executive administrators, lecturers, and support staff. The study's findings indicated that the priority was placed on the design of educational technology that aided in teaching and learning. Following that, there was a need in designing educational technology to support research and educational technology designed to support academic services.
The Development of Pragmatics in Morris&apos;s Behavioral Semiotics: Semiotic Perspective
Author: Niu, Min
Date: 2022-00-00
Type: Journal Articles
Subjects: Semiotics,Pragmatics,Linguistic Theory,Comparative Analysis,Philosophy,Classification,Signs,Social Behavior,College Faculty

Many pragmaticists claimed that pragmatics developed mainly from 1970s and 1980s, taking the emergence of its series of pragmatic theories (e.g., Cooperative Principle, Conversation Implicature, Relevance Theory, etc.), methodology, and the official issue of &quot;Journal of Pragmatics&quot; (1977) in Amsterdam as its marks. However, few scholars reinstate the historical truth of the real development of pragmatics by comparing and reviewing the semiotic thoughts between Peirce and Morris so as to discover the development and prosperity of pragmatics in Morris's times (around 1930s). As one of the founders of modern semiotics, Morris's major contribution derives from his proposal of trichotomy theory of semiosis, that is, syntactics, semantics, and pragmatics, grounded on which Morris intends to establish an all-inclusive general theory of signs. Substantially, Morris's semiotic theory involves rich pragmatic thought which is inherited and developed from Peirce theory of signs that has logic and pragmaticism as its foundations. This paper aims to reveal the formation and development of pragmatics in Morris's behavioral semiotics by means of comparing the semiotic thoughts between Peirce and Morris from the perspective of dynamic history. Specifically, this paper involves the analysis of the philosophical foundations of pragmatics, disciplinary classification, the establishment of pragmatics, theory of sign inquiry, the relevant specific semiotic views, and three semiotic dimensions of Morris's pragmatics, which are conducive to explore the pragmatic origin and development from the perspective of Morris's semiotic thoughts. As such, Pragmatics in Morris's behavioral semiotics is established and developed in a systematic and formal way.
The Ways of Developing Basic Competences in the Study of Foreign Languages through Interactive Methods
Author: Latygina, Nataliia A.
Date: 2022-00-00
Type: Journal Articles
Subjects: Competence,Second Language Learning,Interaction,Teaching Methods,Cooperative Learning,Active Learning,Student Projects,English (Second Language),German,French,Spanish,Polish,Instructional Effectiveness,Discussion (Teaching Technique),Role Playing,Case Method (Teaching Technique),Electronic Learning,Foreign Countries,College Students

The search for productive methods of learning foreign languages is urgent for increasing communication needs of society due to globalization. The aim of the research was to find, introduce into the educational process and test the productivity of interactive methods that can be used in the study of foreign languages and the development of basic competencies. The study involved theoretical and empirical methods, pedagogical experiment and observation. The ADDIE (Analysis, Design, Development, Implementation, Evaluation) model was also used in the research. Linguodidactic tests were used to assess the skills acquired by students, which are part of the basic competencies. Mathematical methods for processing experimental data, Pearson's criterion and Cohen's coefficient were also used. The following were chosen among the 20 interactive methods: project method, discussion, conversation, collective analysis of the situation, role play, work in pairs, work in small groups and the use of mobile applications. Their effectiveness was investigated in learning the following foreign languages: English, German, French, Spanish, Polish. It was found that they have a positive effect on improving the correctness of language, its purity, clarity, accuracy, logic, expressiveness, conciseness. They also contribute to the development of skills to structure the report, analyse information, formulate and voice opinions, use language tools. Listening and reading improve the results, they develop the ability to correctly answer questions, express oneself in accordance with the given topic etc. Further research can be focused on identifying and developing new effective interactive teaching methods for development of foreign language competencies in students.
Developing an Innovative Elicited Imitation Task for Efficient English Proficiency Assessment. TOEFL® Research Report. RR-96. ETS RR-21-24
Author: Davis, Larry
Date: 2021-12-00
Type: Journal Articles
Subjects: Task Analysis,Language Proficiency,Speech Communication,Language Tests,English (Second Language),Second Language Learning,Imitation,Holistic Approach,Test Format,Test Construction,Scoring,Test Validity,Linguistic Input,Interrater Reliability,Rating Scales,Correlation,Foreign Countries,Participant Characteristics,College Attendance,Animation

The elicited imitation task (EIT), in which language learners listen to a series of spoken sentences and repeat each one verbatim, is a commonly used measure of language proficiency in second language acquisition research. The &quot;TOEFL® Essentials&quot;™ test includes an EIT as a holistic measure of speaking proficiency, referred to as the &quot;Listen and Repeat&quot; task type. In this report, we describe the design considerations that informed the development of the EIT for TOEFL Essentials. We also report the results of a series of investigations conducted during the prototyping and pilot phases of test development, which were undertaken with the goal of confirming task design specifications, evaluating scoring performance, and obtaining initial validity evidence to support score interpretation and use of the EIT in the TOEFL Essentials test. We found that task design variables generally performed as expected. The length of input sentence was strongly associated with performance (Pearson r = 0.88), consistent with the construct measured by the EIT, while other task variables not directly related to the EIT construct did not impact performance (e.g., graphics, speaker accent, and response time). Scorers drawn from TOEFL iBT test raters were able to score responses consistently with over 98% exact or adjacent interrater agreement on a 6-point scale, and scores on the pilot version of the EIT were highly reliable (Cronbach's [alpha] = 0.93 on the 15-item pilot version). Correlations between EIT scores and other measures were generally as expected: Correlations with other speaking tasks were high (0.78-0.84) and slightly to somewhat lower for other language measures (0.73 for writing, 0.68 for listening, and 0.57 for reading). Correlation with an independent measure of holistic language proficiency (C-test) was moderately high (0.69), as expected. We discuss the study findings in terms of the TOEFL Essentials test validity argument and point out limitations to the current results along with future research needs. Overall, we believe that the findings provide initial support to warrant the use of the EIT as operationalized in the TOEFL Essentials test.
Comparisons among Approaches to Link Tests Using Random Samples Selected under Suboptimal Conditions. Research Report. ETS RR-21-14
Author: Kim, Sooyeon
Date: 2021-12-00
Type: Journal Articles
Subjects: Equated Scores,Weighted Scores,Sampling,Multiple Choice Tests,International Assessment,Test Bias,Testing,Raw Scores

Equating the scores from different forms of a test requires collecting data that link the forms. Problems arise when the test forms to be linked are given to groups that are not equivalent and the forms share no common items by which to measure or adjust for this group nonequivalence. We compared three approaches to adjusting for group nonequivalence in a situation where not only is randomization questionable, but the number of common items is small. Group adjustment through either subgroup weighting, a weak anchor, or a mix of both was evaluated in terms of linking accuracy using a resampling approach. We used data from a single test form to create two research forms for which the equating relationship was known. The results showed that both subgroup weighting and weak anchor approaches produced nearly equivalent linking results when group equivalence was not met. Direct (random groups) linking methods produced the least accurate result due to nontrivial bias. Use of subgroup weighting and linking using the anchor test only marginally improved linking accuracy compared to using the weak anchor alone when the degree of group nonequivalence was small.
Online Literature Circles in Learning Hamlet among Pre-Service Teachers
Author: Abdullah, Tina
Date: 2021-12-00
Type: Journal Articles
Subjects: English Literature,English (Second Language),Second Language Learning,Second Language Instruction,Video Technology,Discussion (Teaching Technique),Language Teachers,Teacher Education Programs,Preservice Teachers,Foreign Countries,Role Playing,Drama,Student Attitudes,Barriers,Educational Benefits,Cooperative Learning,Computer Mediated Communication,Online Courses,Integrated Learning Systems

Literature instruction may serve multiple functions. This case study aimed at investigating the perceptions on the use of online literature circles among 62 first year Teaching English as a Second Language pre-service teachers in a literature course at Universiti Teknologi Malaysia. The participants were assigned to read and participate in online literature circles about the play Hamlet by William Shakespeare. Based on cooperative learning, the participants rotated in playing the roles of the Discussion Director, Device Detective, Imaginative Illustrator, Creative Connector, and Passage Picker in each group. Data from a survey were analyzed in descriptive statistics, while data from the role sheets based on the five roles and the online video of 3 literature circle discussions were analyzed thematically. The findings are significant for teacher training institutions and in-service teachers. This study was able to show that online literature circles were perceived as interesting, engaging, challenging, and fun. The participants engaged in different learning processes involving cognitive, affective, and language skills. Though this study identified some challenges, online literature circles have many benefits and are recommended for literature instruction for pre-service teacher training.
Designing Efficient L2 Writing Assessment Tasks for Low-Proficiency Learners of English. TOEFL® Research Report. RR-97. ETS RR-21-27
Author: Sasayama, Shoko
Date: 2021-12-00
Type: Journal Articles
Subjects: English (Second Language),Language Tests,Second Language Learning,Writing Tests,Test Construction,Language Proficiency,Guidelines,Rating Scales,Photography,Electronic Mail,Friendship,Computer Mediated Communication,Test Items,Writing (Composition),Task Analysis,Cross Cultural Studies,Foreign Countries,Syntax,Comparative Analysis

This report describes the development of efficient second language (L2) writing assessment tasks designed specifically for low-proficiency learners of English to be included in the &quot;TOEFL® Essentials&quot;™ test. Based on the can-do descriptors of the Common European Framework of Reference for Languages for the A1 through B1 levels of proficiency, four task types were identified to be prototypical candidate writing tasks for the target test-taker population (i.e., adolescent and adult low-proficiency English learners). Those four task types included: (a) Describe a Photo, (b) Write a Review, (c) Chat With a Friend, and (d) Write an E-mail. These task types were also considered efficient in the framework of the test in that they had the potential to be accessible to low-proficiency learners and to elicit sufficient spontaneous writing for assessment purposes within a short period of time. In the current study, eight assessment tasks, two for each task type, were developed and piloted with 169 A1-B1 learners of English from Japan and Colombia. The findings revealed that the Describe a Photo and Write an E-mail tasks performed the best in eliciting substantial language use and emphasizing distinct performance attributes, both characteristics needed for efficiently measuring test takers' writing proficiency as well as discriminating among proficiency levels at the lower end of the spectrum. The report concludes by highlighting some observations on L2 writing assessment task design for low-proficiency learners of English.
Phonological Constraints on the Utterance of L2 Clusters by Saudi ESL Learners
Author: Alotaibi, Abdullah N.
Date: 2021-12-00
Type: Journal Articles
Subjects: Foreign Countries,English (Second Language),Second Language Instruction,Phonology,Syllables,Semitic Languages,Phonemes,Native Language,Interference (Language),Young Adults,College Students

The field of L2 phonology did not receive much research compared to the other linguistic domains. To add to the field and expand the current literature, the present paper's goal was to examine the impact of syllable structure differences between Arabic and English in uttering L2 English consonant clusters. The following research question was aimed to be answered: Do the differences between two languages' syllable structure cause production difficulties in the consonant cluster to Saudi Arabian learners of English? The subjects of this investigation were L2 English learners from Saudi with intermediate proficiency levels in English. Applying the descriptive correlational type of research model, the results showed that learners' production is mainly influenced by their native language-specific phonological features. The learners' production of targeted L2 consonant clusters seemed to mirror their underlying phonological system, and syllables structures were modified to match their native Arabic phonological system as a result of language transfer. These findings should be taken into account by L2 speech educators as such speech difficulty is anticipated.
Benchmark Keystroke Biometrics Accuracy from High-Stakes Writing Tasks. Research Report. ETS RR-21-15
Author: Choi, Ikkyu
Date: 2021-12-00
Type: Journal Articles
Subjects: Accuracy,High Stakes Tests,Writing Tests,Benchmarking,Writing Processes,Word Processing,Essay Tests,Computer Security,Reliability,Measurement Techniques,Error of Measurement,Intervals,Computer Assisted Testing,COVID-19,Pandemics,High School Equivalency Programs,Classification,Learning Analytics

&quot;Biometrics&quot; are physical or behavioral human characteristics that can be used to identify a person. It is widely known that keystroke or typing dynamics for short, fixed texts (e.g., passwords) could serve as a behavioral biometric. In this study, we investigate whether keystroke data from essay responses can lead to a reliable biometric measure, with implications for test security and the monitoring of writing fluency and style changes. Based on keystroke data collected from a high-stakes writing testing setting, we established a preliminary biometric benchmark for detecting test-taker identity by using features extracted from their writing process logs. We report a benchmark keystroke biometric accuracy of equal error rate of 4.7% for identifying same versus different individuals on an essay task. In particular, we show that the inclusion of writing process features (e.g., features designed to describe the writing process) in addition to the widely used typing-timing features (e.g., features based on the time intervals between two-letter key sequences) improves the accuracy of the keystroke biometrics. The proposed keystroke biometrics can have important implications for the writing assessments administered through the remotely proctored tests that have been widely adopted during the COVID pandemic.
The Impact of the COVID-19 Pandemic on Primary School Students&apos; Mathematical Reasoning Skills: A Mediation Analysis
Author: Coskun, Kerem
Date: 2022-00-00
Type: Journal Articles
Subjects: COVID-19,Pandemics,Mathematical Logic,Elementary School Students,Mathematics Instruction,School Closing,Distance Education,Conventional Instruction,Mathematics Skills,Foreign Countries,Grade 1,Grade 2,Grade 3,Grade 4

The present research aimed to reveal how the COVID-19 pandemic influenced the mathematical reasoning of primary school students through mediation analysis. It was designed as ex post facto research. The research sample consisted of two cohorts. Cohort 1 included 415 primary school children who received face-to-face instruction by attending school for six months until COVID-19 emerged. Cohort 2 included 964 children who were taught curricular skills through distance education due to COVID-19 and school closures. In total, 1,379 primary school children were recruited into the research sample. Data were collected through a mathematical reasoning test by sending items from the instrument via Google Docs. The data were analysed with mediation analysis. Results demonstrated that the school closures due to the COVID-19 pandemic negatively influenced mathematical reasoning skills. Findings are discussed in the light of human interaction and Cattell's intelligence theory.