ethanporcaro/eric-fulltext-meta huggingface dataset:
Collective Digital Storytelling: An Activity-Theoretical Analysis of Second Language Learning and Teaching
Author: Kalyaniwala-Thapliyal, Carmenne
Date: 2016-00-00
Type: Journal Articles
Subjects: Technology Uses in Education,Story Telling,Second Language Learning,Second Language Instruction,English (Second Language),Cooperative Learning,Teaching Methods,Undergraduate Students,Foreign Countries,Information Technology,Elementary School Teachers

This paper describes the collective activity of a group of four students who created a digital story as a teaching resource that was to be used for teaching English as a foreign language. It uncovers and analyzes the actual processes underlining the activity as it unfolds from one stage to another. Four processes, viz., sociocognitive interactions, methodological processes, reflective processes and techno-semiopragmatic interactions were identified during the unfolding of the collective activity. Moreover, the results of this study highlight the role of the community in determining the activity of a lower level language group.
Adaptive Technologies. Research Report. ETS RR-07-05
Author: Shute, Valerie J.
Date: 2007-03-00
Type: Journal Articles
Subjects: Educational Technology,Educational Environment,Media Adaptation,Models,Educational Trends,Futures (of Society),Systems Approach

This paper describes research and development efforts related to adaptive technologies, which can be combined with other technologies and processes to form an adaptive system. The goal of an adaptive system, in the context of this paper, is to create an instructionally sound and flexible environment that supports learning for students with a range of abilities, disabilities, interests, backgrounds, and other characteristics. After defining key terms and establishing a rationale for adaptation, we present a general framework to organize adaptive technologies. We then describe experts' thoughts on what to adapt and how to adapt and conclude with a summary of key challenges and potential futures of adaptive technologies.
Parameter Recovery and Subpopulation Proficiency Estimation in Hierarchical Latent Regression Models. Research Report. ETS RR-07-27
Author: Li, Deping
Date: 2007-06-00
Type: Journal Articles
Subjects: Hierarchical Linear Modeling,Regression (Statistics),Computation,Comparative Analysis,Error of Measurement,Simulation

The hierarchical latent regression model (HLRM) is a flexible framework for estimating group-level proficiency while taking into account the complex sample designs often found in large-scale educational surveys. A complex assessment design in which information is collected at different levels (such as student, school, and district), the model also provides a mechanism for estimating group differences at various levels and for partitioning variance components among those levels. This study examines parameter recovery in the HLRM and compares it to regular latent regression models (LRMs) through simulation for various levels of cluster variation. Results show that regression effect estimates are similar between the HLRM and the LRM, in particular under small cluster variation. Similarly, student posterior mean estimates and marginal maximum likelihood mean estimates for student groups are comparable across the two model approaches. However, substantial differences are found for the residual variance estimates, the standard errors for regression effect estimates and related standard errors for group estimates, and for students posterior variance estimates. As expected, these differences are larger when the variation across clusters is larger, since a substantial portion of variance remains unexplained in LRM.
Focus on Formative Feedback. Research Report. ETS RR-07-11
Author: Shute, Valerie J.
Date: 2007-03-00
Type: Journal Articles
Subjects: Formative Evaluation,Feedback (Response),Learning Processes,Instructional Effectiveness,Student Evaluation,Individual Differences,Goal Orientation,Learning Motivation,Task Analysis,Difficulty Level,Scaffolding (Teaching Technique),Time Factors (Learning),Models,Guidelines,Literature Reviews

This paper reviews the corpus of research on feedback, with a particular focus on formative feedback--defined as information communicated to the learner that is intended to modify the learner's thinking or behavior for the purpose of improving learning. According to researchers in the area, formative feedback should be multidimensional, nonevaluative, supportive, timely, specific, credible, infrequent, and genuine (e.g., Brophy, 1981; Schwartz &amp; White, 2000). Formative feedback is usually presented as information to a learner in response to some action on the learner's part. It comes in a variety of types (e.g., verification of response accuracy, explanation of the correct answer, hints, worked examples) and can be administered at various times during the learning process (e.g., immediately following an answer, after some period of time has elapsed). Finally, there are a number of variables that have been shown to interact with formative feedback's success at promoting learning (e.g., individual characteristics of the learner and aspects of the task). All of these issues will be discussed in this paper. This review concludes with a set of guidelines for generating formative feedback.
Comparison of Multistage Tests with Computerized Adaptive and Paper-and-Pencil Tests. Research Report. ETS RR-07-04
Author: Rotou, Ourania
Date: 2007-03-00
Type: Journal Articles
Subjects: Comparative Analysis,Test Format,Computer Assisted Testing,Models,Test Length,Item Response Theory,Test Reliability,Error of Measurement,Test Content,Scores,Simulation,Statistical Analysis

Traditionally, the fixed-length linear paper-and-pencil (P&amp;P) mode of administration has been the standard method of test delivery. With the advancement of technology, however, the popularity of administering tests using adaptive methods like computerized adaptive testing (CAT) and multistage testing (MST) has grown in the field of measurement in both theory and practice. In practice, several standardized tests have sections that include only set-based items. To date, there is no study in the literature that compares these testing procedures when a test is completely set-based under various item response theory (IRT) models. This study investigates the measurement precision of MST compared to CAT and compared to P&amp;P tests for the one-, two-, and three-parameter logistic (1-, 2-, and 3PL) models when the test is completely set-based. Results showed that MST performed better for the 2- and 3PL models than an equivalent-length P&amp;P test in terms of reliability and conditional standard error of measurement. In addition, findings showed that MST performed better for the 1- and 2PL models than for an equivalent length CAT test. For the 3PL model, MST and CAT performed about the same.
A Note on Gain Scores and Their Interpretation in Developmental Models Designed to Measure Change in the Early School Years. Research Report. ETS RR-07-08
Author: Rock, Donald A.
Date: 2007-03-00
Type: Journal Articles
Subjects: Scores,Cognitive Development,Computation,Test Interpretation,Adaptive Testing,Children,Longitudinal Studies,Surveys,Computer Assisted Testing,Criterion Referenced Tests,Comparative Analysis,Kindergarten,Grade 1,Grade 3,Reading Achievement,Achievement Gains,Regression (Statistics)

This paper presents a strategy for measuring cognitive gains in reading during the early school years. It is argued that accurate estimates of gain scores and their appropriate interpretation requires the use of adaptive tests with multiple criterion referenced points that mark learning milestones. It is further argued that two different measures of gain are necessary because of the nonequivalence of scale score points. Empirical results are presented in support of the use of adaptive tests with multiple criteria referenced scale points for the measurement of gain.
Reliability and the Nonequivalent Groups with Anchor Test Design. Research Report. ETS RR-07-16
Author: Moses, Tim
Date: 2007-04-00
Type: Journal Articles
Subjects: Reliability,Equated Scores,Test Items,Statistical Analysis,True Scores,Comparative Analysis,Generalizability Theory,Scaling,Ability,Differences,Statistical Bias,Error of Measurement

This study evaluated the impact of unequal reliability on test equating methods in the nonequivalent groups with anchor test (NEAT) design. Classical true score-based models were compared in terms of their assumptions about how reliability impacts test scores. These models were related to treatment of population ability differences by different NEAT equating methods. A score model was then developed based on the most important features of the reviewed score models and used to study reliability in a simulation study across a total of 45 measurement conditions (= 5 test and anchor reliability combinations × 3 population ability difference conditions × 3 sample sizes). Ten equating methods were considered: chained linear, chained equipercentile with raw and smoothed frequencies, Tucker, frequency estimation equipercentile with raw and smoothed frequencies, Levine observed using Angoff-estimated and the &quot;correct&quot; reliabilities based on the data generation model used in this study, and Levine true using Angoff-estimated and correct reliabilities. The results were consistent with what is known about equating functions and their variability. Unequal and/or low reliability inflates equating function variability and alters equating functions when population abilities differ.
On Multidimensional Item Response Theory: A Coordinate-Free Approach. Research Report. ETS RR-07-30
Author: Antal, Tamás
Date: 2007-07-00
Type: Journal Articles
Subjects: Item Response Theory,Models,Test Items,Computation,Maximum Likelihood Statistics

A coordinate-free definition of complex-structure multidimensional item response theory (MIRT) for dichotomously scored items is presented. The point of view taken emphasizes the possibilities and subtleties of understanding MIRT as a multidimensional extension of the classical unidimensional item response theory models. The main theorem of the paper is that every monotonic MIRT model looks the same; they are all trivial extensions of univariate item response theory.
The Standardized Letter of Recommendation: Implications for Selection. Research Report. ETS RR-07-38
Author: Liu, Ou Lydia
Date: 2007-08-00
Type: Journal Articles
Subjects: Letters (Correspondence),Graduate Students,College Applicants,Cognitive Ability,Student Evaluation,Knowledge Level,Thinking Skills,Communication Skills,Learning Motivation,Self Management,Professionalism,Maturity (Individuals),Teamwork,Item Response Theory,Internship Programs,Admission Criteria,Psychometrics,Reliability,Goodness of Fit,Test Items,Item Analysis,Predictor Variables,Correlation,Likert Scales,Evaluation Criteria,Maximum Likelihood Statistics

In an effort to standardize academic application procedures, the Standardized Letter of Recommendation (SLR) was developed to capture important cognitive and noncognitive qualities of graduate school candidates. The SLR consists of seven scales (&quot;knowledge,&quot; &quot;analytical skills,&quot; &quot;communication skills,&quot; &quot;motivation,&quot; &quot;self- organization,&quot; &quot;professionalism and maturity,&quot; and &quot;teamwork&quot;) and was applied to an intern-selection scenario. Both professor ratings (N = 414) during the application process and mentor ratings of the selected students (N = 51) after the internship was completed were collected using the SLR. A multidimensional Rasch investigation suggests that the seven scales of the SLR displayed satisfactory psychometric properties in terms of reliability, model fit, item fit statistics, and discrimination. The two cognitive scales, &quot;knowledge&quot; and &quot;analytical skills,&quot; were found to be the best predictors for intern selection. The professor ratings and mentor ratings had moderate to high correlations, with the professor ratings being systematically higher than the mentor ratings. Possible reasons for the rating discrepancies are discussed. Also, implications for how the SLR can be used and improved in other selection situations are suggested.
An Illustration of the Use of Markov Decision Processes to Represent Student Growth (Learning). Research Report. ETS RR-07-40
Author: Almond, Russell G.
Date: 2007-11-00
Type: Journal Articles
Subjects: Markov Processes,Decision Making,Student Evaluation,Learning Processes,Academic Achievement,Computation,At Risk Students,Identification,Models,Cognitive Development,Simulation,Benchmarking,Reliability,Bayesian Statistics

Over the course of instruction, instructors generally collect a great deal of information about each student. Integrating that information intelligently requires models for how a student's proficiency changes over time. Armed with such models, instructors can &quot;filter&quot; the data--more accurately estimate the student's current proficiency levels--and &quot;forecast&quot; the student's future proficiency levels. The process of instructional planning can be described as a &quot;partially observed Markov decision process&quot; (POMDP). Recently developed computer algorithms can be used to help instructors create strategies for student achievement and identify at-risk students. Implementing this vision requires models for how instructional actions change student proficiencies. The &quot;general action model&quot; (also called the &quot;bowtie model&quot;) separately models the factors contributing to the success or effectiveness of an action, proficiency growth when the action is successful, and proficiency growth when the action is unsuccessful. This class of models requires parameterization, and this paper presents two: a simple linear process model (suitable for continuous proficiencies) and a birth-and-death process model (for proficiency scales expressed as ordered categorical variables). Both models show how to take prerequisites and zones of proximal development into account. The filtering process is illustrated using a simple artificial example.
Refinement of a Bias-Correction Procedure for the Weighted Likelihood Estimator of Ability. Research Report. ETS RR-07-23
Author: Zhang, Jinming
Date: 2007-05-00
Type: Journal Articles
Subjects: Item Response Theory,Ability,Error of Measurement,Maximum Likelihood Statistics,Computation,Statistical Bias,Comparative Analysis,Simulation

In practical applications of item response theory (IRT), item parameters are usually estimated first from a calibration sample. After treating these estimates as fixed and known, ability parameters are then estimated. However, the statistical inferences based on the estimated abilities can be misleading if the uncertainty of the item parameter estimates is ignored. Instead, estimated item parameters can be regarded as covariates measured with error. Along the line of this measurement-error-model approach, asymptotic expansions of the maximum likelihood estimator (MLE) and weighted likelihood estimator (WLE) of ability were derived by Zhang, Xie, Song, and Lu (2007). In this paper, we propose an estimator of an ability parameter based on the asymptotic formula of the WLE. A simulation study shows that the new estimator effectively reduces the bias of the MLE or WLE of ability caused by the uncertainty of the item parameter estimates not taken into account.
The Impact of Anchor Test Length on Equating Results in a Nonequivalent Groups Design. Research Report. ETS RR-07-44
Author: Ricker, Kathryn L.
Date: 2007-12-00
Type: Journal Articles
Subjects: Equated Scores,Test Items,Statistical Analysis,Test Length,Comparative Analysis,Licensing Examinations (Professions),Correlation

This study explored the effects of external anchor test length on final equating results of several equating methods, including equipercentile (frequency estimation), chained equipercentile, kernel equating (KE) poststratification PSE with optimal bandwidths, and KE PSE linear (large bandwidths) when using the nonequivalent groups anchor test (NEAT) design. This study used pseudotests constructed of item responses from a real operational test. The equating methods were evaluated using an equating criterion. Conditional differences between the criterion scores and equated scores, and root mean square error of the differences (RMSE) were used as measures to compare the methods to the criterion equating, which in this study is an equivalent groups (EG) equipercentile equating function. The results indicate that bias tended to increase in the conversions as the anchor test length decreased, but the KE PSE with optimal bandwidths and equipercentile (frequency estimation) methods were less sensitive to this change than the other methods. The KE PSE linear method with large bandwidths performed poorly compared to the criterion across all anchor test lengths. [This paper was originally presented at the annual meetings of the American Educational Research Association (AERA) and the National Council on Measurement in Education (NCME), held between April 10 and April 12, 2006, in San Francisco, CA.]
The Impact of Short-Term Use of Criterion? on Writing Skills in Ninth Grade. Research Report. ETS RR-07-07
Author: Rock, JoAnn Leah
Date: 2007-03-00
Type: Journal Articles
Subjects: Hierarchical Linear Modeling,Teaching Methods,Grade 9,English Instruction,Essays,Writing Assignments,Writing Skills,Scores,Feedback (Response),Educational Technology,Technology Uses in Education,High School Students,Control Groups,Experimental Groups,Instructional Effectiveness

In this study, a hierarchical linear model (HLM) was used within a cluster random assignment design to test the impact of &quot;Criterion&quot;? on students' writing skill when &quot;Criterion&quot; was used as a supplementary instructional tool in ninth grade English classrooms over a 4 week period. The study was conducted in three states: Indiana, Pennsylvania, and New Jersey. The sample consisted of 1,464 ninth graders aggregated across 10 schools, 34 teachers, and 61 classes. Findings indicate that over a short time period, &quot;Criterion&quot; use improved the mechanical aspects of student essays as measured by the analytic scores given to their essays. Students in ninth grade English classes who used &quot;Criterion&quot; several times a week over a 1 month period received higher analytic scores on their essays written at the end of the study period than those who were in classes that did not. The effect was small, but statistically significant at the 0.05 level using a one-tailed test. These findings suggest a role for automated feedback systems in supporting writing instruction.
An Exploration of Kernel Equating Using SAT® Data: Equating to a Similar Population and to a Distant Population. Research Report. ETS RR-07-17
Author: Liu, Jinghua
Date: 2007-04-00
Type: Journal Articles
Subjects: College Entrance Examinations,Equated Scores,Comparative Analysis,Evaluation Methods,Raw Scores,Regression (Statistics),Error of Measurement,Beginning Teachers,Verbal Ability,Test Format,Correlation

This study applied kernel equating (KE) in two scenarios: equating to a very similar population and equating to a very different population, referred to as a distant population, using SAT® data. The KE results were compared to the results obtained from analogous classical equating methods in both scenarios. The results indicate that KE results are comparable to the results of other methods. Further, the results show that when the two populations taking the two tests are similar on the anchor score distributions, different equating methods yield the same or very similar results, even though they have different assumptions.
Beyond &quot;Empty Verbalism&quot;: How Teacher Candidates Benefit from Blogging about a Tutoring Practicum
Author: Howard, Kathryn
Date: 2015-00-00
Type: Journal Articles
Subjects: Preservice Teachers,Electronic Publishing,Web Sites,Tutoring,Practicums,English (Second Language),Reading Instruction,Second Language Learning,Electronic Learning,Comparative Analysis,Educational Practices,Teacher Education,Tutors,Online Courses,Professional Development,Peer Teaching,Tutorial Programs,Reflection,Learning Activities

TESOL programs and courses around the world are increasingly offered partly or wholly online. Online instruction offers both new affordances and distinct challenges for effective instruction, particularly when it comes to supervising fieldwork. This article compares 2 distinct online formats for student reflections on their tutoring experiences in the practicum component of a course on teaching second language reading. In particular, the article examines how 2 different reflection formats afforded qualitatively distinct student reflections on how they modified their understandings of learning, learners, teaching, and the contexts of learning through (a) interacting with their tutee, (b) implementing new instructional practices, and (c) interacting with their peers about their tutorial. Findings are discussed in terms of the affordances and challenges provided by distinct ways of configuring the online reflection for these field experiences, comparing student work in blogs, discussion boards, and reports submitted individually to the instructor.
Scale Drift in Equating on a Test That Employs Cut Scores. Research Report. ETS RR-07-34
Author: Puhan, Gautam
Date: 2007-06-00
Type: Journal Articles
Subjects: Equated Scores,Cutting Scores,Scaling,Statistical Analysis,Tests

The purpose of this study is to determine the extent of scale drift on a test that employs cut scores. It is essential to examine scale drift in a testing program using new forms that are often put on scale through a series of intermediate equatings (known as equating chains). This may cause equating error to accumulate to a point where scale scores are rendered incomparable across two parallel chains or time periods. The study examined whether scale drift occurred for two conditions (i.e., parallel equating chains and a single long chain). Data from three tests that employed cut scores were used in this study. Results indicated that although there was some difference observed between the actual equating conversions derived via different equating chains, the effect of these differences on actual pass/fail status of test takers was not large. However, these results may change if new cut scores are introduced. Also, for tests that do not employ cut scores, stability in most parts of the score scale is important, not just at particular cut score points.
Benefits and Challenges of Supervising an International Practicum
Author: Gilliland, Betsy
Date: 2015-00-00
Type: Journal Articles
Subjects: Foreign Countries,Practicums,Teacher Education,Mentors,Beginning Teachers,English (Second Language),Second Language Learning,Language Teachers,Graduate Students,Teaching Methods,Teacher Student Relationship,Supervisors,College Faculty,Cross Cultural Studies,English for Academic Purposes

In this reflective essay, I share my experiences as a US-based professor leading graduate students on a 2-month teaching practicum in northern Thailand. I describe the process of arranging the practicum in coordination with the host university and the challenges of teaching 2 teacher-education courses while also mentoring novice teachers and addressing cross-cultural issues. I discovered that leading a practicum in an international context required me to play multiple roles--as supervisor, professor, and personal mentor. The teachers encountered challenges but also experienced empowerment working in a foreign language context; they wrestled with cultural issues such as negotiating among their expectations as teachers, the university's demands, and my requirements from them as students. The international practicum setting means that teachers need to develop an understanding of a new culture and how they fit personally into that culture. Being far from home requires everyone--teachers and supervisors--to form a new community supporting each other and to redraw previously established professor-student boundaries.