ethanporcaro/eric-fulltext-meta huggingface dataset:
Language Anxiety: A Case Study of the Perceptions and Experiences of Students of English as a Foreign Language in a Higher Education Institution in the United Arab Emirates
Author: Lababidi, Rola Ahmed
Date: 2016-00-00
Type: Journal Articles
Subjects: Anxiety,Communication (Thought Transfer),Males,College Students,Second Language Learning,Second Language Instruction,Foreign Countries,Student Attitudes,Severity (of Disability),Interviews,Learning Strategies,Case Studies,Qualitative Research,Statistical Analysis,Focus Groups

This case study explores and investigates the perceptions and experiences of foreign language anxiety (FLA) among students of English as a Foreign Language in a Higher Education Institution in the United Arab Emirates. The first phase explored the scope and severity of language anxiety among all Foundation level male students at a college in the UAE. In the second phase of the study, focus group (FG) interviews with students were conducted to gain a deeper understanding of the possible sources of anxiety about language learning and its manifestations, as well as consideration of the strategies that may be used to alleviate its negative effects. The findings indicate that some of the student participants in this case study experienced moderate to high levels of anxiety in the second language classrooms. The findings from the FG indicate that foreign language anxiety could be attributed to a number of variables. The results highlight a number of implications and recommendations for future research.
Weighting Procedures and the Cluster Forming Algorithm for Delete-k Jackknife Variance Estimation for Institutional Surveys. Research Report. ETS RR-06-15
Author: Qian, Jiahe
Date: 2006-06-00
Type: Journal Articles
Subjects: Weighted Scores,Sampling,Statistical Analysis,Higher Education,Surveys,Information Technology,Probability,Inferences,Literacy,Computation,Gender Differences,Error of Measurement

Weighting and variance estimation are two statistical issues involved in survey data analysis for large-scale assessment programs such as the Higher Education Information and Communication Technology (ICT) Literacy Assessment. Because survey data are always acquired by probability sampling, to draw unbiased or almost unbiased inferences for the populations, weights are required in making use of estimators such as a Horvitz-Thompson type. Variance estimation provides the basis for reporting errors. The weighting procedure generates weights based on statistical principles that are consistent with the sampling design. The estimation of the variance from survey data uses the delete-&quot;k&quot; jackknife resampling replicate (JRR) approach, which can be adapted for variant institutional sampling designs and for dissimilarity in institute conditions. To form clusters of &quot;k&quot; cases, a merge-dilute algorithm is proposed. The algorithm merges the cases of different groups into a queue and then allocates the cases of the queue to form homogeneous clusters of required sizes. The new algorithm is applied to the ICT sample from an institute taking the 2004 fall trial assessment.
An Elementary Test of the Normal 2PL Model against the Normal 3PL Alternative. Research Report. ETS RR-06-14
Author: Haberman, Shelby J.
Date: 2006-06-00
Type: Journal Articles
Subjects: Statistical Analysis,Models,Licensing Examinations (Professions),Teacher Certification,Scores

A simple score test of the normal two-parameter logistic (2PL) model is presented that examines the potential attraction of the normal three-parameter logistic (3PL) model for use with a particular item. Application is made to data from a test from the Praxis™ series. Results from this example raise the question whether the normal 3PL model should be used routinely in preference to the normal 2PL model unless evidence exists that a substantial gain in description of data is achieved.
Using Past Data to Enhance Small-Sample DIF Estimation: A Bayesian Approach. Research Report. ETS RR-06-09
Author: Sinharay, Sandip
Date: 2006-05-00
Type: Journal Articles
Subjects: Test Bias,Computation,Sample Size,Bayesian Statistics,Statistical Analysis,Licensing Examinations (Professions),Teacher Certification,Gender Differences,Racial Differences

The application of the Mantel-Haenszel test statistic (and other popular DIF-detection methods) to determine DIF requires large samples, but test administrators often need to detect DIF with small samples. There is no universally agreed upon statistical approach for performing DIF analysis with small samples; hence there is substantial scope of further work on the problem. One advantage of a Bayesian approach over a frequentist approach is that the former can incorporate, in the form of a prior distribution, existing information on the inference problem at hand; a prior distribution often leads to improved estimation, especially for small samples. Further, for any operational test, a huge volume of past data is available, and for any item appearing in a present test, there is a high chance that a number of similar items have appeared on past operational administrations of the test. Therefore, ideally, it will be possible to use that past information as a prior distribution in a Bayesian DIF analysis. This paper discusses how to perform such an analysis. The suggested Bayesian DIF analysis method is shown to be an improvement over the existing methods in a realistic simulation study.
Joint and Conditional Estimation for Implicit Models for Tests with Polytomous Item Scores. Research Report. ETS RR-06-03
Author: Haberman, Shelby J.
Date: 2006-03-00
Type: Journal Articles
Subjects: Computation,Models,Test Items,Scores,Item Response Theory,Maximum Likelihood Statistics,Probability,Reliability

Multinomial-response models are available that correspond implicitly to tests in which a total score is computed as the sum of polytomous item scores. For these models, joint and conditional estimation may be considered in much the same way as for the Rasch model for right-scored tests. As in the Rasch model, joint estimation is only attractive if both the number of items and the number of examinees are large, while conditional estimation can be employed for a large number of examinees whether or not the number of items is large. In neither case is computation difficult given currently available computers. Large-sample results favor use of conditional estimation, although some use of joint estimation can be contemplated if the number of items is large.
The Correlation between the Scores of a Test and an Anchor Test. Research Report. ETS RR-06-04
Author: Sinharay, Sandip
Date: 2006-03-00
Type: Journal Articles
Subjects: Test Items,Equated Scores,Correlation,Tests,Statistical Data,Simulation,Difficulty Level,Item Response Theory,Regression (Statistics)

It is a widely held belief that an anchor test used in equating should be a miniature version (or &quot;minitest&quot;) of the tests to be equated; that is, the anchor test should be proportionally representative of the two tests in content and statistical characteristics. This paper examines the scientific foundation of this belief, especially regarding statistical characteristics. In particular, we study, through several simulation studies and a real data example, the correlation of the raw score of the total test with several anchor tests, including those having statistical characteristics that differ from those of the total test. Anchor tests with spread of item difficulties less than that of a total test are shown to consistently have higher correlations than a minitest. Because of the general view that higher correlations of the scores of an anchor test and the total test lead to better equating, our results suggest that requiring an anchor test to mimic the statistical characteristics of the total test may be too restrictive and may not be optimal.
An Evaluation of the Kernel Equating Method: A Special Study with Pseudotests Constructed from Real Test Data. Research Report. ETS RR-06-02
Author: von Davier, Alina A.
Date: 2006-03-00
Type: Journal Articles
Subjects: Equated Scores,Statistical Analysis,Simulation,Tests,Test Items,Comparative Analysis,Error of Measurement

This study examines how closely the kernel equating (KE) method (von Davier, Holland, &amp; Thayer, 2004a) approximates the results of other observed-score equating methods--equipercentile and linear equatings. The study used pseudotests constructed of item responses from a real test to simulate three equating designs: an equivalent groups (EG) design and two non-equivalent groups with anchor test (NEAT) designs, one with an internal anchor and another with an external anchor. To compare results, the study sets the equating function in the EG design as the equating criterion. In these examples, the KE results were very close to the results from the other equating methods. Moreover, in almost all situations investigated, the KE results were closer to the equating criterion.
Linking Competencies in Educational Settings and Measuring Growth. Research Report. ETS RR-06-12
Author: von Davier, Alina A.
Date: 2006-06-00
Type: Journal Articles
Subjects: Scores,Measurement Techniques,Competence,Comparative Analysis,Data Collection,Scaling,Item Response Theory,Equated Scores,Longitudinal Studies,Growth Models,Structural Equation Models,Hierarchical Linear Modeling

Measuring and linking competencies require special instruments, special data collection designs, and special statistical models. The measurement instruments are tests or tests forms, which can be used in the following situations: The same test can be given repeatedly; two or more parallel tests forms (i.e., forms intended to be similar in difficulty and content and that still need to equated) can be given at different time points; or two or more test forms that may be less parallel to various degrees can be given at different points in time. In some circumstances, the goal of the analysis is to make the scores of parallel test forms comparable across different time points and different samples of relatively similar ability (horizontal equating). In other situations, we aim at the comparability of scores of tests forms that are not parallel, and although they are intended to measure the same competencies, they are of different difficulties and are taken by samples that show large differences in ability (vertical linking). In other cases, we want to evaluate the change in competencies over time (with or without covariates) for the same individuals measured by the same instrument or by different instruments (longitudinal linking). This paper will briefly discuss a variety of techniques for relating scale scores from different data collections points and will then discuss models for measuring growth. Each of these areas is a large field in itself, and each has a potential strong impact on educational policies such as the No Child Left Behind Act (for example, the vertical linking of state or national assessments and longitudinal studies are a potential basis for informative analyses for the policy makers), on the life of students and parents (equating of achievement tests), or on the life of professionals (equating of licensure tests). In these times when more and more standardized testing is used for assessing competencies in different domains nationally and internationally, we are also discovering more challenges in ensuring that the process and the results are fair and accurate. In turn, these challenges and these new social implications open the door toward more research in support of fair assessments, both in improving upon the test construction process and in advancing the statistical methods involved.
The Effectiveness of Geography Student Worksheet to Develop Learning Experiences for High School Students
Author: Utami, Wiwik Sri
Date: 2016-00-00
Type: Journal Articles
Subjects: Worksheets,Learning Experience,High School Students,Geography Instruction,Instructional Effectiveness,Skill Development,Achievement Gains,Natural Resources,Resource Allocation,Pretests Posttests,Skill Analysis,Foreign Countries

The purpose of this study is to examine the effectiveness of geography student worksheet in developing high school students' learning experiences. The student worksheet was planned to gain opportunity to develop creative and geography skills. The effectiveness is assessed from the contribution of the worksheets in improving the skills of geography, geography learning outcomes, and students' responses in learning geography worksheet. The students' geography skills are considered good, learning outcomes also increased. Worksheet is able to increase the knowledge of learners. The response of student is very good towards geography worksheet, learning becomes active, interesting and not boring.
Designing Adaptive, Diagnostic Math Assessments for Individuals with and without Visual Disabilities. Research Report. ETS RR-06-01
Author: Shute, Valerie J.
Date: 2006-01-00
Type: Journal Articles
Subjects: Middle School Students,Secondary School Mathematics,Mathematics Instruction,Visual Impairments,Academic Accommodations (Disabilities),Evaluation Methods,Adaptive Testing,Design,Test Construction,Mathematics Tests,Accessibility (for Disabled),Summative Evaluation,Auditory Stimuli,Tactual Perception,Technology Uses in Education

This report summarizes the design and development of an adaptive e-learning prototype for middle school mathematics for use with both sighted and visually disabled students. Adaptation refers to the system's ability to adjust itself to suit particular characteristics of the learner. The main parts of the report describe the system's theoretical foundation, architecture, models, and adaptive algorithm. We also review approaches for making assessment systems accessible to students with visual disabilities. Finally, we conclude with a summary of upcoming studies in relation to important research questions concerning micro- and macroadaptation. Using a design approach like the one described in this report may set a new precedent for environments that adapt to support student learning based on larger sets of incoming abilities and disabilities than have been considered previously.
A SAS Macro for Loglinear Smoothing: Applications and Implications April. Research Report. ETS RR-06-05
Author: Moses, Tim P.
Date: 2006-04-00
Type: Journal Articles
Subjects: Statistical Analysis,Statistical Distributions,Equated Scores,Models,Goodness of Fit,Computer Software,Computer Oriented Programs

The two purposes of this paper are to provide a SAS IML macro that performs loglinear smoothing and to apply this macro to loglinear smoothing problems that have not been extensively discussed in the test-equating literature. The SAS macro is demonstrated on univariate, bivariate, and trivariate smoothing problems. The univariate and bivariate examples reproduce published results (von Davier, Holland, &amp; Thayer, 2004). The trivariate example extends the bivariate smoothing example to allow for comparisons of subgroups' univariate and bivariate distributions. The implications are that important questions about distribution differences and subpopulation invariance of equating functions can be considered through comparisons and evaluations of complex loglinear models that are easily fit with this SAS IML macro.
Cognitive Diagnosis for NAEP Proficiency Data. Research Report. ETS RR-06-08
Author: Xu, Xueli
Date: 2006-05-00
Type: Journal Articles
Subjects: Statistical Analysis,National Competency Tests,Diagnostic Tests,Item Response Theory,Models,Data Analysis,Guidelines,Simulation,Comparative Analysis,Item Analysis,Mastery Learning,Reading Tests,Grade 12,Test Items,Gender Differences,Racial Differences,Scores,Mathematics Tests,High School Students,Limited English Speaking,Disabilities

More than a dozen statistical models have been developed for the purpose of cognitive diagnosis. These models are supposed to extract a much finer level of information from item responses than traditional unidimensional item response models. In this paper, a general diagnostic model (GDM) was used to analyze a set of simulated sparse data and real data from National Assessment of Educational Progress (NAEP) assessments. The purpose of this study was to find out (a) whether the parameters can be recovered for a sparse data matrix in the framework of the GDM and (b) how to estimate group characteristics for large survey data, such as NAEP, in the framework of the GDM. The results of the simulation study show that GDM parameters can be recovered satisfactorily. The GDM under single group and multiple group assumptions were employed to fit NAEP assessment data. The results under these two assumptions and between the GDM and the statistics derived using the operational NAEP model were compared. The findings indicate that it is possible to conduct cognitive diagnosis for NAEP proficiency data.
Strategies in Responding to the New TOEFL Reading Tasks. TOEFL Monograph Series. MS-33. ETS RR-06-06
Author: Cohen, Andrew D.
Date: 2006-04-00
Type: Journal Articles
Subjects: Reading Tests,Test Items,Courseware,Item Analysis,Second Language Learning,Second Language Instruction,English (Second Language),Language Tests,Reading Strategies,Familiarity,Multiple Choice Tests,Simulation,Student Attitudes,Chinese,Japanese,Korean,Pretests Posttests,Asians,Difficulty Level,Foreign Students,Qualitative Research,Statistical Analysis

This study describes the reading and test-taking strategies that test takers used in the Reading section of the LanguEdge courseware (ETS, 2002a). These materials were developed to familiarize prospective respondents with the new TOEFL®. The investigation focused on strategies used to respond to more traditional single selection multiple-choice formats (i.e., Basic Comprehension and Inferencing questions) and the new selected-response (multiple selection, drag and drop) Reading to Learn items. The latter were designed to simulate the academic task of forming a comprehensive and coherent representation of an entire text, rather than focusing on discrete points in the text. Verbal report data were collected from 32 students, representing four language groups (Chinese, Japanese, Korean, and Other) doing the Reading section tasks from the LanguEdge courseware materials. Students were randomly assigned to two of the six reading subtests, each consisting of a 600-700 word text with 12-13 items. Subjects' verbal report accompanying items representing each of the ten item types was evaluated to determine strategy use. The findings provide insights into the response behaviors prompted by the reading tasks on the new TOEFL.
Testing the Untestable Assumptions of the Chain and Poststratification Equating Methods for the NEAT Design. Research Report. ETS RR-06-17
Author: Holland, Paul W.
Date: 2006-06-00
Type: Journal Articles
Subjects: Equated Scores,Test Items,Statistical Analysis,Comparative Analysis,Prediction,Accuracy,Licensing Examinations (Professions),Elementary School Teachers,Teacher Certification,Multiple Choice Tests,Goodness of Fit

This paper focuses on the Non-Equivalent Groups with Anchor Test (NEAT) design for test equating and on two classes of observed--score equating (OSE) methods--chain equating (CE) and poststratification equating (PSE). These two classes of methods reflect two distinctly different ways of using the information provided by the anchor test for computing OSE functions. Each of the two classes includes linear and nonlinear equating methods. In practical situations, it is known that the PSE and CE methods tend to give different results when the two groups of examinees differ in ability. However, given that both methods are justified by making untestable assumptions, it is difficult to conclude which, if either, of the two equating approaches is more correct. This study compares predictions from both the PSE and the CE assumptions that can be tested in a comparable way with the data from a special study. Results indicate that both CE and PSE make very similar predictions but that those of CE are slightly more accurate than those of PSE.
A General Diagnostic Model Applied to Language Testing Data. Research Report. ETS RR-05-16
Author: von Davier, Matthias
Date: 2005-09-00
Type: Journal Articles
Subjects: Models,Diagnostic Tests,Language Tests,Language Proficiency,Generalization,Markov Processes,Monte Carlo Methods,Maximum Likelihood Statistics,Item Response Theory,Multivariate Analysis,Simulation,Computer Assisted Testing,English (Second Language),Second Language Learning,Profiles,Language Skills,Test Items

Probabilistic models with more than one latent variable are designed to report profiles of skills or cognitive attributes. Testing programs want to offer additional information beyond what a single test score can provide using these skill profiles. Many recent approaches to skill profile models are limited to dichotomous data and have made use of computationally intensive estimation methods like the Markov chain Monte Carlo (MCMC), since standard maximum likelihood (ML) estimation techniques were deemed infeasible. This paper presents a class of general diagnostic models (GDMs) that can be estimated with customary ML techniques and applies to polytomous response variables as well as to skills with two or more proficiency levels. The model and the algorithm for estimating model parameters handles directly missing responses without the need of collapsing categories or recording the data. Within the class of GDMs, compensatory as well as noncompensatory models may be specified. This report uses one member of this class of diagnostic models, a compensatory diagnostic model that is parameterized similar to the generalized partial credit model (GPCM). Many well-known models, such as uni- and multivariate versions of the Rasch model and the two parameter logistic item response theory (2PL-IRT) model, the GPCM, and the FACETS model, as well as a variety of skill profile models, are special cases of this member of the class of GDMs. This paper describes an algorithm that capitalizes on using tools from item response theory for scale linking, item fit, and parameter estimation. In addition to an introduction to the class of GDMs and to the partial credit instance of this class for dichotomous and polytomous skill proles, this paper presents a parameter recovery study using simulated data and an application to real data from the field test for TOEFL® Internet-based testing (iBT).
Precision and Volatility in School Accountability Systems. Research Report. ETS RR-06-26
Author: Way, Walter D.
Date: 2006-09-00
Type: Journal Articles
Subjects: Accountability,Scores,Measurement Techniques,Educational Change,Educational Legislation,Federal Legislation,Educational Quality,Scaling,Educational Research,Correlation,Educational Improvement,Models,Accuracy

This paper contrasts the positions of Kane and Staiger (2002) and Linn and Haug (2002) with that of Rogosa (2002, 2003a) on the volatility in test scores as measures of school growth. In particular, these researchers disagree on whether school growth can be measured reliably in school accountability systems. The different positions of these authors are examined in some detail. In addition, several issues related to their debate are examined in the context of the No Child Left Behind (NCLB) legislation, which has imposed a particular structure on the accountability systems currently in place in many states. Finally, some possibilities and speculations about the future of school accountability systems are provided in relation to the NCLB requirements and their impact.