ethanporcaro/eric-fulltext-meta huggingface dataset:
When a Picture Isn&apos;t Worth 1000 Words: Learners Struggle to Find Meaning in Data Visualizations
Author: Stofer, Kathryn A.
Date: 2016-08-00
Type: Journal Articles
Subjects: Visualization,Pattern Recognition,Semi Structured Interviews,Oceanography,Higher Education,Constructivism (Learning),Geography,Comparative Analysis,Adults,Adult Students,Expertise,Novices

The oft-repeated phrase &quot;a picture is worth a thousand words&quot; supposes that an image can replace a profusion of words to more easily express complex ideas. For scientific visualizations that represent profusions of numerical data, however, an untranslated academic visualization suffers the same pitfalls untranslated jargon does. Previous research and commentary suggests scaffolding from cognitive, constructivist, and sociocultural traditions to promote meaning-making by broad audiences, but limited empirical research examines the effectiveness of these scaffolds for adult learners viewing global ocean data. Five versions of visualizations including translating titles and measurement units, altering color schemes, and geographic labels were presented on three topics to expert oceanographers and novice nonscientists. Qualitative analysis of semistructured clinical interviews suggest that these scaffolds assist these audiences but are not sufficient for novices to make meaning similar to experts without further instruction or assistance in interpreting and judging patterns of data in visualizations.
Adaptive Numerical Integration for Item Response Theory. Research Report. ETS RR-07-06
Author: Antal, Tamás
Date: 2007-03-00
Type: Journal Articles
Subjects: Item Response Theory,Computation,Regression (Statistics),National Competency Tests,Mathematics

Well-known numerical integration methods are applied to item response theory (IRT) with special emphasis on the estimation of the latent regression model of NAEP [National Assessment of Educational Progress]. An argument is made that the Gauss-Hermite rule enhanced with Cholesky decomposition and normal approximation of the response likelihood is a fast, precise, and reliable alternative for the numerical integration in NAEP and in IRT in general.
Disclosure Risk in Educational Surveys: An Application to the National Assessment of Educational Progress. Research Report. ETS RR-07-24
Author: Oranje, Andreas
Date: 2007-06-00
Type: Journal Articles
Subjects: Student Surveys,Risk,National Competency Tests,Data Analysis,Data Collection,Sampling,Computation,Statistical Inference,Information Security,Problems,Error of Measurement,Statistical Bias,Statistical Surveys,Disclosure,Grade 4,Grade 8

In this paper, a data perturbation method for minimizing the possibility of disclosure of participants' identities on a survey is described in the context of the National Assessment of Educational Progress (NAEP). The method distinguishes itself from most approaches because of the presence of cognitive tasks. Hence, a data edit should have minimal impact on both relations among demographic variables and relations between demographic and proficiency variables. Furthermore, since only a few students are at risk to be disclosed in a typical sampling setting common to educational surveys, the proposed data perturbation is governed by a nonuniform probabilistic process. The method is applied to data from NAEP and impact is computed using proficiency averages, demographic proportions, statistical inference results, and loglinear models. Results show that the proposed perturbation method has very little impact on NAEP results, even at relatively large editing rates. Some data coarsening results are reported as well. While the univariate results are relatively unaffected from the coarsening, loglinear models from higher order contingency tables are affected. It is recommended to restrict disclosure limitation techniques to perturbation methods in the case of NAEP.
Sensitivity to Landscape Features: A Spatial Analysis of Field Geoscientists on the Move
Author: Baker, Kathleen M.
Date: 2016-08-00
Type: Journal Articles
Subjects: Problem Solving,Correlation,Geographic Information Systems,Geography Instruction,Spatial Ability,Geology,Questionnaires,Maps,Thinking Skills,Critical Thinking,Expertise

Intelligent behavior in everyday contexts may depend on both ability and an individual's disposition toward using that ability. Research into patterns of thinking has identified three logically distinct components necessary for dispositional behavior: ability, inclination, and sensitivity. Surprisingly, sensitivity appears to be the most common bottleneck to achieving quality outcomes. Using global positioning system data taken when field geologists (n = 66) performed a bedrock mapping task, we assess sensitivity to landscape and its relationship to expertise, content knowledge in the discipline, thoroughness of route, and quality of outcome. A discipline-specific concept inventory served as a proxy measure for participant knowledge and ability. Thoroughness in the field served as an indicator of a participant's inclination to employ abilities critical to the success of the task. To assess sensitivity to opportunities in the landscape, we identified relatively small areas frequently visited by experienced geologists, then quantified the frequency with which participants across a continuum of mapping experience placed themselves in these areas. Results showed clear spatial differences in landscape use across scales by expertise level, suggesting that sensitivity to physical locations for quality thinking is less pronounced among inexperienced field geologists and correlates strongly with quality of outcome. These results are significant for the education of field researchers because cultivation of sensitivity to opportunities in the landscape for higher-quality thinking presents a different sort of instructional challenge than that of directly teaching discipline specific content or traditional mapping skills.
Growth in Reading Performance during the First Four Years in School. Research Report. ETS RR-07-39
Author: Rock, Donald A.
Date: 2007-10-00
Type: Journal Articles
Subjects: Reading Achievement,Achievement Gains,Elementary School Students,Longitudinal Studies,Multiple Regression Analysis,Bayesian Statistics,Learning Processes,Early Childhood Education,Cognitive Development,Children,Surveys,Acceleration (Education),Reading Tests,Racial Differences,Minority Group Students,White Students,African American Students,Asian American Students,Gender Differences,Parent Background

This study addressed concerns about the potential for differential gains in reading during the first 2 years of formal schooling (K-1) versus the next 2 years of schooling (1st-3rd grade). A multilevel piecewise regression with a node at spring 1st grade was used in order to define separate regressions for the two time periods. Empirical Bayes estimates of individual growth curves were developed, and smoothed estimates were plotted for selected demographic groups.
Investigating the Effectiveness of a Synthetic Linking Function on Small Sample Equating. Research Report. ETS RR-07-37
Author: Kim, Sooyeon
Date: 2007-08-00
Type: Journal Articles
Subjects: Equated Scores,Sample Size,Statistical Analysis,Licensing Examinations (Professions),Difficulty Level

The synthetic function, which is a weighted average of the identity (the trivial linking function for forms that are known to be completely parallel) and a traditional equating method, has been proposed as an alternative for performing linking with very small samples (Kim, von Davier, &amp; Haberman, 2006). The purpose of the present study was to investigate the benefits of the synthetic function using various real data sets gathered from different administrations of tests from a licensure testing program. We investigated the chained linear, Tucker, Levine, and mean equating methods, along with the identity and the synthetic functions with small samples (N = 19 to 70). Neither the identity nor the synthetic functions worked as well as did other linear equating methods, because test forms differed markedly in difficulty. The synthetic function cannot be used as a solution or methodological fix to a problem that is caused by poor data collection design. [Paper presented at the Annual Meeting of the National Council on Measurement in Education (NCME) (April 9-13, 2007, Chicago, IL).]
Construct Validity of &quot;e-rater&quot;® in Scoring TOEFL® Essays. Research Report. ETS RR-07-21
Author: Attali, Yigal
Date: 2007-05-00
Type: Journal Articles
Subjects: Construct Validity,Computer Assisted Testing,Scoring,English (Second Language),Language Tests,Second Language Learning,Essay Tests,Correlation,Test Reliability,Bias,Factor Analysis,Prediction,Comparative Analysis,Regression (Statistics),Models,Weighted Scores,True Scores,Comparative Education,Prompting

This study examined the construct validity of the &quot;e-rater&quot;® automated essay scoring engine as an alternative to human scoring in the context of TOEFL® essay writing. Analyses were based on a sample of students who repeated the TOEFL within a short time period. Two &quot;e-rater&quot; scores were investigated in this study, the first based on optimally predicting the human essay score and the second based on equal weights for the different features of &quot;e-rater.&quot; Within a multitrait-multimethod approach, the correlations and reliabilities of human and &quot;e-rater&quot; scores were analyzed together with TOEFL subscores (structured writing, reading, and listening) and with essay length. Possible biases between human and &quot;e-rater&quot; scores were examined with respect to differences in performance across countries of origin and differences in difficulty across prompts. Finally, a factor analysis was conducted on the &quot;e-rater&quot; features to investigate the interpretability of their internal structure and determine which of the two &quot;e-rater&quot; scores reflects this structure more closely. Results showed that the &quot;e-rater&quot; score based on optimally predicting the human score measures essentially the same construct as human-based essay scores with significantly higher reliability and consequently higher correlations with related language scores. The equal-weights &quot;e-rater&quot; score showed the same high reliability but significantly lower correlation with essay length. It is also aligned with the 3-factor hierarchical (word use, grammar, and discourse) structure that was discovered in the factor analysis. Both &quot;e-rater&quot; scores also successfully replicate human score differences between countries and prompts.
Small-Sample DIF Estimation Using Log-Linear Smoothing: A SIBTEST Application. Research Report. ETS RR-07-10
Author: Puhan, Gautam
Date: 2007-03-00
Type: Journal Articles
Subjects: Test Bias,Computation,Sample Size,Accuracy,Teacher Certification,Licensing Examinations (Professions),Whites,African Americans,Comparative Analysis

The purpose of the current study was to examine whether log-linear smoothing of observed score distributions in small samples results in more accurate differential item functioning (DIF) estimates under the simultaneous item bias test (SIBTEST) framework. Data from a teacher certification test were analyzed using White candidates in the reference group and African American candidates in the focal group. Smoothed and raw DIF estimates from 100 replications under seven different sample-size conditions were compared to a criterion to determine the effect of smoothing on small-sample DIF estimation. Root-mean-squared deviation and bias were used to evaluate the accuracy of DIF detection in the smoothed versus raw data conditions. Results indicate that, for most studied items, smoothing the raw score distributions reduced variability and bias of the DIF estimates especially in the small-sample-size conditions. Implications of these results for actual testing programs and future directions for research are discussed.
Impact of Group Work and Extended Essay Writing on Online Advanced Placement Program® Student Performance. Research Report. ETS RR-07-22
Author: Handwerk, Phil
Date: 2007-05-00
Type: Journal Articles
Subjects: Advanced Placement Programs,Advanced Placement,Computer Assisted Testing,Models,Aptitude Tests,High Schools,Regression (Statistics),Online Courses,Scores,Multiple Choice Tests,Writing Tests,Essays,Gender Differences,Race,High School Students,College Entrance Examinations

Online high schools are growing significantly in number, popularity, and function. However, little empirical data has been published about the effectiveness of these institutions. This research examined the frequency of group work and extended essay writing among online Advanced Placement Program® (AP®) students, and how these tasks may have impacted their AP exam performance. Hierarchical, set-wise regression models found little to no impact of group work or extended essay writing on multiple-choice, free response, or composite scores across 9 AP exams administered from 2003-2005. These models included student sex, race, and year in school as independent variables; covariates were the number of previous AP exams taken by the student and PSAT/NMSQT® scores. Implications for online providers and for future research of online high schools are discussed.
Monotone Properties of a General Diagnostic Model. Research Report. ETS RR-07-25
Author: Xu, Xueli
Date: 2007-06-00
Type: Journal Articles
Subjects: Models,Statistical Analysis,Educational Testing,Item Response Theory

Monotonicity properties of a general diagnostic model (GDM) are considered in this paper. Simple data summaries are identified to inform about the ordered categories of latent traits. The findings are very much in accordance with the statements made about the GPCM (Hemker, Sijtsma, Molenaar, &amp; Junker, 1996, 1997). On the one hand, by fitting a GDM with equal slopes across items, the observed total score X[subscript +] = [sigma][subscript j:qjk=1]xj demonstrates the monotone likelihood ratio (MLR) property in individual coordinate at the cost of losing model fit. On the other hand, fitting a GDM without slope restriction increases the model fit but by sacrificing MLR property. Trade-offs between these two situations should be considered in practice.
Confidence and Cognitive Test Performance. Research Report. ETS RR-07-03
Author: Stankov, Lazar
Date: 2007-03-00
Type: Journal Articles
Subjects: English (Second Language),Grade Point Average,High Schools,Personality Traits,Metacognition,Correlation,Accuracy,Scores,Test Bias,Self Esteem,Individual Differences,Gender Differences,Ethnic Groups,Racial Differences,Cognitive Tests,Validity,Reliability,College Students,Language Tests,Reading Skills,Listening Skills,Native Speakers,Questionnaires,Statistical Analysis,Second Language Learning,Factor Analysis

This paper examines the nature of confidence in relation to cognitive abilities, personality traits, and metacognition. Confidence was measured as it was expressed in answers to each test item during the administration of reading and listening sections of the TOEFL® iBT. The confidence scores were correlated with the accuracy scores from the TOEFL iBT, SAT®, high school grade point averages (HS-GPA), and several measures of personality and metacognition. The results indicate that confidence is a separate psychological trait, somewhere between cognitive ability and personality traits. In addition, our findings suggest that confidence is related to, but separate from, metacognition. We also demonstrated gender and ethnic differences in confidence, with male and African American students showing higher overconfidence bias than females and White or Hispanic students, respectively. Finally, our data show small but statistically significant incremental validity of the confidence scores above and beyond the accuracy scores in predicting numeracy test scores, the total TOEFL iBT scores, and TOEFL iBT subscores on the writing and speaking sections. We found no incremental validity of the confidence scores in predicting the SAT and HS-GPA. Theoretical and practical implications of these findings are discussed as well.
Symmetric Nonlinear Regression. Research Report. ETS RR-07-13
Author: Antal, Tamás
Date: 2007-04-00
Type: Journal Articles
Subjects: Computation,Regression (Statistics),Algebra,Mathematics Instruction,Tests,Problem Solving

An estimation tool for symmetric univariate nonlinear regression is presented. The method is based on introducing a nontrivial set of affine coordinates for diffeomorphisms of the real line. The main ingredient making the computations possible is the Connes-Moscovici Hopf algebra of these affine coordinates.
Estimation of Standard Error of Regression Effects in Latent Regression Models Using Binder's Linearization. Research Report. ETS RR-07-09
Author: Li, Deping
Date: 2007-03-00
Type: Journal Articles
Subjects: Error of Measurement,Regression (Statistics),Trend Analysis,National Competency Tests,Item Response Theory,Comparative Analysis,Models,Factor Analysis,Item Analysis,Race,Gender Differences,Ethnicity

Two versions of a general method for approximating standard error of regression effect estimates within an IRT-based latent regression model are compared. The general method is based on Binder's (1983) approach, accounting for complex samples and finite populations by Taylor series linearization. In contrast, the current National Assessment of Educational Progress procedure assumes a simple random sample for standard error of regression effects and applies a jackknife estimator to statistics of interest as a way to account for NAEP's complex sample. In this study, the versions of the general method are formally defined and the general method is extended to multiple dimensions. Furthermore, they are applied in an empirical study to the 2004 NAEP long-term trend data comparing both large, nearly saturated, and small models. Subsequently, the results are compared to the operational-based imputation method. Results show no impact on the imputation-based results, limited impact on large models, and reasonable impact on small models. While it is not readily apparent to what this differential impact can be attributed, several explanations are discussed.
Hierarchical General Diagnostic Models. Research Report. ETS RR-07-19
Author: von Davier, Matthias
Date: 2007-05-00
Type: Journal Articles
Subjects: Multivariate Analysis,Models,Item Response Theory,Probability,Equations (Mathematics),Measurement Techniques,Correlation,Hierarchical Linear Modeling

This paper introduces multilevel extensions for the general diagnostic model (GDM) following recent developments on extensions of latent class analysis (LCA) to hierarchical models. The GDM is based on LCA as well as discrete latent trait models and may be viewed as a general modeling framework for conrmatory multidimensional item response models. The multilevel extensions presented in this paper enable one to check the impact of clustered data, such as data for students within schools in large scale educational surveys, on the structural parameter estimates of the GDM. Moreover, the multilevel version of the GDM allows study of dierences in skill distributions across these clusters.
On the Latent Regression Model of Item Response Theory. Research Report. ETS RR-07-12
Author: Antal, Tamás
Date: 2007-03-00
Type: Journal Articles
Subjects: Regression (Statistics),Item Response Theory,National Competency Tests,Evaluation Methods,Mathematical Formulas,Models,Maximum Likelihood Statistics,Computation,Error of Measurement

Full account of the latent regression model for the National Assessment of Educational Progress is given. The treatment includes derivation of the EM algorithm, Newton-Raphson method, and the asymptotic standard errors. The paper also features the use of the adaptive Gauss-Hermite numerical integration method as a basic tool to evaluate expectations necessary to perform the parameter estimations.
Mixture Distribution Diagnostic Models. Research Report. ETS RR-07-32
Author: von Davier, Matthias
Date: 2007-07-00
Type: Journal Articles
Subjects: Item Response Theory,Models,Classification,Statistical Analysis,Skills,Test Bias

This paper introduces the mixture general diagnostic model (MGDM), an extension of the general diagnostic model (GDM). The MGDM extension allows one to estimate diagnostic models for multiple known populations as well as discrete unknown, or not directly observed mixtures of populations. The GDM is based on developments that integrate located latent class models; multiple classification latent class models; and discrete, multidimensional item response models into a common framework. Models of this type express the probability of a response vector as a function of parameters that describe the individual item response variables in terms of required skills and of indirectly observed (latent) skill profiles of respondents. The skills required for solving the items are, as in most diagnostic models, represented as a design matrix that is often referred to as a Q-matrix. This Q-matrix consists of rows describing, for each item response, what combination of skills is needed to succeed or to obtain partial or full credit. The hypothesized Q-matrix is either the result of experts rating items of an existing assessment (retrofitting) or comes directly out of the design of the assessment instrument, in which it served as a tool to design the items. The MGDM takes the GDM and integrates it into the framework of discrete mixture distribution models for item response data (see von Davier &amp; Rost, 2006). This increases the utility of the GDM by allowing the estimation and testing of models for multiple populations. The MGDM allows for complex scale linkages that make assessments comparable across populations and makes it possible to test whether items function the same in different subpopulations. This can be done with known subpopulations (defined by grade levels, cohorts, etc.), as well as with unknown subpopulations that need to be identified by the model. In both cases, MGDMs make it possible to determine whether different sets of item-by-skill parameters and/or different skill distributions have to be assumed for different subpopulations. This amounts to a generalized procedure that can be used to test for differential item functioning (DIF) on one item or on multiple-response variables using multiple-group or mixture models. This procedure enables testing DIF models against models that allow additional skills for certain items in order to account for differences between subpopulations.
Notions of Literacy in the K-12 School System in British Columbia Education since 2002: A Contested Terrain
Author: Park, Jeff
Date: 2016-08-17
Type: Journal Articles
Subjects: Elementary Secondary Education,Educational Policy,Literacy,Foreign Countries,Accountability,School Districts,Semi Structured Interviews,Public Schools,Social Environment,Reading Skills,Writing Skills

Developing literacy competencies has become a central component of educational policy in British Columbia (BC), with policies calling for province-wide assessment and school accountability. Based on the critical policy analysis (Blaikie &amp; Soussan, 2000) of provincial and school district documents, complemented by semi-structured interviews of senior government officials, district-based administrators, and literacy coordinators, this article discusses how the synergistic effect of the policy discourses of accountability and assessment framed the ways of thinking, conceptually and practically, about literacy mainly in terms of its instrumental value in holding public schools accountable to performance indicators. The article concludes by discussing the necessity of redefining literacy to challenge the view of literacy as a mere set of technical and human skills for economic growth to shift the dialogue toward policy alternatives that view literacy as set of capabilities for sociocultural and political change.