bluuebunny/crossref_metadata_2025_split huggingface dataset:
AN ALGORITHM FOR CONVERTING CONTOURS TO ELEVATION GRIDS
Authors: Reid‐Green Keith S. 
Published: August 2014

ABSTRACTSome of the test questions for the National Council of Architectural Registration Boards (NCARB) deal with the site, including drainage, regrading, and the like. A site is described by its shape and by contours shown on the plan view. Existing structures may be shown on the site. Some questions are most easily scored by examining contours, such as determining if, in regrading the site, the student changed a contour, causing it to pass through an existing structure. However, water flow questions are best scored from a grid of elevations. This document describes a method for creating an elevation grid from a set of contour lines.
EVIDENCE AND INFERENCE IN EDUCATIONAL ASSESSMENT1,2
Authors: Mislevy Robert J. 
Published: August 2014

ABSTRACTEducational assessment concerns inference about students' knowledge, skills, and accomplishments. Because data are never so comprehensive and unequivocal as to ensure certitude, test theory evolved in part to address questions of weight, coverage, and import of data. The resulting concepts and techniques can be viewed as applications of more general principles for inference in the presence of uncertainty. Issues of evidence and inference in educational assessment are discussed from this perspective.
DEPICTING ERROR
Authors: Wainer Howard 
Published: August 2014

ABSTRACTCommunicating data without some measure of their precision can lead to misinterpretation and incorrect inferences. Several conventions for displaying error along with the data they modify are described and illustrated. Alternatives are offered that seem to provide improvements in the effective communication of error as well as increasing their ease, and hence their likelihood, of use. These alternatives are illustrated principally with data from the National Assessment of Educational Progress.
PERFORMANCE BY GENDER ON AN UNCONVENTIONAL VERBAL REASONING TASK: ANSWERING READING COMPREHENSION QUESTIONS WITHOUT THE PASSAGES
Authors: POWERS DONALD E. 
Published: August 2014

ABSTRACTData were reanalyzed from a previously reported study of the passage dependence of reading comprehension questions being developed for the revised SAT. The objective was to uncover any gender differences in approaches to and performance on a task requiring examinees to answer reading comprehension questions without reading the passages on which the questions were based. Verbally able males and females were compared with respect to their reported use of a variety of test‐taking strategies involving reasoning, personal knowledge, and guessing. A few relatively small (and often inconsistent) differences were detected between male and female test takers. However, far more similarities than differences were noted with respect to both test performance and test‐taking behavior. If these results are generalizable to more typical tests of verbal ability, they would seem to suggest that males and females employ quite similar approaches to standardized test taking.
ESTIMATING RELIABILITY FOR TESTS COMPOSED OF ITEM SETS
Authors: Lawrence Ida M. 
Published: August 2014

ABSTRACTTo what extent, if any, are estimates of reliability for a multiple choice test affected by the presence of large item sets where each set shares common reading material? The purpose of this research was to assess the effect of local item dependence on estimates of reliability for verbal portions of the old and new SAT, where the new test contains larger item sets associated with reading passages. Estimates based on a single administration of the test (estimates based on internal consistency and estimates based on covariances among parts) were compared to estimates based on two administrations of the test. When adjusted for a fixed standard deviation, estimates based on covariances among parts tended to be similar to estimates based on parallel forms. Both types of estimates were lower than the internal consistency estimates.
THE INTRODUCTION AND COMPARABILITY OF THE COMPUTER ADAPTIVE GRE GENERAL TEST
Authors: Schaeffer Gary A. , Steffen Manfred , Golub‐Smith Marna L. , Mills Craig N. , Durso Robin 
Published: August 2014

ABSTRACTThis report summarizes the results from two studies. The first study assessed the comparability of scores derived from linear computer‐based (CBT) and computer adaptive (CAT) versions of the three GRE General Test measures. The verbal and quantitative CATs were found to produce scores that were comparable to their CBT counterparts. However, the analytical CAT produced scores that were judged not to be comparable to the analytical CBT scores. As a result, a second study was performed to further examine the analytical measure to ascertain the extent of the lack of comparability and to obtain statistics that would permit adjustments to restore comparability.Results of the additional study of the analytical measure indicated that the differences in analytical CAT and CBT scores due to the testing paradigm were large enough to require an adjustment in scores. Therefore, in order to enhance the comparability of analytical CAT and CBT scores, the analytical CAT was equated to the analytical CBT. This equating provided new analytical CAT conversions that resulted in comparable analytical CAT and CBT scores.
DIF DATA ON FREE‐RESPONSE SAT I MATHEMATICAL ITEMS1
Authors: Lawrence Ida M. , Lyu C. Felicia , Feigenbaum Miriam D. 
Published: August 2014

ABSTRACTThe present research summarized Differential Item Functioning (DIF) results for Student Produced Response (SPR) items, a non‐multiple choice mathematical item type in the SAT I. DIF data from four field trial pretest administrations (620 SPR items) and ten final forms (100 SPR items) were examined. For the Male/Female comparison, the SPR item type exhibits negligible DIF. The Male/Female DIF results for the SPR items resemble DIF results for the multiple choice items in the SAT. For the African American/White comparison, SPR items exhibit more DIF than the multiple choice items. (Moderate or sizable DIF on multiple choice items in the old or new SAT‐M is rare.) In particular, when African American test takers are compared to matched groups of White test takers, they tend to perform less well than White test takers on the SPR items. Although the level of African American/White DIF on SPR items is elevated, relative to the multiple choice items, none of the SPR items in any of the final forms were flagged for exhibiting extreme DIF. The relationship between DIF and discrimination is discussed as a possible explanation for the elevated level of DIF on the SPR items.
EFFECTS OF MODE OF ITEM PRESENTATION ON STANDARD SETTING
Authors: Faggen Jane , Melican Gerald , Powers Don 
Published: August 2014

ABSTRACTThe objective of this study was to determine the degree to which recommendations for passing scores, calculated on the basis of a traditional standard‐setting methodology, might be affected by the mode (paper vs. computer screen prints) in which test items were presented to standard setting panelists. Results were based on the judgments of 31 panelists for items in each of three parts of a computer‐based academic skills assessment designed for use as part of the teacher licensing process. None of the evidence collected–either the analysis of panelists' estimates of item difficulty or their perceptions of effects–revealed any differences in recommended passing standards resulting from the mode of item presentation.
INTELLIGENCE TESTING AND SOCIAL POLICY
Authors: Laosa Luis M. 
Published: August 2014

ABSTRACTThere is a resurgence of scientific and public interest and controversy centering on four interrelated themes: intelligence testing; racial, ethnic, and socioeconomic differences in measured IQ; genetic and environmental influences on abilities; and the role of scientific research in social policy. Given the polarization of views, dispassionate discussion is needed to help inform the ongoing debate. The aim of this paper is to contribute to this discussion by helping to clarify certain issues, assumptions, and concerns that, because they are often misunderstood or ignored, tend to obfuscate the debate.
THE PERSONAL STATEMENT AS AN INDICATOR OF WRITING SKILL: A CAUTIONARY NOTE
Authors: Powers Donald E. , Fowles Mary E. 
Published: August 2014

ABSTRACTThe objective of this study was to evaluate the personal statement as an indicator of writing skill. The evaluation was based on a comparison of specially‐evaluated personal statements with a standardized measure of writing ability requiring examinees to write timed expository essays. A sample of prospective graduate students wrote essays and provided copies of personal statements they had submitted for application to graduate school. A majority of the sample acknowledged receiving help in either drafting or revising their statements. Correlations of the two indicators (test essay and personal statement) with each of several non‐test indicators of writing skill revealed that the traditional expository essay was significantly more highly related to nearly all of the indicators. It is suggested that, while the personal statement may provide certain unique and important information about applicants (and thus be valid for this purpose), its validity as an indicator of writing skill is suspect.
CHOICE AMONG ESSAY TOPICS: IMPACT ON PERFORMANCE AND VALIDITY
Authors: Bridgeman Brent , Morgan Rick , Wang Ming‐mei 
Published: August 2014

ABSTRACTThis study assessed the ability of history students to choose the essay topic on which they can get the highest score. A second, equally important, question was whether the score on the chosen topic was more highly related to other indicators of proficiency in history than the score on the unchosen topic. Overall, for both US and European history, scores were about one‐third of a standard deviation higher for the preferred topic than for the other topic. For US history, about 32% of the students made the wrong choice, that is 32% got a higher score on the other topic than on the preferred topic. In European history, 29% made the wrong choice. In the US history sample, the preferred essay correlated .40 with an external criterion score vs .34 for the other essay; in the European history sample, the preferred essay correlated .52 with the external criterion compared to .44 for the other topic.
RESEARCH PERSPECTIVES ON CONSTRUCTS OF CHANGE: INTERCULTURAL MIGRATION AND DEVELOPMENTAL TRANSITIONS
Authors: Laosa Luis M. 
Published: August 2014

ABSTRACTIn every society there are certain important life events that are predictable and normative therein and that demand from the individual some major and lasting personal change; they are often–but not always–connected with a biological change of life (e.g., puberty, graduations, maturity, parenthood, retirement). In contrast to these transitions, there also exist other critical life events, which are less prevalent and less predictable, and which place extreme demands on the individual for personal change and adaptation. These extraordinary experiences include intercultural transitions, particularly those associated with international migration. In recent years, the need for research attention to this type of life transition in individual development has increased, largely as a result of significant changes in immigration trends and in the ethnocultural mix of the nation's population. A multifaceted, complex, and consequential event, intercultural migration poses special challenges as a subject of scientific inquiry. At the same time, it provides a unique window for viewing processes of change and adaptation. In this paper some of these needs, challenges, and opportunities are identified and discussed in the context of varied research perspectives and constructs of change.
MEASURING ACCOMPLISHMENTS: PSEUDOIPSATIVITY, QUANTITY VS. QUALITY, AND DIMENSIONALITY
Authors: Stricker Lawrence J. , Rock Donald A. 
Published: August 2014

ABSTRACTThis study examined three questions about measures of accomplishments–notable attainments that have been publicly recognized: their pseudoipsativity, the correspondence between quantity and quality scores, and their dimensionality. Comparable samples of graduate students described their accomplishments on a questionnaire or judged the similarity of the same accomplishments. Accomplishments in the same field were positively correlated, while accomplishments in different fields varied in their correlations, some being positive and others being negative; these results are inconsistent with the predominantly negative correlations that would occur with ipsative measures. Measures of the quantity and quality (importance, rarity) of accomplishments correlated highly, after correction for attenuation, and appeared to assess the same thing. And analyses of the self‐report data found that accomplishments were factorially complex, with many of the same factors identified in other kinds of measures appearing; the factors observed in the judgment data largely corresponded to the semantic features of the accomplishment items.
DIFFERENCES IN STRATEGIES USED TO SOLVE STEM‐EQUIVALENT CONSTRUCTED‐RESPONSE AND MULTIPLE‐CHOICE SAT®‐MATHEMATICS ITEMS
Authors: KATZ IRVIN R. , FRIEDMAN DEBRA E. , BENNETT RANDY ELLIOT , BERGER ALIZA E. 
Published: August 2014

ABSTRACTThis study investigated the strategies subjects adopted to solve stem‐equivalent SAT‐Mathematics (SAT‐M) word problems in constructed‐response (CR) and multiple‐choice (MC) formats. Parallel test forms of CR and MC items were administered to subjects representing a range of mathematical abilities. Format‐related differences in difficulty were more prominent at the item level than for the test as a whole. At the item level, analyses of subjects' problem‐solving processes appeared to explain difficulty differences as well as similarities.Differences in difficulty derived more from test‐development than from cognitive factors: On items in which large format effects were observed, the MG response options often did not include the erroneous answers initially generated by subjects. Thus, the MC options may have given unintended feedback when a subject's initial answer was not an option or allowed a subject to choose the correct answer based on an estimate.Similarities between formats occurred because subjects used similar methods to solve both CR and MC items. Surprisingly, when solving CR items, subjects often adopted strategies commonly associated with MC problem solving. For example, subjects appeared adept at estimating plausible answers to CR items and checking those answers against the demands of the item stem.Although there may be good reasons for using constructed‐response items in large‐scale testing programs, multiple‐choice questions' of the sort studied here should provide measurement that is generally comparable to stem‐equivalent constructed‐response items.
COMPARING THE EFFECTIVENESS OF PROCEDURES TO DETECT ITEM PARAMETER DRIFT
Authors: Donoghue John R. , Isham Steven P. 
Published: August 2014

ABSTRACTThis study used Monte Carlo methods to examine the use of several methods to identify item parameter drift. Number of examinees, number of items, and number of drift items in the test were manipulated. Overall, Lord's χ2 measure was the most effective in identifying items that exhibited drift. However, the measure was only accurate when the c‐parameter was constrained to be equal across calibrations. Of the remaining methods, the best (z‐test based on Raju's exact unsigned integral, the PARSCALE computer program's χ2 by subgroup, and Kim and Cohen's closed interval signed area measure) required empirical estimates of critical values for the test statistics in order to function well. This aspect detracts from their usefulness.
ON THE CONSEQUENCES OF SOME TEST RESCORING POLICIES
Authors: Wainer Howard , Bradlow Eric T. 
Published: August 2014

ABSTRACTWhen tests or portions of tests are scored subjectively by raters, a rescoring will yield a change in the ratings of some examinees. In a high stakes test with a fixed passing score a rescoring will result in the change of some pass/fail decisions. The number of changes depends on three things: (1) the reliability of the rating system, (2) the proportion of examinees that are expected to pass, and (3) the policy used to incorporate the rescore into the pass/fail decision. In this study, we provide a model that facilitates the evaluation of various rescoring strategies. We illustrate the use of this model for rescoring schemes that concatenate all ratings, old and new. The estimation of rescoring effects are computed by direct simulation. A further generalization of the basic model is also considered in which a test is comprised of a mixture of objectively and subjectively scored items. Last, we note that rescoring is merely a special case of the more general problem of the optimal allocation of judges and demonstrate an algorithm that accomplishes this.
MODELING THE RELATIONSHIP BETWEEN SCHOOL DISTRICT SPENDING AND ACADEMIC ACHIEVEMENT: A MULTIVARIATE ANALYSIS OF THE 1992 NATIONAL ASSESSMENT OF EDUCATIONAL PROGRESS AND THE COMMON CORE OF DATA
Authors: Wenglinsky Harold 
Published: August 2014

ABSTRACTIn the wake of the Coleman Report, which found student socio‐economic status (SES) to play a large role in student performance and the economic resources of the school to play a small or non‐existent one, sociologists of education have tended to assume that inequalities in the economic resources of school districts do not reinforce inequalities in achievement between high SES and low SES students. Yet there is in fact a lack of consensus among educational researchers as to whether or not economic resources are associated with student achievement. This study attempts to test the hypothesis that money does matter, by applying the statistical technique of linear structural modeling to a nationally representative sample of eighth graders from data drawn from the National Assessment of Educational Progress and the Common Core of Data. The analysis finds that the data support the hypothesis that per‐pupil expenditures on instructional and central office administration are positively related to small class size, which in turn is positively related to academic achievement.
A TREE‐BASED APPROACH TO PROFICIENCY SCALING
Authors: Sheehan Kathleen M. 
Published: August 2014

ABSTRACTThe process of translating examinees' observed test results into probabilistic statements about the patterns of skill mastery associated with performance at successive points on a test's reported score scale has been termed proficiency scaling. This paper introduces a new approach to proficiency scaling which was specifically developed to provide accurate, instructionally‐relevant score interpretations, even when the test under consideration is a non‐diagnostic, broad‐based ability test such as the SAT. The approach employs techniques derived from the realm of tree‐based regression to classify test items into nonoverlapping clusters such that all of the items in each cluster have similar values of item difficulty and require similar combinations of skills. A nonparametric smoothing procedure is then used to summarize cluster performance conditional on underlying test score. The resulting cluster characteristic curves provide the probability of responding correctly to items with specified skill requirements, expressed as a function of the underlying test score. This new approach is described in the context of a specific application: developing proficiency interpretations for the reading comprehension subsection of the SAT I Verbal Reasoning Test. It is shown that the generated proficiency interpretations account for 91% of the variation which would be accounted for by an optimal clustering solution. These results suggest that the approach is capable of extracting useful information about the specific combinations of skills needed to perform at increasingly higher levels on a test's reported score scale.
USING JUST NOTICEABLE DIFFERENCES TO INTERPRET TEST OF SPOKEN ENGLISH SCORES
Authors: Stricker Lawrence J. 
Published: August 2014

ABSTRACTThis study explored the value of obtaining a Just Noticeable Difference (JND) — the difference in scores needed before observers discern a difference in examinees' English proficiency – for the current Test of Spoken English (TSE®) as a means of interpreting scores in practical terms, using college students' ratings of their international teaching assistants' English proficiency and adapting classical psychophysical methods. The test's concurrent validity against these ratings was also appraised. Three estimates of the JND were obtained. They varied considerably in size, but all were substantial when compared with the standard deviation of the TSE scores, the test's standard error of measurement, and guidelines for the effect size for mean differences. The TSE test correlated moderately with the rating criterion. The JND estimates appear to be meaningful and useful in interpreting the practical significance of TSE scores, and the test has some concurrent validity.
LINKING SCORES FOR COMPUTER‐ADAPTIVE AND PAPER‐AND‐PENCIL ADMINISTRATIONS OF THE SAT
Authors: Lawrence Ida , Feigenbaum Miriam 
Published: August 2014

ABSTRACTIn October 1996, a computer‐adaptive version of the SAT was administered in an operational setting for the first time. This version of the test, the Computerized SAT, was administered to students who are seeking placement into various talent search programs. Because many students seeking placement into talent search programs will take the paper‐and‐pencil (P&P) SAT I, and scores between the traditional P&P test and the Computerized SAT (CAT) will be treated as interchangeable, it was necessary to conduct a field trial to insure that reported scores for the two modes of testing were on the same scale. In addition, because students seeking placement into talent search programs are typically in the 7th‐ or 8th‐grade and not part of the SAT I target population (i.e., high school juniors and seniors), it was necessary that the field trial use samples taken from the target population. The data collected in the field trial were used investigate the relationship between CAT and P&P scores in the target population for the SAT I. An anchor test design was used to scale the scores from the CAT to scores on the P&P measure. In addition to providing scaling data for the CAT, this study provided an opportunity to gather information about examinees' perceptions of the test‐taking experience in comparison to their experience with the paper‐based version of the SAT I.
POSSIBLE DETERMINANTS OF GENDER DIF ON THE ADVANCED PLACEMENT PSYCHOLOGY TEST: FAMILIARITY, INTEREST, AND EMOTIONAL REACTION
Authors: Stricker Lawrence J. , Emmerich Walter 
Published: August 2014

ABSTRACTThis study evaluated the connection between gender differences in Advanced Placement (AP) Psychology students' familiarity, interest, and negative emotional reaction to items on the AP Psychology test and the items' gender differential item functioning (DDF). Gender DDF and gender differences in interest and emotional reaction varied appreciably with the content of the items. Gender differences in the three variables were substantially related to the items' gender DDF (e.g., R = .50). Much of the gender DDF on this test may be attributable to gender differences in these variables.