bluuebunny/crossref_metadata_2025 huggingface dataset:
CHILDREN'S ATTENTION TO STIMULUS COMPONENTS WITH VARIATION IN RELATIVE SALIENCE OF COMPONENTS AND DEGREE OF STIMULUS INTEGRATION1
Authors: Hale Gordon A. , Green Roberta Z.
Published: August 2014

A total of 400 children at ages 5, 9 and 12 years were given a component selection task with stimuli varying in both color and shape. Component salience was manipulated (across subjects) in two steps, in an effort to redirect attention from the normally dominant shape component toward color: the standard shapes and colors from earlier studies were compared with standard shapes in fluorescent colors, and the latter materials in turn were compared with nonsense shapes in fluorescent colors. For each salience condition, the color was either contained within the shape (integrated stimuli) or served as a background for the shape (nonintegrated stimuli). While fluorescence of color had little effect, the replacement of standard geometric forms with nonsense figures caused a shift in attention toward color and away from shape. This attentional shift became more marked between ages 5 and 9 years; with age, children apparently withdraw attention from the normally dominant feature to an increasing degree, while they adopt another component as the primary cue for stimulus identification. Integration of components did not influence the pattern of salience effects but generally increased attention to color. Comparison of the component selection task with a dimension‐preference matching test showed that only the former measure was sensitive to the effects of component salience on attention.
FLEXIBILITY OF CHILDREN'S ATTENTION: INSTRUCTIONS REGARDING DEPLOYMENT OF ATTENTION IN A COMPONENT SELECTION TASK1
Authors: Hale Gordon A. , Taweel Suzanne S. , Green Roberta Z. , Flaugher Jan
Published: August 2014

The effects of instructions on children's component selection were examined in three experiments. In Experiment 1, 5‐ and 8‐year‐old children were given: (a) instructions to attend to the dominant stimulus component (shape), (b) instructions to attend to the nondominant or secondary feature (color), or (c) no instructions regarding the components of the stimuli. Both age groups showed an ability to vary their attention to the secondary feature in accord with the instructions. However, the 8‐year‐olds showed greater flexibility in that they employed a type of attentional trading–withdrawing attention from the dominant shape component in exchange for increased attention to color. Experiment 2 replicated conditions (b) and (c) above with 5‐, 9‐ and 12‐year‐olds and also included a condition requiring attention to both stimulus components. Again, only children beyond age 5 tended to withdraw attention from the dominant shape component as they increased attention to color. Instructions to attend to both features proved relatively unsuccessful with 5‐ and 9‐year‐olds and were accommodated with only moderate success by 12‐year‐olds. In Experiment 3 the latter instructions, applied to stimuli with spatially separated components, were accommodated quite successfully by 9‐year‐olds, although not by 6‐year‐olds. Active attention to two components is apparently within the capabilities of 9‐year‐old children when the components are spatially separated and can readily be viewed as independent entities. However, purposeful attention to two integrated features is believed to require additional analytic abilities which do not develop until a later age.
INFANTS' SOCIAL RESPONSES TO SOCIAL EVENTS
Authors: Brooks Jeanne , Lewis Michael
Published: August 2014

Infants respond differentially, and at times with fear‐like behavior, to unfamiliar persons. This study was designed to see how infants discriminate among strangers. Since it has been shown that infants respond differently to children and adults, the physical characteristics of persons used to make such differentiations were of interest. Facial configuration and height were systematically varied as five different strangers–a male and a female child, a male and a female adult, and a small adult female the same height as the children (midget)–each approached 40 different infants. The infants responded as if there were three classes of persons–adults, children, and small adults. Infants as young as seven months of age reacted to the size‐facial configuration discrepancy of the small adult condition.
SEX DIFFERENCES IN CHILD SPEECH: McCARTHY REVISITED
Authors: Cherry Louise J.
Published: August 2014

This paper critically reviews empirical studies of sex differences in preschool children's spontaneous speech at the levels of phonology, syntax/grammar, semantics, and conversational usage. A discussion of McCarthy's 1930 study includes consideration of methodological issues of data collection and analysis as well as the results of the study. It is suggested that further research include longitudinal study of the stability of early sex‐typed vocalization/verbalization patterns in infants and study of the relationship of sex role learning to sex‐differentiated communicative behaviors.
PSYCHOLOGICAL DIFFERENTIATION IN CROSS‐CULTURAL PERSPECTIVE
Authors: Witkin H. A. , Berry J. W.
Published: August 2014

Cross‐cultural studies of psychological differentiation are reviewed with the objectives of: examining the applicability, across cultures, of the main propositions of differentiation theory and the generality of its intracultural supporting data; identifying extensions of the theory, and of its empirical base, suggested by the cross‐cultural findings; and delineating problems in the existing data and useful lines of further cross‐cultural inquiry. The evidence on self‐consistency, age changes and stability indicates that these aspects of differentiation show patterns in other cultures essentially similar to those observed in the original American studies. Numerous cross‐cultural studies have sought the sources of individual and group differences in differentiation in family practices, in cultural influences and in ecological pressures. The evidence from these studies suggests that less differentiated functioning, including a more field‐dependent perceptual mode, is associated with insistence upon adherence to adult authority, female salience and the absence of strong male role models in the family; “tight” organization and stress upon conformity in society; and sedentary agricultural and pastoral ecological settings. In contrast, more differentiated functioning, including relative field independence, are associated with encouragement of autonomy in the family, “loose” social structure and mobile hunting ecological settings. The small sex differences in field‐dependence‐independence, beginning in adolescence, repeatedly observed in Western studies are not universally evident in the non‐Western data. Sex differences appear to be common in samples at the sedentary agricultural end of the ecological spectrum and less evident in mobile hunting samples. Sex differences also seem to be more prevalent in “tight” than in “loose” social settings.
CHILDREN'S INCIDENTAL LEARNING WITH VARIATION IN AMOUNT OF STIMULUS EXPOSURE AND METHOD OF ASSESSMENT1
Authors: Hale Gordon A. , Alderman Linda B.
Published: August 2014

Children aged 9 and 12 years performed a task requiring attention to the “relevant” component of two‐component stimuli, after which they were given a test measuring either of two types of incidental learning: (a) recognition of the other, “incidental” component of each stimulus, or (b) association between the incidental and relevant components. In each of two experiments the principal variables were: (a) exposure time per trial of the initial, or central task (6 vs. 12 seconds), and (b) number of trials of the central task (8 vs. 16). Doubling the exposure time per trial improved central task performance to a greater degree at age 12 than age 9 but increased incidental learning only at age 9. Adolescents apparently redirect their attention from incidental to relevant stimulus information during the period of stimulus inspection on each trial. Doubling the number of trials, however, had little overall effect at either age level, indicating no developmental difference in the way children alter their manner of attention deployment across trials. Recognition and association measures of incidental learning did not produce markedly different patterns of results. A 4‐minute filled interval prior to the recognition test in Experiment 1 had little effect on the recognition scores.
THE ECONOMIC BENEFITS OF SCHOOLING AND READING COMPETENCE
Authors: Young Kan‐Hua , Jamison Dean T.
Published: August 2014

The purpose of this paper is to ascertain what economic benefits would accrue to compensatory education programs that were effective in raising the schooling and literacy levels of individuals after they left school. To do this we use a generalized earnings function format to examine the effects of schooling and literacy on individual earnings. The literacy measure was obtained for a 1972 probability sample of the U.S. population, and was administered with a background questionnaire.

Empirical results were first obtained from simple earnings functions. These results were compared with results obtained from a more elaborate conceptual framework incorporating a labor market model and an educational achievement model. The results of the models based on this more refined framework were then used to evaluate the economic benefits of schooling and compensatory reading education in terms of both annual earnings and discounted lifetime earnings. Methodologically, in both labor market and educational achievement analyses, both recursive and simultaneous models have been examined. However, in both situations, problems of identification were encountered in implementing the simultaneous equation approach, and our final results are based on recursive models that appear to provide reasonable results.
A FREE‐RESPONSE DESCRIPTION APPROACH TO THE ANALYSIS OF PERSON‐SITUATION INTERACTION12
Authors: Pervin Lawrence A.
Published: August 2014

A report on the utilization of a free‐response description approach to address the following question: In what ways does an individual remain stable and in what ways vary as a function of which situational characteristics? Each of four subjects generated a list of situations in his current life which he then rated on lists of situation traits, feelings, and behaviors which also were generated by the individual subject. For each subject the data were factor analyzed to determine groups of situations which were distinctive in terms of the feelings and behaviors associated with them. It was suggested that the personality of each individual could be understood in terms of the pattern of stability and change, in feelings and behaviors, in relation to defined groups of situations. It was also suggested that the approach utilized might serve as the basis for the future development of a taxonomy of situations and behaviors in situations.
THE INFLUENCE OF DIFFERENT DIRECTIONS ON STUDENT RATINGS OF INSTRUCTION
Authors: Centra John A.
Published: August 2014

Do students evaluate instruction differently depending on their understanding of the intended use of the results? Rating theory and results from related research led to the prediction that ratings would be more lenient when students were informed that the results would be used for tenure, salary, or promotion purposes, than when told that the results would be used only by the instructor for improvement. Data were collected in 16 classes in spring 1974, and from 25 classes the following fall semester. Every other student in each class received alternate directions. In general, the differences due to the different directions were not especially large or consistent.
GROUP COMPARISONS FOR THE TEST OF STANDARD WRITTEN ENGLISH
Authors: Breland Hunter M.
Published: August 2014

Group comparisons for male, female, majority, and minority students were conducted for the Test of Standard Written English (TSWE). Data for two academic years and from 18 different institutions were analyzed by pooling data across institutions within each of the academic years. Analyses of data from the first academic year focused on relationships between TSWE scores and grades. Analyses of the second academic year focused on relationships between TSWE scores and scores on graded writing samples. No important group differences in traditional correlational analyses for either grade or essay prediction were observed. No important group differences were observed in either correlational analyses or regression analyses of the second year essay data. Analyses of correct and incorrect placement decisions (hits and misses) at specific TSWE cut‐off scores revealed no noteworthy group differences whether outcomes were based on English course grades or on freshman year writing performance. Whether grades or essays were used as the outcome, the proportion of incorrect decisions (misses) was less for minority students than for any of the groups. For all groups, the TSWE appeared to predict freshman year writing performance as well as or better than pre‐course writing samples, high school English grades, or high school rank in class.
PSYCHOLOGICAL DIFFERENTIATION: CURRENT STATUS
Authors: Witkin Herman A. , Goodenough Donald R. , Oltman Philip K.
Published: August 2014

The status of the differentiation hypothesis, derived from differentiation theory, is examined in light of the evidence that has accumulated since the hypothesis was proposed a decade and a half ago. Assuming that greater or less differentiation is an organismic attribute, the hypothesis postulates that behaviors reflecting extent of differentiation are likely to be interrelated, making for self‐consistency in individual functioning across domains. The newer evidence has on the whole confirmed the linkages among the behaviors examined in the earlier research that was done to test the differentiation hypothesis. The differentiation hypothesis has also proven useful in generating predictions about linkages to behaviors in new domains (cognitive restructuring, interpersonal competencies and cerebral lateralization); these predictions have been tested and generally confirmed. The differentiation construct appears able to account for phenomena that cannot be accommodated by other lower‐order constructs, such as field dependence‐independence. On all these grounds the differentiation construct continues to serve a useful function. Whereas the differentiation hypothesis has dealt only with the interrelatedness among components of a cluster of behaviors subsumed under differentiation, the newer evidence carries suggestions for a hierarchical ordering of these components and for the nature of causal connections among them.
AN EXPLORATION OF SPEAKING PROFICIENCY MEASURES IN THE TOEFL CONTEXT
Authors: Clark John L. D. , Swinton Spencer S.
Published: August 2014

This report describes a three‐year study involving the initial development and experimental administration of a variety of test formats and item types aimed at measuring the English speaking proficiency of TOEFL candidates or other non‐native speakers of English on a direct basis, i.e., through the evaluation of actual speech production on the part of the examinee.

Although practical constraints of the TOEFL program ruled out of consideration, for operational purposes, face‐to‐face testing procedures of the Foreign Service Institute (FSI) type, a variety of “semi‐direct” testing formats (involving test stimuli delivered by tape recording and accompanying booklet, with examinee speech tape recorded for evaluation) were developed as part of the study. These were administered to undergraduate and graduate level non‐native English speaking students on a small‐scale clinical basis and, following revisions, to a fairly large (N = 155) examinee group. An FSI‐type interview was also administered to all study participants as an external criterion measure of speaking proficiency.

Correlational and scalar analyses were used to identify subsets of item formats and individual items having satisfactory correlations with the FSI criterion interview and also meeting relevant linguistic and administrative requirements. These were grouped into a prototype “Test of Spoken English” (TSE) that was found to increase predictability of total FSI variance by approximately 8% when added to score data provided by the regular (paper‐and‐pencil) TOEFL. Further modifications in TSE content and follow‐up administration with larger and more heterogeneous candidate groups are expected to increase this value.

Two additional related studies are currently under way, involving pilot test administration at regular TOEFL sites and use‐validation of the TSE as a measure of the “classroom communication” effectiveness of non‐native English speaking teaching assistants in U.S. academic settings.
ITEM PERFORMANCE ACROSS NATIVE LANGUAGE GROUPS ON THE TEST OF ENGLISH AS A FOREIGN LANGUAGE
Authors: Alderman Donald L. , Holland Paul W.
Published: August 2014

The Test of English as a Foreign Language (TOEFL) was examined for instances in which the item performance of examinees with comparable scores differed according to their native languages. A chi‐square procedure, sensitive to deviations of less than ten percent from the expected frequencies of correct item responses across several language groups, revealed significant differences on seven‐eighths of the TOEFL items. Reviewers familiar with particular languages could attribute the relative advantage or disadvantage of those language groups on a specific item to linguistic similarities or dissimilarities with the English language. Reviewers could not, however, identify which items would exhibit differential performance across groups based upon inspection of a test form and answer key alone. These findings suggest that examinees' performance on given items in a test of proficiency in a second language will vary according to linguistic contrasts with their native language and that statistical procedures will be necessary for identifying items with exaggerated or unexplained differences across language groups.
COGNITIVE STYLES IN EDUCATIONAL PRACTICE*
Authors: Messick Samuel
Published: August 2014

This paper examines characteristic features of cognitive styles and the distinctive ways in which they differ from one another and from intellective abilities. Educational implications of cognitive styles are discussed in terms of six main rubrics: Improving instructional methods, Enriching teacher behavior and conceptions, Enhancing student learning and thinking strategies, Expanding guidance and vocational decision making, Broadening educational goals and outcomes, and Tuning the stylistic demands of educational environments. Reasons why cognitive styles should have educational impact are addressed as well as reasons why such educational benefits are difficult to realize.
REDESCENDING M‐TYPE ESTIMATORS OF LATENT ABILITY
Authors: Jones Douglas H.
Published: August 2014

New ability estimators have been proposed by Wainer and Wright (1980) and Mislevy and Bock (1981), that are resistant against guessing and careless behaviors exhibited by some examinees. This paper presents another class of ability estimators that are resistant to departures from the underlying assumptions concerning guessing and carelessness. In addition to computing the asymptotic relative efficiency of such estimators, this paper evaluates estimators by comparing their influence curves (Huber, 1981).
SMALL SAMPLE PROPERTIES OF ASYMPTOTIC FORMULAS FOR EXPECTED SAMPLE SIZE SAVINGS IN CURTAILED BINOMIAL TESTS
Authors: Herrmann Nira , Szatrowski Ted H.
Published: August 2014

Asymptotic formulas for expected sample size savings in curtailed binomial tests are derived using a central limit theorem of Feller (1943). Exact results for the expected sample size savings and the forms of the one‐ and two‐sided curtailed tests are reviewed. The small sample properties of the asymptotic approximation are discussed and comparisons with the exact values and the recent results of Eisenberg and Ghosh (1981) are presented. The asymptotic results derived here compare very favorably to the exact results.
DEVELOPING A COMMON METRIC IN ITEM RESPONSE THEORY
Authors: Stocking Martha L. , Lord Frederic M.
Published: August 2014

A common problem arises when independent estimates of item parameters from two separate data sets must be expressed in the same metric. This problem is frequently confronted in studies of horizontal and vertical equating and in studies of item bias. This paper discusses a number of methods for transforming one metric to another metric and presents a new method. Data are given comparing this new method with a current method and recommendations are made.
INTEGRATING COGNITIVE AND PSYCHOMETRIC MODELS TO MEASURE DOCUMENT LITERACY
Authors: Sheehan Kathleen , Mislevy Robert J.
Published: August 2014

The Survey of Young Adult Literacy conducted in 1985 by the National Assessment of Educational Progress included sixty‐three items that elicited skills in acquiring and using information from written documents. These items were analyzed in two distinct ways: (1) with an item response theory (IRT) model, which characterized items' difficulties and respondents' proficiencies as revealed simply by tendencies toward correct response, and (2) a qualitative cognitive model, which characterized items in terms of the processing tasks they required. This paper demonstrates how a generalization of Fischer and Scheiblechner's Linear Logistic Test Model can be used to integrate information from the cognitive analysis into the IRT analysis.

Subject Terms: Bayesian estimation; cognitive processing models; Item Response Theory; Linear Logistic Test Model; literacy assessment;National Assessment of Educational Progress
COMPUTERIZED MASTERY TESTING WITH NONEQUIVALENT TESTLETS*
Authors: Sheehan Kathleen , Lewis Charles
Published: August 2014

A practical procedure for determining the effect of testlet nonequivalence on the operating characteristics of a testlet‐based computerized mastery test (CMT) is introduced. The procedure involves estimating the CMT decision rule twice, once with testlets treated as equivalent and once with testlets treated as nonequivalent. In the equivalent testlet mode, the likelihood functions estimated for specific number right scores are assumed to be constant across testlets and a single set of cutscores is estimated for all testlets. In the nonequivalent testlet mode, the likelihood functions estimated for specific number‐right scores are allowed to vary from one testlet to another and a different set of cutscores is estimated for each permutation of testlet presentation order. Small differences between the estimated operating characteristics of the equivalent testlet decision rule and the nonequivalent testlet decision rule indicate that the assumption of equivalent testlets was warranted. This procedure is illustrated with data from the Architect Registration Examination, a professional certification examination administered by the National Council of Architectural Registration Boards (NCARB).
COMPARISON OF EXPECTED WITH ACTUAL FIELD OF GRADUATE STUDY: AN ANALYSIS OF GRE SURVEY DATA
Authors: Grandy Jerilee
Published: August 2014

This study analyzed data from a survey conducted by Nettles in the fall of 1987. The sample consisted of 2,136 examinees who took the GRE during 1986/87. The purpose of this project was to determine how well the intended field‐of‐study item in the GRE background questionnaire can be relied upon as an indicator of what examinees will actually study in graduate school at the beginning of the next academic year.

Major findings were as follows:

Fifty‐six percent of all examinees in the sample became enrolled in graduate or professional school in the fall of 1987. The percentages varied somewhat across intended fields of study. Of those planning to study education, for example, 71% became enrolled in graduate school; of those planning to study computer science, only 49% became enrolled.

Enrollment rates did not differ significantly between Black, Hispanic, and predominantly White examinees, nor did it differ between male and female examinees.

Subsequent enrollment status was correlated very slightly with college grades but not with test scores.

Of those examinees who became enrolled in graduate or professional school and who specified a definite intended field of study, 72% were enrolled in exactly the same field of study they had indicated on the background questionnaire. Another 10% were enrolled in a specialty field within the same department. In total, 82% could be said to have enrolled in the same general field of study.

Stepwise regressions suggested that examinees with low GRE scores, particularly quantitative scores, were somewhat more likely to change field than examinees with high scores. With other variables held constant, females were slightly more likely to change field than were males. The strengths of the predictions, however, were extremely small, with multiple correlations of only 0.2.

The study concluded that test scores, grades, and known demographic variables in the GRE files are not strongly enough associated with changes in field of study to be useful as predictors of enrollment or of change in field of study.

The study concluded that the intended‐field‐of‐study item in the background questionnaire is a useful and reasonably valid indicator of actual field of graduate study.
AN ADAPTIVE ALGEBRA TEST: A TESTLET‐BASED, HIERARCHICALLY‐STRUCTURED TEST WITH VALIDITY‐BASED SCORING§
Authors: Wainer Howard , Lewis Charles , Kaplan Bruce , Braswell James
Published: August 2014

Earlier (Wainer & Lewis, 1990) we reported the initial development of a testlet‐based algebra test. In this account we provide the details of this excursion into the use of hierarchical testlets and validity‐based scoring. A pretest of two 15 item hierarchical testlets was carried out in which examinees' performance on a four item subset of each testlet was used to predict performance on the entire testlet. Four models for constructing hierarchies were considered. These presentation hierarchies were compared with one another and with an optimally chosen set of four linearly administered items. The comparison was carried out using both the root mean square error and the conditional posterior variance as the criterion. It was found on cross validation that although an adaptive test is everywhere superior to a fixed format test, this superiority is crucially dependent upon the quality of the items. When items vary considerably in quality a fixed format test, which uses the best items, can do almost as well as an adaptive test of equal length.
USING BAYESIAN DECISION THEORY TO DESIGN A COMPUTERIZED MASTERY TEST*
Authors: Lewis Charles , Sheehan Kathleen
Published: August 2014

Mastery testing is used in educational and certification contexts to decide, on the basis of test performance, whether or not an individual has attained a specified level of knowledge, or mastery, of a given subject. A theoretical framework for mastery testing, based on Item Response Theory and Bayesian decision theory, is described in this paper. In this framework, the idea of sequential testing is developed, with the goal of providing shorter tests for individuals who have clearly mastered (or clearly not mastered) a given subject, and providing longer tests for those individuals for whom the mastery decision is not as clear‐cut. In a simulated application of the approach to a professional certification examination, it is shown that average test lengths can be reduced by half without sacrificing classification accuracy.
PERFORMANCE MODELING THAT INTEGRATES LATENT TRAIT AND CLASS THEORY1
Authors: Gitomer ew H. , Yamamoto Kentaro
Published: August 2014

The qualitative characterization of individual performance that is central to modern psychological theory is not adequately modeled by traditional psychometric theory that assumes, among other things, unidimensionality. In the present study, data are presented that are more adequately modeled by HYBRID, a model that incorporates both latent trait and latent class components. The latent classes were defined by a cognitive analysis of the understanding that individuals have for a circumscribed domain. In addition to providing a better statistical fit, the analysis also improves the amount of diagnostic information available for a given individual.
THE CONVERGENT VALIDITY OF EXPERT SYSTEM SCORES FOR COMPLEX CONSTRUCTED‐RESPONSE QUANTITATIVE ITEMS
Authors: Bennett Randy Elliot , Sebrechts Marc M. , Rock Donald A.
Published: August 2014

This study investigated the convergent validity of expert‐system scores for four mathematical constructed‐response item formats. A five‐factor model was posed comprised of four constructed‐response format factors and a GRE General Test quantitative factor. Confirmatory factor analysis was used to test the fit of this model and to compare it with several alternatives. The five‐factor model fit well, although a solution comprised of two highly correlated dimensions—GRE‐quantitative and constructed‐response—represented the data almost as well. These results extend the meaning of the expert system's constructed‐response scores by relating them to a well‐established quantitative measure and by indicating that they signify the same underlying proficiency across item formats.
ON THE RELIABILITY OF TESTLET‐BASED TESTS
Authors: Sireci Stephen G. , Thissen David , Wainer Howard
Published: August 2014

If a test is constructed of testlets one must take into account the within‐testlet structure in the calculation of test statistics. Failing to do so may yield serious biases in the estimation of such statistics as reliability. In this paper we demonstrate how to calculate the reliability of a testlet‐based test. We show that traditional reliabilities calculated on two reading comprehension tests constructed of four testlets are substantial overestimates.
PAIRWISE COMPARISON PROCEDURES FOR ONE‐WAY ANALYSIS OF VARIANCE DESIGNS
Authors: Zwick Rebecca
Published: August 2014

Research in the behavioral and health sciences frequently involves the application of one‐factor analysis of variance models. The goal may be to compare several independent groups of subjects on a quantitative dependent variable or to compare measurements made on a single group of subjects on different occasions or under different conditions. In analyzing data of this kind, it is usually of interest to determine which pairs of population means are likely to differ. In this paper, the selection of pairwise multiple comparison procedures for one‐way analysis of variance designs is considered, following a discussion of Type I error and power issues as they apply to the testing of multiple hypotheses. Procedures are included which are appropriate when normality or variance homogeneity assumptions are violated. The focus is on procedures that are easy to understand and apply. Single‐step procedures are emphasized because of their simplicity and because they allow for the construction of confidence intervals.
PERFORMANCE AND PERSISTENCE: A VALIDITY STUDY OF THE SAT® FOR STUDENTS WITH DISABILITIES
Authors: Ragosta Marjorie , Braun Henry , Kaplan Bruce
Published: August 2014

This study was designed to test the validity of the Scholastic Aptitude Test (SAT) in predicting overall performance and persistence in college of students with disabilities, especially those participating in special test administrations. An earlier validity study (Braun, Ragosta, and Kaplan 1986) used first‐year grade‐point averages in college to study the validity issue. The current study returned to the schools that had originally provided data and obtained information on overall grade‐point averages and graduation status.

The study answered a number of questions about the usefulness of the SAT as a predictor of performance and persistence and concluded that the SAT does an adequate job. Overall grade‐point averages of disabled and nondisabled students were rather accurately predicted, although there was some slight overprediction for students with learning disabilities whether they took the test during regular or special test administrations. Persistence in college was also quite well predicted except for hearing‐impaired students from special test administrations.
THE CULTURAL CONTEXT OF CONSTRUCT VALIDITY AND THE ETHICS OF GENERALIZABILITY*
Authors: Laosa Luis M.
Published: August 2014

This article considers the mutual relevance of four themes: cultural diversity, construct validity, population generalizability, and professional ethics. Questions regarding construct validity are considered in the context of both measurement and intervention. The implications of the existence and the absence of empirical evidence on which to base validity generalizations are considered in reference to decision making in the provision of services. Ethical dilemmas that arise from these considerations, which face the applied professional in a culturally diverse society, are discussed within an emerging framework for resolving them. Previous research is reviewed to illustrate validity generalization limits. Also included are an assessment of research needs and a discussion of future directions for research and evaluation.
FITTING NEW MEASUREMENT MODELS TO GRE GENERAL TEST CONSTRUCTED‐RESPONSE ITEM DATA
Authors: Bennett Randy Elliot , Sebrechts Marc M. , Yamamoto Kentaro
Published: August 2014

This exploratory study applied two new cognitively sensitive measurement models to constructed‐response quantitative data. The models, intended to produce qualitative characterizations of examinee performance, were fitted to algebra word problem solutions produced by examinees taking the GRE General Test. Two types of response data were modeled—error diagnoses and partial‐credit scores—both produced by an expert system. Error diagnoses, analyzed using Yamamoto's (1989a) Hybrid model, detected a class of examinees who tended to miss important pieces of the problem solution but made relatively few errors of other types. Group members were of low quantitative proficiency overall, though considerable variability was evident. Comparisons with matched examinees whose response patterns were better captured by the unidimensional IRT model suggested subtle differences in error frequency rather than sharp qualitative distinctions. In contrast with the error data, partial‐credit scores modeled using Rock's (Rock & Pollack, 1987) HOST procedure did not fit well, in part owing to limitations of the task theory being tested. Implications for the development of refined task and error theories, improvements to expert‐system scoring procedures, and response modeling are discussed.
SMALL‐SAMPLE EQUATING WITH LOG‐LINEAR SMOOTHING
Authors: Livingston Samuel A.
Published: August 2014

This study investigated the extent to which log‐linear smoothing could improve the accuracy of common‐item equating by the chained equipercentile method in small samples of examinees. Examinee response data from a 100‐item test were used to create two overlapping forms of 58 items each, with 24 items in common. The criterion equating was a direct equipercentile equating of the two forms in the full population of 93,283 examinees. Anchor equatings were performed in samples of 25, 50, 100, and 200 examinees, with fifty pairs of samples at each size level. Four equatings were performed with each pair of samples: one based on unsmoothed distributions and three based on varying degrees of smoothing. Smoothing reduced, by at least half, the sample size required for a given degree of accuracy. Smoothing that preserved only two moments of the marginal distributions resulted in equatings that failed to capture the curvilinearity in the population equating.
UNDERSTANDING GRAPHS AND TABLES
Authors: Wainer Howard
Published: August 2014

Quantitative phenomena can be displayed effectively in a variety of ways, but to do so requires an understanding of both the structure of the phenomena and the limitations of candidate display formats. This paper:

• Recounts three historic instances of the vital role data displays played in important discoveries,

• Provides three levels of information which form the basis of a theory of display to help us better measure both display quality and human graphicacy, and

• Describes three steps to improve the quality of tabular presentation.
COMPARABILITY OF COMPUTER AND PAPER‐AND‐PENCIL SCORES FOR TWO CLEP® GENERAL EXAMINATIONS
Authors: Mazzeo John , Druesne Barry , Raffeld Paul C. , Checketts Keith T. , Muhlstein Alan
Published: August 2014

This report describes two studies that investigated the comparability of scores from paper‐and‐pencil and computer‐administered versions of the College‐Level Examination Program (CLEP) General Examinations in Mathematics and English Composition. The first study used a prototype computer‐administered version of each examination. Based on the results of the first study and feedback from the study participants, several modifications were made to these prototype versions. A second study was then conducted using the modified computer versions. Both studies used a single‐group counterbalanced equating design. Data for the Mathematics Examination were collected at Southwest Texas State University, and data for the English Composition Examination were collected at Utah State University. The results of Study 1 suggest that, despite efforts to design computer versions of the CLEP Mathematics and English Composition General Examinations that were administratively similar to the paper‐and‐pencil examinations (i.e., allowed item review and answer changing and were comparably timed), mode‐of‐administration effects (i.e., changes in average scores as a function of the mode of test delivery) were found. The results of Study 2 suggest that the modifications made to the computer versions eliminated the mode‐of‐administration effects for the English Composition Examination but not for the Mathematics Examination. The results of both studies underscore the need to determine empirically (rather than to just assume) the equivalence of computer and paper versions of an examination.
THE ROLE OF THE MENTOR TEACHER: A TWO‐PHASE STUDY OF TEACHER MENTORING PROGRAMS
Authors: Wilder Gita Z.
Published: August 2014

Increasing attention to mentoring in the teaching profession in recent years has been accompanied by a proliferation of programs, literature and expertise related to experienced teachers helping new teachers. To assess the extent to which there is a common set of functions performed by teacher mentors, a two‐phase study of mentoring was undertaken. In the first phase, a telephone survey of the 50 states and of more than 100 districts was conducted to gather data about the existence and nature of mentoring programs. The survey revealed that about 60 percent of the states have enacted legislation to create formal programs of mentoring, although many of the programs had not yet been implemented at the time of the survey. In addition, mentoring programs existed in districts without state programs. In the second phase of the study, visits were made to eight mentoring programs, selected to represent a range of sponsorship and program structures. Interviews were conducted with program, district and school administrators, and with mentors and their teacher‐proteges. The results suggest that, although mentoring programs differ with respect to the ways in which the work of mentors is structured, there are functions that are common to the role of mentor across programs. The common functions and the implications of these for selecting and training mentors are discussed.
SIGNIFICANT IMPROVEMENTS TO LOGIST*
Authors: Wingersky Marilyn S.
Published: August 2014

Stocking (1989) discovered a problem with the LOGIST estimation procedure. The automatic procedure in this program produced different item parameter estimates when the true item discriminations were used as starting values for the iteration procedure than when the default starting value of one for the item discriminations was used. When a straight run to convergence was performed, the different initial starting values converged to the same item parameter estimates.

This study investigated several methods for improving the automatic procedure. When these methods failed to give the improvement necessary, several methods were tried to obtain the same results as the run to convergence in a shorter amount of computer time. A method was devised that takes much less time and gives nearly as good, and in some cases better, estimates as estimates obtained from running to convergence. This method involves adding a step to get better initial item parameter estimates for the automatic procedure. In this step, the abilities are grouped very coarsely and the item parameters and the grouped abilities are estimated iteratively, alternating back and forth between items and abilities until the maximum difference between the estimated item characteristic curves is less than some criterion.

The new procedure gives item parameter estimates that are closer to the true values than the current 4‐step method does. However, there is a definite nonlinear relationship between the estimated item parameters for the two methods after the parameters have been linearly transformed to the same scale. Consequently, in an ongoing series of calibrations, switching to this procedure from the old procedure will produce a discontinuity in the parameter estimates in the same manner as would be caused by switching from LOGIST to BILOG.

The effects of putting a beta prior on c were also investigated. The results were not conclusive. An option to put a beta prior on c was added to the program.

This new method has been incorporated into a new version of LOGIST called LOGIST7.
COMPARISON OF EFFICIENCY OF JACKKNIFE AND VARIANCE COMPONENT ESTIMATORS OF STANDARD ERRORS
Authors: Longford Nicholas T.
Published: August 2014

Large scale surveys usually employ a complex sampling design, and as a consequence no standard methods for estimation of the standard errors associated with the estimates of population means are available. Resampling methods, such as jackknife or bootstrap are often used, with reference to their properties of robustness and reduction of bias. We examine a method based on variance component models as an alternative to the jackknife procedure used for calculation of the standard errors for the subpopulation means of proficiency scores in a large scale survey of education in the U.S.A.
PLACEMENT VALIDITY OF A PROTOTYPE SAT WITH AN ESSAY
Authors: Bridgeman Brent , Hale Gordon A. , Lewis Charles , Pollack Judith , Wang Ming‐mei
Published: August 2014

Grades in college freshman English composition courses were predicted from high school rank in class, multiple‐choice writing scores, essays, current SAT‐Verbal scores, and scores from a revised version of SAT‐Verbal. Data were obtained from 21 English courses at 17 different colleges with some supplementary data provided by an additional college. In general, a writing composite score consisting of essay and multiple‐choice writing scores appeared to outperform current or revised SAT‐Verbal scores; validity coefficients were as high or higher for the writing composite score, and the underprediction of the grades of women students was reduced. The best predictions were obtained from the combination of high school rank with the writing composite score.
AN EXPLORATORY STUDY OF CHARACTERISTICS RELATED TO IRT PARAMETER INVARIANCE WITH THE TEST OF ENGLISH AS A FOREIGN LANGUAGE
Authors: Way Walter D. , Carey Patricia A. , Golub‐Smith Marna L.
Published: August 2014

IRT equating methods have been used successfully with the TOEFL® test for many years, and for the most part the observed properties of items have been consistent with model predictions. However, items that do not appear to hold their IRT pretest estimates do exist. If relationships can be found between features of TOEFL items in pretest calibrations and subsequent lack of model‐data fit when these items are used in final forms, steps to eliminate the use of such items in TOEFL final forms can be taken. The purpose of this study was to provide an exploratory investigation of item features that may contribute to a lack of invariance of TOEFL item parameters.

The results of the study indicated the following: (1) subjective and quantitative measures developed for the study provided consistent information related to the model‐data fit of TOEFL test items, (2) for Sections 1 and 2, items that were pretested before 1986 exhibited poorer model‐data fit than items that were pretested after 1986, and (3) for Section 3 reading comprehension, model‐data fit appeared to be related to changes in the relative position of items within the sections from the pretest to the final form administrations.

Based on the results of the study, it was recommended that (1) the TOEFL program investigate the feasibility of not using pretest IRT statistics for items pretested before 1986 for Sections 1 and 2 and (2) that guidelines be developed for test developers to use with reading comprehension items to limit the change in relative positions of items in the test from pretest to final form administrations.
WILL THEY THINK LESS OF MY HANDWRITTEN ESSAY IF OTHERS WORD PROCESS THEIRS? EFFECTS ON ESSAY SCORES OF INTERMINGLING HANDWRITTEN AND WORD‐PROCESSED ESSAYS
Authors: Powers Donald E. , Fowles Mary E. , Farnum Marisa , Ramsey Paul
Published: August 2014

A study was undertaken to determine the effects on essay scores of intermingling handwritten and word‐processed versions of student essays. A sample of examinees, each of whom had produced both a handwritten and a word‐processed essay, was drawn from a larger sample of students who had participated in a pilot study of item types being considered for the new academic skills assessments of The Praxis Series: Professional Assessments for Beginning Teachers™.

Students' original handwritten essays were converted to word‐processed versions, and their original word‐processed essays were converted to handwritten versions. In a preliminary study, handwritten and word‐processed essays were then intermingled and rescored.

Analyses revealed higher average scores for essays scored in the handwritten mode than for essays scored as word‐processed, regardless of the mode in which essays were originally produced. Several hypotheses were advanced to explain the discrepancies between scores on handwritten and word‐processed essays. The training of essay readers was subsequently modified on the basis of these hypotheses, and the experiment was repeated using the modified training with a new set of readers.

The results of this second study showed an average reduction of about 25% in the discrepancy between essays read in the handwritten and word‐processed modes, compared with the results of the initial study. The effects computed in the second experiment were small by most standards, and predicted to have very little if any impact on certification decisions. Nonetheless, a recommendation is made not only to adopt the modified training but also to monitor this effect throughout the operational scoring sessions.