bluuebunny/crossref_metadata_2025 huggingface dataset:
Using computers to Analyze Item Response Data
Authors: Hsu Tse‐chi , Yu Lifa
Published: October 2005

A survey of existing item analysis programs shows that the information provided by most of the programs is either inappropriate for guiding instruction or too difficult for practitioners to use. Studies comparing the performance of IRT item calibration programs are summarized. Future directions for the development of item analysis programs are discussed.
Screening for Potentially Biased Items in Testing Programs
Authors: Hills John R.
Published: October 2005

What statistical methods are being used to identify biased test items? What are the advantages and disadvantages of each method? What sample sizes are required to use each method? Which methods are most useful to the practitioner?
The State of NCME: Remembering the Past, Looking to the Future
Authors: Lehmann Irvin J.
Published: October 2005

What are the roots of NCME? What were our initial goals and how well have we attained them? Are we drifting away from our practitioner orientation? What should be our policy on important matters in educational practice and our role in sociopolitical issues in educational measurement?
Ranking School Districts on the Basis of Statewide Test Results: Is It Meaningful or Misleading?
Authors: Guskey Thomas R. , Kifer Edward W.
Published: October 2005

How do state educational authorities use statewide test data to rank schools? Can refinements in the procedures change the rankings of schools? Will interpretations based on such rankings be fair, accurate, and valid? Does ranking school districts reveal anything meaningful about the schools? Can ranking schools be counterproductive?
Adapting Adaptive Testing: Using the MicroCAT Testing System in a Local School District
Authors: Kingsbury G. Gage
Published: October 2005

What is the rationale for adapting an existing testing system instead of developing your own? What are the limitations of MicroCAT? What has to be modified in order to meet local needs and to realize the potential of adaptive testing in the context of an existing testing system?
Development, Implementation, and Validation of a Computerized Test for Statewide Assessment
Authors: Olsen James B. , Cox Apryl , Price Charles , Strozeski Mike , Vela Idolina
Published: October 2005

How does a testing component function in an integrated learning system? How can you customize computerized tests to meet local specifications? How are computerized tests implemented and evaluated? Is the pay‐off of computerized testing justified?
Practical Applications of Student Response Analysis
Authors: Switzer Deborah M. , Connell Michael L.
Published: October 2005

What standard reports are included in TAP and SPP? What information is provided in item analysis output? How may teachers use the information in their instruction?
The Lake Wobegon Effect
Authors: Phillips Gary W.
Published: October 2005

What is the Lake Wobegon Effect? How can we explain this phenomenon? Is the Lake Wobegon Effect an artifact of the methodology used in the 1987 study?
Inflated Test Score Gains: Is the Problem Old Norms or Teaching the Test?
Authors: Shepard Lorrie A.
Published: October 2005

What explanations have been provided for spurious test score gains? Are states and districts narrowing the curriculum and teaching the test? What effect does teaching the test have on the norms themselves? What alternatives must be sought to protect the integrity of instruction and the validity of scores?
The Profession's Evolving Standards
Authors: Faggen Jane
Published: October 2005

What steps has the measurement profession taken to bridge the gap between professionally and legally acceptable practice in licensure testing? What questions should be raised about the progress made by the profession in supporting valid inferences from certification test scores?
Legal Challenges to Licensing Examinations
Authors: Pyburn Keith M.
Published: October 2005

Do the courts view tests differently than testing professionals? What conclusions may be gleaned from court cases which challenge the legality of tests?
Content Validity Studies of Licensing Examinations
Authors: Smith I. Leon , Hambleton Ronald K.
Published: October 2005

Can measurement specialists’current ideas about content validation be implemented with licensure examinations? Does pressure of litigation facilitate or inhibit conducting validity studies?
Social Considerations in the Validation of Licensing and Certification Exams
Authors: Shimberg Benjamin
Published: October 2005

How do the concerns of critics become linked to test regulation legislation? Is a self‐regulatory agency consistent with social concerns about licensure testing?
Establishing Standards for Teacher Certification Tests
Authors: Jaeger Richard M.
Published: October 2005

Are teacher certification tests subject to the same concerns as other licensure examinations? How should standards for such examinations be established?
Court‐Defined Job Analysis Requirements for Validation of Teacher Certification Tests
Authors: Kuehn Phyllis A. , Stallings William M. , Holland C.L.
Published: October 2005

What are the implications for teacher certification tests as courts begin to interpret the need for using a job analysis as a basis for testing? What is the character of recent court cases related to teacher certification tests?
Essential Assessment Skills in Professional Education of Teachers
Authors: Schafer William D.
Published: October 2005

What content areas should be included in a testing and measurement course? What skills should teachers learn in each content area? How can this content and these skills be rationalized?
Relevant Classroom Assessment Training for Teachers
Authors: Stiggins Richard J.
Published: October 2005

What are the areas of mismatch between the assessment problems teachers face and the type of assessment training they receive? How can the gaps be addressed in testing courses?
Perspectives on Measurement Instruction
Authors: Airasian Peter W.
Published: October 2005

How do teachers obtain assessment information? Should our courses emphasize less formal assessment methods rather than formal methods? What nontraditional assessment topics should be added to our courses?
Measurement‐Related Course Work Requirements for Teacher Certification and Recertification
Authors: O'Sullivan Rita G. , Chalnick Marla K.
Published: October 2005

Do any state agencies require competence in assessment before certifying teachers? Do certification agencies permit measurement‐related course work or continuing education credits meeting recertification requirements?
Can Fantasies Become Facts?
Authors: Cohen S. Alan , Hyman Joan S.
Published: October 2005

In EM'S Spring 1989 issue, Mehrens and Kaminski (1989) questioned both the ethics as well as the validity of some “teach‐to‐the‐test” procedures, and particularly addressed Cohen's application of instructional alignment to teaching test wiseness. Cohen and Hyman now respond with their own questions about the educational value and the ethics of using the types of norm‐referenced tests defended by Mehrens and Kaminski.
Adventures in Implementing a Testing Program
Authors: Fisher Thomas H. , Smith Julia
Published: October 2005

What are the ways in which a testing program can run aground? What are some suggestions for avoiding these problems?
Understanding Reliability
Authors: Traub Ross E. , Rowley Glenn L.
Published: October 2005

The topic of test reliability is about the relative consistency of test scores and other educational and psychological measurements. In this module, the idea of consistency is illustrated with reference to two sets of test scores. A mathematical model is developed to explain both relative consistency and relative inconsistency of measurements. A means of indexing reliability is derived using the model. Practical methods of estimating reliability indices are considered, together with factors that influence the reliability index of a set of measurements and the interpretation that can be made of that index.
Selection of Judges for Standard‐Setting
Authors: Jaeger Richard M.
Published: October 2005

Who should make judgments about test standards? Who is an expert? How many judges should be used in a standard‐setting study? What is the relationship between the number of judges and the standard error of the test?
Defining Minimal Competence
Authors: Mills Craig N. , Melican Gerald J. , Ahluwalia Nancy Thomas
Published: October 2005

How do we explain the standard‐setting process to judges? What context should we set for the standard‐setting exercise? How can we help judges develop a common definition of minimal competence?
Factors Influencing Intrajudge Consistency During Standard‐Setting
Authors: Plake Barbara S. , Melican Gerald J. , Mills Craig N.
Published: October 2005

What factors influence judges when they set standards? How do judges, test questions, and the standard‐setting process interact? How can we improve intrajudge consistency?
Standard Error of Measurement
Authors: Harvill Leo M.
Published: October 2005

The standard error of measurement (SEM) is the standard deviation of errors of measurement that are associated with test scores from a particular group of examinees. When used to calculate confidence bands around obtained test scores, it can be helpful in expressing the unreliability of individual test scores in an understandable way. Score bands can also be used to interpret intraindividual and interindividual score differences. Interpreters should be wary of over‐interpretation when using approximations for correctly calculated score bands. It is recommended that SEMs at various score levels be used in calculating score bands rather than a single SEM value.
Do NAEP Scales Yield Valid Criterion‐Referenced Interpretations?
Authors: Forsyth Robert A.
Published: October 2005

What are some of the ways that the NAEP scales have been interpreted? Are such interpretations valid? What conditions are necessary before valid criterion‐referenced interpretations can be made for the NAEP Proficiency Scales? Does the NAEP test development process meet these conditions?
Effects of Item Order and Context on Estimation of NAEP Reading Proficiency
Authors: Zwick Rebecca
Published: October 2005

What can be learned about the effects of item order and context on invariance of item parameter estimates? Are common‐item equating methods appropriate when measuring trends in educational growth?
Investigating the Validity of a Locally Developed Critical Thinking Test
Authors: Moss Pamela A. , Koziol Stephen M.
Published: October 2005

What can be done at the local school level to study the validity of measures of critical thinking? What information other than test scores should be analyzed? What validity can a test have for diagnosis, monitoring achievement, or curriculum reform?
Educational Testing in Poland
Authors: Niemierko Boleslaw
Published: October 2005

What has been the history of educational assessment in Poland? How are the social system, school system, examination forms, measurement theory, and attitudes toward tests related to each other? How are recent efforts at democratization of political systems likely to influence educational testing in Poland?
IRT Equating Methods
Authors: Cook Linda L. , Eignor Daniel R.
Published: October 2005

The purpose of this instructional module is to provide the basis for understanding the process of score equating through the use of item response theory (IRT). A context is provided for addressing the merits of IRT equating methods. The mechanics of IRT equating and the need to place parameter estimates from separate calibration runs on the same scale are discussed. Some procedures for placing parameter estimates on a common scale are presented. In addition, IRT true‐score equating is discussed in some detail. A discussion of the practical advantages derived from IRT equating is offered at the end of the module.
State‐wide Student Assessment
Authors: Cooley William W.
Published: October 2005

What are the purposes of state‐wide student assessment? Why do state testing programs seldom inform state policy? Why is it essential for these programs to be based on state curricula? Who should be accountable for improving the learning outcomes of students?
Appropriateness of Teachers' Test‐Preparation Practices
Authors: Popham W. James
Published: October 2005

What two standards can teachers and administrators use to decide whether a particular way of preparing students for a test is appropriate? How do five commonly used test‐preparation practices stack up to these standards? How do educators and school board members view these five practices?
Performance Assessment: Moonstruck or the Real Thing?
Authors: Frechtling Joy A.
Published: October 2005

What are the advantages of performance assessment? What are the risks of employing only performance assessments in large‐scale testing programs?
NATD Survey of Testing Practices and Issues
Authors: Bauer Ernest A.
Published: October 2005

Why do school districts use Standardized tests? What kinds of testing programs are most common? How much do school testing programs cost? Should publishers coordinate their test standardization efforts?
Context‐Dependent Item Sets
Authors: Haladyna Thomas M.
Published: October 2005

What are item sets and how are they being used in testing? What methods are being used to score them? What are the prospects for future use of item sets?
An Independent Auditing Mechanism for Testing
Authors: Madaus George F.
Published: October 2005

Do we need an independent auditing mechanism for standardized testing? How effective are existing approaches for maintaining high standards of test development and use? What approaches to auditing testing seem promising?
Using Portfolios of Student Work in Instruction and Assessment
Authors: Arter Judith A. , Spandel Vicki
Published: October 2005

The tern portfolio has become a popular buzz word. Unfortunately, it is not always clear exactly what is meant or implied by the term, especially when used in the context of portfolio assessment. This training module is intended to clarify the notion of portfolio assessment and help users design such assessments in a thoughtful manner. We begin with a discussion of the rationale for assessment alternatives and then discuss portfolio definitions, characteristics, pitfalls, and design considerations.
Educational Testing in Hungary
Authors: Csapó Ben˝
Published: October 2005

What has been the role of educational testing in Hungarian education? How are the political changes in the country expected to affect testing in the future?
Uses and Abuses of Achievement Test Scores
Authors: Nolen Susan Bobbitt , Haladyna Thomas M. , Haas Nancy S.
Published: October 2005

Are variations in test‐preparation practices from school to school undermining the meaningfulness of achievement test results? Is there pressure to raise achievement test scores by the use of educationally unsound practices? What uses of achievement test scores are most common? Do teachers and administrators have reasonably accurate views of test score uses?
Educators' Perceptions of NRT Misuse
Authors: Hall Janie L. , Kleine Paul F.
Published: October 2005

What has been the impact of high stakes achievement testing on curriculum and testing practice? How serious is the problem of cheating on Standardized achievement tests? What steps can we take to improve the validity of test‐based information for a variety of educational purposes?
Computerized Adaptive Testing With Different Groups
Authors: Legg Sue M. , Buhr Dianne C.
Published: October 2005

How does the use of computerized adaptive testing affect the performance of students from different groups? How consistent were the results of computerized adaptive and “conventional” tests? What did the students think about the test experience? What advice do the authors have for test developers and users?
Ethical Codes in the Professions
Authors: Schmeiser Cynthia B.
Published: October 2005

Why do professions develop ethical codes? How similar are ethical codes across professions? How are codes enforced? Should the measurement profession develop such a code?
Portfolios, Accountability, and an Interpretive Approach to Validity
Authors: Moss Pamela A. , Beck Jamie Sue , Ebbs Catherine , Matson Barbara , Muchmore James , Steele Dorothy , Taylor Caroline , Herter Roberta
Published: October 2005

How can the results of classroom‐based portfolio assessment be communicated outside the classroom? How might a portfolio‐based assessment system be designed and implemented? How can we evaluate the merits of portfolio‐based assessments?
Educational Assessment in Czechoslovakia
Authors: Mareš Jiři , Byčkovský Petr
Published: October 2005

What is the history of educational assessment and research in Czechoslovakia? How have attitudes toward testing changed over the years? What are the prospects for the future?
True‐False, Alternate‐Choice, and Multiple‐Choice Items
Authors: Downing Steven M.
Published: October 2005

What does research tell us about true‐false tests? Is there a place for this format in standardized tests?
Developing a Personal Grading Plan
Authors: Frisbie David A. , Waltman Kristie K.
Published: October 2005

The purpose of this instructional module is to assist teachers in developing defensible grading practices that effectively and fairly communicate students' achievement status to their parents. In formulating such practices, it is essential that teachers first consider their personal grading philosophy and then create a compatible personal grading plan. The module delineates key philosophical issues that should be addressed and then outlines the procedural steps essential to establishing a grading plan. Finally, the features of several common methods of absolute and relative grading are compared.
What Assessment and Research Say About the Condition of Education
Authors: Bracey Gerald W.
Published: October 2005

Do students today know less than their parents or grandparents did as students? Are they being academically outclassed by students of other nations? Or have ideologically driven policy makers painted an unnecessarily negative picture of student achievement?
What Happened to Test Scores, and Why?
Authors: Koretz Daniel
Published: October 2005

What has happened in recent decades to the test scores of American students? What can be said about the causes of trends in scores? Do these trends provide an indictment of the effectiveness of American schools? How should test scores be used to inform the debate about education policy?
A School District Perspective on Appropriate Test‐Preparation Practices: A Reaction to Popham's Proposals
Authors: Kilian Lawrence J.
Published: October 2005

Recent discussions of the appropriateness of various test‐preparation activities have focused on the responsibilities of the teacher. Do those who make and mandate tests used in high‐stakes situations bear any responsibility for the validity of the scores? Is it possible to construct a comprehensive set of guidelines to protect the validity of test scores?
The Multiple True‐False Item Format: A Status Review
Authors: Frisbie David A.
Published: October 2005

How do multiple true‐false items differ from other item formats? What does past research indicate about the quality of multiple true‐false items? What additional research is needed?
Generalizability Theory
Authors: Brennan Robert L.
Published: October 2005

Generalizability theory consists of a conceptual framework and a methodology that enable an investigator to disentangle multiple sources of error in a measurement procedure. The roots of generalizability theory can be found in classical test theory and analysis of variance (ANOVA), but generalizability theory is not simply the conjunction of classical theory and ANOVA. In particular, the conceptual framework in generalizability theory is unique. This framework and the procedures of generalizability theory are introduced and illustrated in this instructional module using a hypothetical scenario involving writing proficiency.
Applications of Educational Measurement: I s Optimum Optimal?
Authors: Flake Barbara S.
Published: October 2005

What would you do if you were asked to design an “optimal” school testing program, but had doubts about the criteria to apply? What strategies might help us make more use of performance assessment and computer‐adaptive testing in district‐level testing programs? Can an NCME President be whimsical when delivering her presidential address?
Educational Testing in America: What's Right, What's Wrong? A Criterion‐Referenced Perspective
Authors: Popham W. James
Published: October 2005

What is a criterion‐referenced test? What are the good and bad points of authentic assessment, criterion‐referenced testing, and Item Response Theory?
Some Practical Considerations When Converting a Linearly Administered Test to an Adaptive Format
Authors: Wainer Howard
Published: October 2005

“We seek the truth and will endure the consequences.” Charles Seymour President of Yale (1937–1950)
Assessing the Utility of Item Response Models: Computerized Adaptive Testing
Authors: Kingsbury G. Gage , Houser Ronald L.
Published: October 2005

How has Item Response Theory helped solve problems in the development and use of computer‐adaptive tests? Do we need to balance item content with computer‐adaptive tests? Could we use IRT to evaluate unusual responses to computer‐delivered tests?
Type K and Other Complex Multiple‐Choice Items: An Analysis of Research and Item Properties
Authors: Albanese Mark A.
Published: October 2005

What is a complex multiple‐choice test item? What is the evidence that such items should be avoided?
Alternative Assessment in a HighmStakes Environment
Authors: Miller M. David , Legg Sue M.
Published: October 2005

How generalizable are the results of alternative assessments? How much psychometric evidence d o we have about these approaches? What effects do alternative assessments have on subgroup differences in performance? What are the probable consequences of using alternative assessments?
Coaching for the SAT: A Summary of the Summaries and an Update
Authors: Powers Donald E.
Published: October 2005

Why is it important to investigate the effectiveness of coaching for a test such as the SAT? How can you evaluate the effectiveness of coaching? What are some common misconceptions? What do we know and what is still unclear about the effects of coaching? What can we tell students and their families about the results of coaching studies ?
Book Review
Authors: Stone Clement A.
Published: October 2005

Book reviewed in this article:

Using National Data Bases in Educational Research Thomas L. Hilton (Ed.)
National Testing or Political Testing: I s There a Difference?
Authors: Rudman Herbert C.
Published: October 2005

Are two national reports on standardized testing realistic or simplistic? What is the difference between “political” testing and conventional testing? What are some likely consequences of politicizing school testing?
Cognitive‐Style Differences in Testing Situations
Authors: Armstrong Anne‐Marie
Published: October 2005

Why will multiple‐choice tests continue to serve the needs of education, business, and government? How d o differences in cognitive style affect performance on multiple‐choice tests of different types? How does clarity and care in writing multiple‐choice items affect performance?
Testing and Grading Practices and Opinions of Secondary Teachers of Academic Subjects: Implications for Instruction in Measurement
Authors: Frary Robert B. , Cross Lawrence H. , Weber Larry J.
Published: October 2005

What are secondary school teachers' views on testing and grading practices? What clusters of opinions can be categorized as ‘flexible,”“opposed to difficult testing,” or “hard‐nosed?” How should the use of test results for grading be treated in teacher in‐service training programs?
Survey of Test Security Practices
Authors: Mehrens William A. , Phillips S. E. , Schram Christine M.
Published: October 2005

How common are securitg breaches in state testing programs? What problems are most frequently encountered? Does the number of problems seem small or large, given the nature of the tests and how they are used?
Assessment Competencies of Teachers: A National Survey
Authors: Plake Barbara S. , Impara James C. , Fager Jennifer J.
Published: October 2005

What are the “Standards for Teacher Competence in Educational Assessment of Students”? In what domains do teachers have the greatest knowledge of measurement? Where are they least strong? Do teachers with measurement training show greater knowledge than teachers without such training?
The Effects of Standardized Testing on Teaching and Schools
Authors: Herman Joan L. , Golan Shari
Published: October 2005

How much influence does standardized testing exert on teaching and learning in upper elementary classrooms? Is there any difference between the impact of testing on low‐SES and high‐SES classrooms? How much impact does “teaching to the test” appear to have on test scores?
Book Review
Authors: Qualls Audrey L.
Published: October 2005

Book reviewed in this article:

Testing, Reform, and Rebellion H. D. Corbett and B. L. Wilson
The Vemont Portfolio Assessment Program: Findings and Implications
Authors: Koretz Daniel , Stecher Brian , Klein Stephen , McCaffrey Daniel
Published: October 2005

How effective has the Vermont program been in meeting its goals? What goals appear to be in conflict in programsof this type? Why is there a need for “caution and moderate expectations”?
The Americans With Disabilitie Act Implications for Measurement
Authors: Fischer Richard J.
Published: October 2005

What does the Americans with Disabilities Act require of assessment programs? What does “reasonable accommodation” mean? What is the best overall guidance that is available regarding ways to meet the spirit and letter of the act?
Job Analysis for Lkensure and Certfication Exams: Science or Politics?
Authors: Nelson David S.
Published: October 2005

How might empirical j o b analysis give the appearance of science to a flawed public policy? How do “political” factors have an impact on the development of licensure tests? What are areas for future research and for improvement of prevailing practices in licensure and certification testingy?
Criterion‐Referenced Tests: Part I. origins
Authors: Glaser Robert
Published: October 2005

Where were you in 1963, when this article was first published? What are the two kinds of information you can get from achievement tests? What is a‘triterion level?”
The Instructional Consecpences of Criterion Referenced Clarity
Authors: Popham W. James
Published: October 2005

What has been the impact of CRTs on instruction? What are some dangers involved in specifying instructional targets? Why should we focus on score‐based inferences? How should we evaluate the quality of our CRTs?
Criterion Referencd Testing 30 Yeam Later: Promise Broken, Promise Kept
Authors: Millman Jason
Published: October 2005

What promises has CRT kept or broken? Why do we need denser tests to make good, criterion‐referenced interpretations? What wisdom has grown from experience with CRTs?
The Rise and Fall of Criterion Referenced Measurement?
Authors: Hambleton Ronald K.
Published: October 2005

How frequently have articles about CRT appeared? What advances in measurement have been influenced by the CRT movement? What has been the lasting contribution of the CRT movement?
Performance‐Based Assessment: Implications of Task Specificity
Authors: Linn Robert L. , Burton Elizabeth
Published: October 2005

How generalizable are the scores from performance‐based assessments? How can we best increase the gener‐alizability of results? What level of generalizability is needed for different testing purposes?
Recontextualizing Mental Measurement
Authors: Goldstein Harvey
Published: October 2005

Why should we be concerned about “physics envy” as a feature of our approach to educational measurement? What are some limitations of current measurement models? Why should there be greater public participation in decisions regarding testing?
The Publication Process in Educational Measurement
Authors: Jaeger Richard M. , Hendricks Astrid Y.
Published: October 2005

Why is it important to publish your research? How are papers reviewed before publication decisions are made? How can you select the right journal for your paper? How do EM:” and other measurement journals compare in content, orientation, and other characteristics?
An NCME Instructional Module: Guidelines for the Development of Item Banks
Authors: Ward Annie W. , Murray‐Ward Mildred
Published: October 2005

Use of item banking technology can provide much relief for the chores associated with preparing assessments; it may also enhance the quality of the items and improue the quality of the assessments. Item banking programs provide for item entry and storage, item retrieval and test creation, and maintenance of the item history. Some programs also provide companion programs for scoring, analysis, and reporting. There are many item banking programs that may be purchased or leased, and there are banks of items auailable for purchase. This module is designed to help those who develop assessments of any kind to understand the process of item banking, to analyze their needs, and to find or develop programs and materials that meet those needs. It should be useful to teachers at all leuels of education and to school‐district test directors who are responsible for developing district‐wide tests. It may also provide some useful information for those who are responsible for large‐scale assessment programs of all types.
Psychometric and Social Issues in Admissions to Israeli Universities
Authors: Beller Michal
Published: October 2005

How are students selected for university in Israel? What does the Psychometric Entrance Test measure? How effective is the process of translating tests? What studies have been conducted on the tests?
The Educational Testing Movement in the Netherlands: Some Historical and Political Aspects
Authors: Thio Djien K.
Published: October 2005

What factors influenced the growth of educational testing in the Netherlands? How are tests used in primary and secondary education? What has been the role of the National Institute for Educational Measurement (CITO)?
Perspectives on Educational Testing: Discussion
Authors: Jones Lyle V.
Published: October 2005

What are the similarities and differences among the testing practices and tests that have been described in the preceding articles? How do the situations in these countries compare to what we are experiencing in the US.?
Educational Testing Abroad and Lessons for the United States
Authors: Feuer Michael J. , Fulton Kathleen
Published: October 2005

What are key features of schooling and testing in some major industrialized countries? What lessons can we learn u from other countries? How important is the issue of confidence in the teachers who will be called on to judge student performance?
Educational and Employment Testing: Changing Concepts in Measurement and Policy
Authors: Camara Wayne J. , Brown Dianne C.
Published: October 2005

How will the expansion of the concept of construct validity affect validation practice in employment testing? How does the need for consequential validity differ in educational and employment testing? How do the research bases differ for performance assessment in these settings? Are there parallel trends in policies for test use in education and industry?
Measuring Socioeconomic Status at the School Level
Authors: McMillan James H. , Henry Gary T. , Crosby Diane , Dickey Kent C.
Published: October 2005

Why is there an increased need for quantitative indicators of student socioeconomic status at the school level? What alternative approaches have been used to measure SES? Can principals accurately estimate the SES distribution of their student populations? How do their estimates correlate with other indexes o f family SES?
Report on Divedty Within the Leadership of the National Council on Measurement in Education
Authors: Murray‐Ward Mildred
Published: October 2005

What activities constitute leadership in NCME? Do those in leadership roles represent the general membership in terms of gender, ethnicity, and employment affiliation? Is there sufficient diversity in the organization and in the profession at largey
Is the Curriculum a Reasonable Basis for Assessment Reform?
Authors: Nitko Anthony J.
Published: October 2005

How should curriculum‐assessment linkages be forged? Do local districts have sufficient resources and expertise to sustain these ongoing assessments? Can assessments developed at state level accurately reflect local curricula? What are the essential characteristics of a curriculum‐based assessment system?
Portfolios in Large‐scale Assessment: Difficult But Not Impossible
Authors: LeMahieu Paul G. , Gitomer Drew H. , Eresh Jo Anne T.
Published: October 2005

Can districtwide portfolio assessment yield reliable scores? How can content and scoring be standardized? How can such assessment be integrated with instruction? What are the costs?
A Theory‐Based Framework for Assessing Domainl‐Specific Problem‐Solving Ability
Authors: Sugrue Brenda
Published: October 2005

Is problem solving a global construct? What components can be assessed? What multiple assessment strategies can be used?
Have Changes in the SAT Affected Women's Mathematics performance?
Authors: Burton Nancy
Published: October 2005

How has the SAT changed in content and format? Why was it changed? How have changes to the mathematics test affected men's and women's performance?
Grades or Scores: Predicting Future College Mathematics Peflormance
Authors: Kessel Cathy , Linn Marcia C.
Published: October 2005

Do men and women with similar admissions test scores earn similar grades in mathematics? What factors may explain underprediction for women? Do men and women differ in persistence in mathematics study?
Standard‐Setting Guidelines
Authors: Cizek Gregory J.
Published: October 2005

Do we need guidelines for standard‐setting studies? What should the guidelines address? What additional research on standard‐setting is needed?
How Is Reliability Related to the Quality of Test Scores? What Is the Effect of Local Dependence on Reliability?
Authors: Wainer Howard , Thissen David
Published: October 2005

What are the practical implications of small decreases in reliability coefficients? How does increased item local dependence decrease reliability? How does the new format of more “authentic” reading tests affect reliability?
Legal Defensibility of Standards: Issues and Policy Perspectives
Authors: Phillips S. E.
Published: October 2005

What legal issues may arise in setting content or performance standards? What precedents are relevant? What practices are defensible?
Professional Development in Student Assessment for Educational Administrators: An Instructional Framework
Authors: Impara James C. , Plake Barbara S.
Published: October 2005

What assessment tasks do administrators perform? What knowledge and skills do they need for these tasks? How is this information useful for professional development?
An NCME Instructional Module on: Setting Passing Scores
Authors: Cizek Gregory J.
Published: October 2005

This module describes standard setting for achievement measures used i n education, licensure, and certification. On completing the module, readers will be able to: describe what standard setting is, why it is necessary, what some of the purposes of standard setting are, and what professional guidelines apply to the design and conduct of a standard‐setting procedure; differentiate among different models of standard setting; calculate a cutting score using various methods; identify appropriate sources of validity evidence and threats to the validity of a standard‐setting procedure; and list some elements to be considered when evaluating the success of a standard‐setting procedure. A self‐test and annotated bibliography are provided at the end of the module. Teaching aids to accompany the module are available through NCME.
Effects of Introducing Classroom Peflormance Assessments on Student Learning
Authors: Shepard Lorrie A. , Flexer Roberta J. , Hiebert Elfrieda H. , Marion Scott F. , Mayfield Vicky , Weston Timothy J.
Published: October 2005

How can performance assessments be used as part of regular instruction? Will this raise student performance on external achievement measures? What aspects of examinee performance improve on the assessment exercises?
Beyond Computation and Correctness: Contributions of Open‐Ended Tasks in Examining U. S. and Chinese Students’Mathematical Performance
Authors: Cai Jinfa
Published: October 2005

How do Chinese and American students compare on open‐ended mathematics tasks? Do they employ similar solution strategies? What challenges in test development emerge in such studies?
Problems and Issues in Linking Assessments Across Languages
Authors: Sireci Stephen G.
Published: October 2005

What problems arise in translating a test to other languages? How can performance be compared for students who take different language versions of a test? What designs can be used for linking studies?
The Centrality of Test Use and Consequences for Test Validity
Authors: Shepard Lorrie A.
Published: October 2005

What are the origins of consequential validity? What is the role of intended test use in validation? Is the study of unintended effects part of validation? What practical problems does this pose?
Consequential validity: Right Concern‐Wrong Concept
Authors: Popham W. James
Published: October 2005

Why is the I985 view of validity in the Standards clear and useful? Does the issue of social consequences confuse the meaning o f validity? Can we address test use consequences without making them a facet of validity?
Development and Validation of Professional Development Resource Materials for Teachers Cbvering Commwnicathg and Interpreting Assessment Results
Authors: Plake Barbara S. , Impara James C. , Wise Vicki L.
Published: October 2005

How were the materials tested? How did the reactions of teachers compare to those of measurement students? How can these materials be used effectively?
Development of Performance Assessments in Science: Conceptual, Practical, and Logistical Issues
Authors: Wested Gullermo Solano‐Flores , Shavelson Richard J.
Published: October 2005

What are practical and logistical constraints in developing science performance assessments (SPAs)? What are key components in a framework for conceptualizing the process? What are the major steps in SPA development?
A Brief History of Item Theory Response
Authors: Bock R. Darrell
Published: October 2005

How did early work of Binet and Thurstone foreshadow item response theory? What connections to work in other areas are not widely recognized? What are current application trends?
A History of Social Science Measurement
Authors: Wright Benjamin D.
Published: October 2005

What are the historic oriains of measurement? What v are the basic requirements for fundamental measurement? What are some common pitfalls to avoid? D o item response models meet these needs?
Developing a Strong Program of Construct Validation: A Test Anxiety Example
Authors: Benson Jeri
Published: October 2005

What would a strong program of construct validation look like for the concept of test anxiety? What are the components of strong validation programs? In particular, how does structural equation modeling fit into such a program?
How to Prepare Students for PerEorrnance Assessments
Authors: Mehrens William A. , Popham W. James , Ryan Joseph M.
Published: October 2005

Now that the use of performance assessments for certification, graduation, and classroom use is burgeoning, what are the responsibilities of educators for preparing students for such assessments? Do the same guidelines apply for both multiple‐choice tests and performance assessments?
The Demand for Standardized Student Testing
Authors: Phelps Richard P.
Published: October 2005

What does the American public think about standardized testing? D o Americans test too much? Does the public believe that tests are a good way to learn about how well students are doing and to encourage them to d o better? What d o students, teachers, and administrators think?
Professional Responsibility: Does the End Always Justm the Means?
Authors: Schmeiser Cynthia Board
Published: October 2005

What are the responsibilities of measurement professionals in ensuring the integrity of standardized test results? How can political and practical issues be balanced against professional concerns? Are there instances where the ends justify the means?
Validity and Automad Scoring: It's Not Only the Scoring
Authors: Bennett Randy Elliot , Bejar Isaac I.
Published: October 2005

What are the validity issues involved in automated scoring of tests? What is the nature of the interplay among construct definition, task design, examinee interface, tutorial, test development tools, and automated scoring and reporting?
Protecting the Integrity of Computerized Testing Item Pools
Authors: Way Walter D.
Published: October 2005

What are the issues and techniques involved in protecting the integrity of item pools in computerized testing? How can item exposure be limited? How do security issues differ in computerized testing and paper‐and‐pencil testing?
Using Focus Groups to Examine the Consequential Aspect of validity
Authors: Chudowsky Naomi , Behuniak Peter
Published: October 2005

How can focus groups be used to examine issues of consequential validity in large‐scale assessment? In relation to a new large‐scale assessment, what are teacher concerns, and how do these concerns differ by type of school district? What are the strengths and weaknesses in this approach to looking at consequential validity?
Investigating the Consequential Aspects of Validity: Who Is Responsible and What Should They Do?
Authors: Yen Wendy M.
Published: October 2005

Where do we stand today as the concept of the consequential aspect of validity gains maturity? What are the implications for the various stakeholders in the measurement enterprise?
Consequential Validity From the Test Developer's Perspective
Authors: Reckase Mark D.
Published: October 2005

What would a test developer have to do in order to address the consequential aspects of validity? Is it, in fact, possible to meet the requirements implied from a consequential perspective on test validity?
A Framework for Evaluating the Consequences Programs of Assessment
Authors: Lane Suzzane , Parke Carol S. , Stone Clement A.
Published: October 2005

How can the enterprise of looking at the consequences of testing in America be moved forward? What are the responsibilities of the key actors? How can we accumulate the evidence that we need?
Partitioning Responsibility for the Evaluation of the Conseqyences of Assessment Programs
Authors: Linn Robert L.
Published: October 2005

How can the enterprise of looking at the consequences of testing in America be moved forward? What are the responsibilities of the key actors? How can we accumulate the evidence that we need?
Strategies for Identifying Research Topic in Educational Measurement
Authors: Millman Jason
Published: October 2005

How might a novice researcher go about setting a research agenda? What are the most productive avenues for generating research questions? What areas of educational measurement currently seem most promising?
Teaching About Communicating Assessment Results and Grading
Authors: Brookhart Susan M.
Published: October 2005

How should a course on classroom assessment differ from an introductory psychometrics course? How should such a course be taught? How large a role should instruction concerning grading and communicating play in such a course?
Evaluating Classroom Assessment Training in Teacher Education Programs
Authors: Stiggins Richard J.
Published: October 2005

What should training for teachers look like? Do our assessment practices align well with what we would like our students to do? How can you self‐assess assessment training at your institution?
Validating Measures of Performance
Authors: Kane Michael , Crooks Terence , Cohen Allan
Published: October 2005

How can the validity of performance assessments be established? What type of logical argumentation is necessary to generalize from a particular assessment to the ability or abilities of interest?
Teaching About Performance Assessment
Authors: Arter Judy
Published: October 2005

How should we teach prospective teachers about performance assessment? What are the issues and concerns that new teachers will encounter as they begin their teaching careers? How can assessment and instruction be better integrated in classrooms?
Where Large Scale Educational Assessment Is Heading and Why It Shouldn't
Authors: Popham W. James
Published: October 2005

Where are we headed in large scale assessment? Who is determining what the direction should be? Why does instructional enhancement take a back seat to educational accountability?
The Validity of Comparative Educational Studies
Authors: Bechger Timo M. , van den Wittenboer Godfried , Hox Joop J. , De Glopper C.
Published: October 2005

What are the conditions that will allow for appropriate comparisons among groups“ Should the concept of comparative validity be added to the lexicon of the psychometrician“
Is the Akebono School Failing Its Best Students? A Hawaiian Adventure in Regression
Authors: Wainer Howard
Published: October 2005

Why do educators fall prey to the trap of regression toward the mean“ How can we estimate how large an effect regression will have in a particular situation“ Is the Akebono School failing its best students?
Validity Arguments for High‐stakes Testing: In Search of the Evidence
Authors: Haertel Edward H.
Published: October 2005

What do we need to do to establish the validity of high‐stakes testing programs? How can we move away from a “confirmationist” bias? What questions haven't we been asking that we need to ask?
Comparing the Incomparable: An Essay on the Importance of Big Assumptions and Scant Evidence
Authors: Wainer Howard
Published: October 2005

What are we doing when we make a comparison between two groups of people“ How is this comparison complicated when the two groups do not take identical sets of items“ What can be said about the groups if they take no items in common?
Values and State Ratings: An Examination of the State‐By‐State Education Indicators in Quality Counts
Authors: Camilli Gregory , Firestone William A.
Published: October 2005

What are educational indicators? What are the strengths and weaknesses of using such indicators? What do we need to learn about educational indicators?
(Mis) Conception About Generalizability Theory
Authors: Brennan Robert L.
Published: October 2005

In what sense is generalizability theory an extension of classical test theory? In what sense is generalizability theory an application of analysis of variance.
Trends in Large‐Scale Testing Outside the United States
Authors: Phelps Richard P.
Published: October 2005

Is standardized testing on the decline outside the United States? How much testing goes on internationally? What types of testing and for what purposes?
Assessing Differential Item Functioning in Performance Assessment: Review and Recommendations
Authors: Penfield Randall D. , Lam Tony C. M.
Published: October 2005

How can we best extend DIF research to performance assessment? What are the issues and problems surrounding studies of DIF on complex tasks? What appear to be the best approaches at this time?
Impact of Accommodation Strategies on English Language Learners' Test Performance
Authors: Abedi Jamal , Lord Carol , Hofstetter Carolyn , Baker Eva
Published: October 2005

How can we make effective accommodations for nonnative English speakers taking mathematics measures? What kinds of accommodations are useful and what kinds are not? Where does this research need to go in the future?
Schooling and the Norming of Intelligence Test Scores
Authors: Cahan Sorel
Published: October 2005

How does schooling affect the development of intelligence in children? How should the amount of schooling be considered when developing norms for turning intelligence test performance into IQ scores?