bluuebunny/crossref_metadata_2025 huggingface dataset:
Recruiting the Next Generation of Measurement Professionals1
Authors: Sireci Stephen G.
Published: October 2005

How are we going to find the next generation of measurement professionals? What are the issues pertinent to recruitment and training?

If there were any period one would desire to be born in, is it not the age of revolution; when the old and the new stand side by side and admit of being compared; when the energies of all … are searched by fear and by hope; when the historic glories of the old can be compensated by the rich possibilities of the new era? This time, like all times, is a very good one, if we but know what to do with it. (Ralph Waldo Emerson, Phi Beta Kappa Address, 1837.)
Assessing Students' Opportunity to Learn: Teacher and Student Perspectives
Authors: Herman Joan L. , Klein Davina C. D. , Abedi Jamal
Published: October 2005

How can we assess the opportunity that students have to learn the material they find on tests? How do students' perceptions of opportunity to learn differ from their teachers?
Linking Assessments Effectively: Purpose and Design
Authors: Kolen Michael J.
Published: October 2005

What problems can occur in linking old and newer forms of a test? Does the length of the tests being equated affect the results of a linking study? When tests are designed for different purposes, can the scores be meaningfully interpreted after equating?
Can Knowledge of Erasure Behavior be Used as an Indicator of Possible Cheating?
Authors: Qualls Audrey L.
Published: October 2005

What is the typical amount of student answer‐changing that takes place in a low‐stakes test environment? Can we use this information to look for evidence of improper answer‐changing (cheating) on high‐stakes test?
Secondary Teachers' Classroom Assessment and Grading Practices
Authors: McMillan James H.
Published: October 2005

What variables do secondary teachers use to assign grades? What is the relationship between classroom assessment practices and grading? Is the form of classroom assessment related to grade level, subject area, and student ability?
Audio Computer‐Based Tests (CBTs): An Initial Framework for the Use of Sound in Computerized Tests
Authors: Parshall Cynthia G. , Balizet Sha
Published: October 2005

Are there important aspects of human ability that we have not been measuring? What are the purposes and types of audio that are possible in computerized tests? Will the use of audio in computer‐based tests lead to more valid and reliable measurement?
Illustrating the Utility of Differential Bundle Functioning Analyses to Identify and Interpret Group Differences on Achievement Tests
Authors: Gierl Mark J. , Bisanz Jeffrey , Bisanz Gay L. , Boughton Keith A. , Khaliq Shameem Nyla
Published: October 2005

What is differential bundle functioning and how is this different from differential item functioning? Can test specifications be used to identify and aid in the interpretation of differential bundle functioning? How can differential bundle functioning lead to an improved understanding of why groups perform differently on achievement tests?
East Meets West in Assessment Development: Western Technical Assistance to Eastern Needs. Part II: Project and Team Management in the AERo Project in Romania
Authors: Bakker Steven , van Lent Gerben , Eekelen Annemarie de Knecht‐van
Published: October 2005

This is Part II of a three‐part series describing the consultancy and training demands that are encountered in technical assistance projects in Central and Eastern Europe. This part highlights project management and cooperation aspects in delivering support to the Romanian Assessment and Examination Service. Part I dealt with strategic issues for a successful delivery of “Western Technical Assistance to Eastern Needs.” Part III will address training of Polish subject experts, preparing to take on responsibility for a decentralized and drastically changed final exam, the Matura.
The Unfulfilled Promise of Classroom Assessment
Authors: Stiggins Richard J.
Published: October 2005

The purpose of this article is to review the status of classroom assessment as we enter the new millennium. Four key research summaries are reviewed to describe the dismal state of classroom assessment affairs during the past century. Then four critically important developments are cited as reasons for optimism about the future of classroom assessment. Finally, the author details a five‐part action plan for tapping the heretofore untapped potential of classroom assessment as a powerful school improvement tool.
Validity Issues in Computer‐Based Testing
Authors: Huff Kristen L. , Sireci Stephen G.
Published: October 2005

Advances in technology are stimulating the development of complex, computerized assessments. The prevailing rationales for developing computer‐based assessments are improved measurement and increased efficiency. In the midst of this measurement revolution, test developers and evaluators must revisit the notion of validity. In this article, we discuss the potential positive and negative effects computer‐based testing could have on validity, review the literature regarding validation perspectives in computer‐based testing, and provide suggestions regarding how to evaluate the contributions of computer‐based testing to more valid measurement practices. We conclude that computer‐based testing shows great promise for enhancing validity, but at this juncture, it remains equivocal whether technological innovations in assessment have led to more valid measurement.
The Challenge and Attainability of Goals for Adequate Yearly Progress
Authors: Schwarz Richard D. , Yen Wendy M. , Schafer William D.
Published: October 2005

For standards‐referenced assessment systems, there is a tension between setting high goals versus attainable ones for adequate yearly progress. The more relevant information that policymakers can consider in setting these goals, the more likely it will be that productive goals are set. Some strategies are presented here that can be used either singly or in combination to evaluate the level of rigor implied by these expectations. These strategies relate to understanding how likely it is, based on past experience, that a particular goal can be met.
Some Problems, Pitfalls, and Paradoxes in Educational Measurement
Authors: Brennan Robert L.
Published: October 2005

This article provides brief discussions of an eclectic set of problems, pitfalls, and paradoxes that challenge measurement theory and practice, especially for K–12 achievement testing. The technical issues considered are: true scores and error scores, fixed versus random facets, lower limits to reliability, the reliability‐validity paradox, reliability of difference scores, equating and linking, and scaling. The practical issues considered are: validation, standard setting, instructional improvement and high‐stakes accountability, multiple measures, adequate yearly progress, the National Assessment of Educational Progress as a confirmation model, and capacity.
More Unintended Consequences of High‐Stakes Testing
Authors: Cizek Gregory J.
Published: October 2005

This article explores the basis of negative sentiments toward and current critiques of high‐stakes student testing from within the education profession. To promote some balance for current policy debates, evidence for 10 unintended, unrecognized, or unarticulated positive consequences is provided. The article concludes with an examination of the relationship between high‐stakes testing and accountability systems.
Stakeholders in Comprehensive Validation of Standards‐Based Assessments: A Commentary
Authors: Crocker Linda
Published: October 2005

Linda Cracker is a Professor of Educational Psychology, University ofFlorida, P.O. Box 11 7047 Gainesville, FL 32611‐7043. Her areas of specialization are assessment development, validation, and test‐taking behavior.
Assessment Validation in the Context of High‐Stakes Assessment
Authors: Ryan Katherine
Published: October 2005

Including the perspectives of stakeholder groups (e.g., teachers, parents) can improve the validity of high‐stakes assessment interpretations and uses. How stakeholder groups view high‐stakes assessments and their uses may differ significantly from state‐level policy officials. The views of these stakeholders can contribute to identifying the strengths and weaknesses of the intended assessment interpretations and uses. This article proposes a process approach to validity that addresses assessment validation in the context of high‐stakes assessment. The process approach includes a test evaluator or validator who considers the perspectives of five stakeholder groups at four different stages of assessment maturity in relationship to six aspects of construct validity. The tasks of the test evaluator and how stakeholders' views might be incorporated are illustrated at each stage of assessment maturity. How the test evaluator might make judgments about the merit of high‐stakes assessment interpretations and uses is discussed.
Standard Setting as a Participatory Process: Implications for Validation of Standards‐Based Accountability Programs
Authors: Haertel Edward H.
Published: October 2005

In standards‐based accountability programs, test scores are interpreted with reference to cut scores that establish categories like “proficient” or “below basic.” The meaning of these cut scores is set forth in their associated “performance standards.” Validity arguments for such interpretations require both a criterion‐referenced score scale and a legitimate exercise of authority by those who set the standards. Stakeholder participation in a rational and coherent deliberative process is necessary to assure that these conditions are satisfied. This article sets forth a framework for the required validity argument and suggests possible ways to enable such participation. A new standard‐setting method, the “briefing book” method, is suggested for possible study.
Strategies for Examining the Consequences of Assessment and Accountability Programs
Authors: Lane Suzanne , Stone Clement A.
Published: October 2005

This article addresses issues in evaluating the consequences of assessment programs that are developed for the purpose of holding schools accountable to state standards. After providing a brief review of research examining consequential evidence, a validation study to obtain consequential evidence for state assessment and accountability programs is proposed. The proposal includes a validity argument, a set of propositions that follow from the validity argument, a delineation of the consequential evidence needed, and a way to model the relationship between performance gains and school, principal, teacher, and student variables.
Validating High‐Stakes Testing Programs
Authors: Kane Michael
Published: October 2005

Cronbach made the point that for validity arguments to be convincing to diverse stakeholders, they need to be based on assumptions that are credible to these stakeholders. The interpretations and uses of high‐stakes test scores rely on a number of policy assumptions about what should be taught in schools, and more specifically, about the content standards and performance standards that should be applied to students and schools. For example, a high‐school graduation test can be developed as a measure of readiness for the world of work, for college, or for citizenship and the activities of daily life. The assumptions built into the assessment need to be subjected to scrutiny and criticism if a strong case is to be made for the validity of the proposed interpretation and use.
Scoring Examinee Responses for Multiple Inferences: Multiple Scoring in Assessments
Authors: Ercikan Kadriye
Published: October 2005

Multiple scoring is widely used in large‐scale assessments. The use of a single response for making multiple inferences as is done in multiple scoring has implications on the validity of these inferences and interpretations based on assessment results. The purpose of this article is to review two types of multiple scoring practices and discuss how multiple scoring affects inferences.
Training and Scoring Issues Involved in Large‐Scale Writing Assessments
Authors: Moon Tonya R. , Hughes Kevin R.
Published: October 2005

Many states are implementing direct writing assessments to assess student achievement. While much literature has investigated minimizing raters' effects on writing scores, little attention has been given to the type of model used to prepare raters to score direct writing assessments. This study reports on an investigation that occurred in a state‐mandated writing program when a scoring anomaly became apparent once assessments were put in operation. The study indicates that using a spiral model for training raters and scoring papers results in higher mean ratings than does using a sequential model for training and scoring. Findings suggest that making decisions about cut‐scores based on pilot data has important implications for program implementation.
Where Did All the Data Go? Internet Security for Web‐Based Assessments
Authors: Shermis Mark D. , Averitt Jason
Published: October 2005

The purpose of this article is to enumerate a series of security steps that might be taken by those researchers or organizations that are contemplating web‐based tests and performance assessments. From a security viewpoint, much of what goes on with web‐based transactions is similar to other general computer activity, but the recommendations here focus on what can be done to avoid the loss, compromising, or modification of data collected by or stored through the Internet.
Alternative Interpretations of Alternative Assessments: Some Validity Issues in Educational Performance Assessments
Authors: Bachman Lyle F.
Published: October 2005

The use of alternative assessments has led many researchers to reexamine traditional views of test qualities, especially validity. Because alternative assessments generally aim at measuring complex constructs and employ rich assessment tasks, it becomes more difficult to demonstrate (a) the validity of the inferences we make and (b) that these inferences extrapolate to target domains beyond the assessment itself. An approach to addressing these issues from the perspective of language testing is described. It is then argued that in both language testing and educational assessment we must consider the roles of both language and content knowledge, and that our approach to the design and development of performance assessments must be both construct‐based and task‐based.1
NCME Members' Suggestions for Recruiting New Measurement Professionals
Authors: Sireci Stephen G. , Khaliq Shameem Nyla
Published: October 2005

During February and March 2001, a survey on ideas for recruiting new educational measurement professionals was posted on the website of the National Council on Measurement in Education
A Practical Guide to Practice Analysis for Credentialing Examinations
Authors: Raymond Mark R.
Published: October 2005

Practice analysis (i.e., job analysis) serves as the cornerstone for the development of credentialing examinations and is generally used as the primary source of evidence when validating scores on such exams. Numerous methodological questions arise when planning and conducting a practice analysis, but there is little consensus in the measurement community regarding the answers to these questions. This article offers recommendations concerning the following issues: selecting a method of practice analysis; developing rating scales to describe practice; determining the content of test plans; using multivariate procedures for structuring test plans; and determining topic weights for test plans. The article closes by suggesting several references for further reading.
District Accountability Without a State Assessment: A Proposed Model
Authors: Buckendahl Chad W. , Impara James C. , Plake Barbara S.
Published: October 2005

Most states have adopted assessment and accountability systems that involve common measures of student performance. A state assessment system that allows school districts to choose the specific strategies they use to measure student performance on state‐adopted content standards presents a unique state accountability challenge. The authors propose an accountability model that addresses this challenge using a combination of student performance, technical quality, and noncognitive indicators of performance. They also describe a study that evaluated the proposed model using data from all school districts in a southern state.
What Do School‐Level Scores From Large‐Scale Assessments Really Measure?
Authors: Sicoly Fiore
Published: October 2005

Although assessments of mathematics, reading, and writing are assumed to measure distinct academic skills, this may be difficult owing to the pervasive influence of general ability on performance. Factor analyses of school‐level data from 14 large‐scale assessment programs revealed that 80% of the variance in mathematics, reading, and writing scores was due to a common, underlying factor. Multiple regression analyses confirmed that scores contribute little information that is unique to a particular subject (6% or less). Although different assessments may create the illusion of providing unique information, they may be tapping into generic cognitive abilities that cut across content areas. These results raise suspicions about the value and validity of interpretations based on school‐level subject area scores.
Stability of Country Rankings Across Item Formats in the Third International Mathematics and Science Study
Authors: O'Leary Michael
Published: October 2005

Multiple‐choice, short‐answer, and extended‐response item formats were used in the Third International Mathematics and Science Study to assess student achievement in mathematics and science at Grades 7 and 8 in more than 40 countries around the world. Data pertaining to science indicate that the standings of some countries relative to others change when performance is measured via the different item formats. The question addressed in the present article is the following: Can the instability of ranks in this case be attributed principally to item format, or are other important factors at work? It is argued that the findings provide further evidence that comparing student achievement across countries is a very complex undertaking indeed.
News or Noise? An Analysis of U.S. News and World Report's Ranking Scores
Authors: Clarke Marguerite
Published: October 2005

One of the biggest criticisms of the U.S. News and World Report rankings of colleges and graduate schools is that they are “falsely precise,’ creating a vertical column where a group might more properly exist. In this article, the jackknife method coupled with a linear regression model is used to illustrate uncertainty around the overall scores used to rank schools. Results indicate that the top 50 schools in each ranking can be shown to group in two to three bands. The implications of these findings are discussed in terms of the interpretation of a school's position in a ranking as well a s changes in that position over time. The article concludes with some recommenda‐ tions for both producers and consumers of these rankings.
Some Common Misconceptions about Tests and Testing
Authors: Hoover H. D.
Published: October 2005

This article offers an explanation for gender differences in scores on college admissions exams. It also discusses the role of norms in achievement batteries and the use of standardized tests in the primary grades.
The Accuracy of Students' Reports of Course Taking in the 1994 National Assessment of Educational Progress
Authors: Niemi Richard G. , Smith Julia
Published: October 2005

The National Assessment of Educational Progress (NAEP), in addition to the assessment itself, collects students' reports of courses they have taken in the subject area of the test. A comparison of reports from the 1994 history NAEP with information from the 1994 High School Transcript Study reveals that students greatly overestimate the number of history classes taken and fail to distinguish among different types of history classes. Though less definitive, limited evidence suggests that these erroneous reports may result in misunderstandings of the impact of course work on student knowledge. Finally, there is indirect evidence of bias in reports of course content coverage in NAEP. We discuss various alternatives to the continued use of inaccurate reports.
Interpreting the Results of Three Different Standard‐Setting Procedures
Authors: Green Donald Ross , Trimble C. Scott , Lewis Daniel M.
Published: October 2005

Different standard‐setting procedures usually produce different cut points even if each has a rational basis. In 2000, three standard‐setting procedures were implemented to set cut scores in each of the 18 grade/content areas comprising Kentucky's state assessment system: the Contrasting Groups, Bookmark, and Jaeger‐Mills procedures. Subsequently, participants from each of the three procedures worked together in each grade/content area to synthesize the results. These synthesis participants considered the results of, and examined the materials and information provided by, each of the three separate procedures. In this article the synthesis processes are described and discussed.
Historical High‐Stakes Policies Relating to Unintended Consequences of High‐Stakes Testing
Authors: Rich William
Published: October 2005

Current high‐stakes, standardized testing policy is discussed through historical analogy with Chairman Mao's famine in China and the Maginot Line in France. Both of these national, high‐stakes policies resulted in catastrophic failure. If the accountability movement's goals are to improve our ability to compete economically with other nations, we may also be heading for failure. Assessment should be aligned with the skills and knowledge crucial to our success in the future such as collaboration, experimentation, and comfort with ambiguity. Funds currently allocated to standardized testing should be reallocated to the development of measures for such skills and knowledge.
Comment on Cizek's “More Unintended Consequences of High‐Stakes Testing”
Authors: Camilli Gregory
Published: October 2005

This discussion provides a response to Gregory Cizek's “More Unintended Consequences of High‐Stakes Testing.” The current policy debate is characterized by extreme positions both for and against testing, and Cizek's article balances positive consequences against antitesting critics. However, there is no evidence that high‐stakes testing per se has substantial positive consequences–although there is optimism that aligned educational systems, in which testing is a component, may lead to higher levels of student attainment.
E
Authors: Cizek Gregory J.
Published: October 2005

This article responds to two critiques of an earlier article on the unintended consequences of high‐stakes testing. The exchange of viewpoints suggests that the successful resolution of current policy debates surrounding such testing and companion accountability systems rests on the ability to: recognize, document, and disseminate instances of real learning gains; develop improved methods and assessment systems that provide instructional relevance and utility of test results; and negotiate a balance between the role of large‐scale assessment in identifying achievement deficits and the responsibility and resources that educational systems have to redress identified deficiencies.
Multiple Measures: Toward Tiered Systems
Authors: Baker Eva L.
Published: October 2005

This article examines multiple measures of performance in school accountability systems from two perspectives: laterally (different indicators of different domains) and vertically (indicators that are at different levels of depth of the same domain). Organizational responsibility and instructional sensitivity are examined. In particular, alternative procedures are explored for integrating into the multiple measures concept external, uniform top‐down measures and responsive, locally adaptive bottom‐up measures.
Using Multiple Measures to Address Perverse Incentives and Score Inflation
Authors: Koretz Daniel
Published: October 2005

The principle that important decisions should not be based on a single measure is axiomatic, if widelu ignored in practice. The traditional rationale is the risk of incorrect decisions from incomplete and error‐prone data. The current high‐stakes uses of test scores increase the need for multiple measures for two distinct reasons: the risk of score inflation and the potential for perverse incentives for educators and students. Addressing these two issues may require focusing accountability on measures of schooling as well as a much wider range of measures of student outcomes. The difficulties of pursuing this approach are described, and some possible directions for research and development are noted.
A State Perspective on Multiple Measures in School Accountability
Authors: Schafer William D.
Published: October 2005

Several meanings of the term multiple measures exist. One of these is the use of assessments from different sources, such as an external test, along with a state‐developed test. The use of multiple sources is increasing, especially due to increased federal Title I requirements for state accountability programs and associated increases in the amount and costs of mandated testing. Several issues seem pertinent for states considering combining assessments from internal sources (usually criterion‐referenced tests) and external sources (usually norm‐referenced tests) into their accountability programs. These are explored from the standpoint of the impact of federally required decision making for schools based on test data. Other possible uses are mentioned briefly.
Multiple Measures and High‐Stakes Decisions: A Framework for Combining Measures
Authors: Chester Mitchell D.
Published: October 2005

The article presents a framework for combining multiple measures to reach high‐stakes decisions. Criteria are identified for the employment of conjunctive, compensatory, and complementary approaches to combining measures. The framework is illustrated through the documentation of the School District of Philadelphia's initiative to employ multiple measures, including standardized test scores, to determine promotion decisions. The author demonstrates that the use of multiple measures itself does not necessarily improve the reliability and validity of the decisions. It is the logic by which the measures are combined that determines the accuracy and appropriateness of the decisions reached.
Teaching for the Test: Validity, Fairness, and Moral Action
Authors: Crocker Linda
Published: October 2005

In response to heightened levels of assessment activity at the K‐12 level to meet requirements of the No Child Left Behind Act of 2001, measurement professionals are called to focus greater attention on four fundamental areas of measurement research and practice: (a) improving the research infrastructure for validation methods involving judgments of test content; (b) expanding the psychometric definition of fairness in achievement testing; (c) developing guidelines for validation studies of test use consequences; and (d) preparing teachers for new roles in instruction and assessment practice. Illustrative strategies for accomplishing these goals are outlined.
Reliability of No Child Left Behind Accountability Designs
Authors: Hill Richard K. , DePascale Charles A.
Published: October 2005

The No Child Left Behind Act of 2001 requires states to establish accountability systems that are both valid and reliable. If one follows the language of the law literally, there is no design that will meet both requirements. If one interprets the law more flexibly, it is possible to create such a design. States will need to approach the problem carefully if they are going to appropriately balance the various probabilities of making incorrect decisions about schools.
Aligning Tests with States' Content Standards: Methods and Issues
Authors: Bhola Dennison S. , Impara James C. , Buckendahl Chad W.
Published: October 2005

The No Child Left Behind Act of 2001 (NCLB) requires states to have developed content standards in Reading and Mathematics, and to employ some form of statewide assessment. For accurate inferences about student achievement and growth over time, these assessments must measure the knowledge and skills deemed valuable and described in these content standards. This article provides a description of models for evaluating the degree of alignment between assessments and content standards. Challenges to these models are described and discussed, and solutions are suggested. Two empirical alignment studies are summarized and implications of using different alignment models for aligning commercially available tests with content standards are also discussed.
Using Multidimensional Item Response Theory to Evaluate Educational and Psychological Tests
Authors: Ackerman Terry A. , Gierl Mark J. , Walker Cindy M.
Published: October 2005

Many educational and psychological tests are inherently multidimensional, meaning these tests measure two or more dimensions or constructs. The purpose of this module is to illustrate how test practitioners and researchers can apply multidimensional item response theory (MIRT) to understand better what their tests are measuring, how accurately the different composites of ability are being assessed, and how this information can be cycled back into the test development process. Procedures for conducting MIRT analyses–from obtaining evidence that the test is multidimensional, to modeling the test as multidimensional, to illustrating the properties of multidimensional items graphically‐are described from both a theoretical and a substantive basis. This module also illustrates these procedures using data from a ninth‐grade mathematics achievement test. It concludes with a discussion of future directions in MIRT research.
Reconceptualizing Validity for Classroom Assessment
Authors: Moss Pamela A.
Published: October 2005

This article explores the shortcomings of conventional validity theory for guiding classroom assessment practice and suggests additional theoretical resources from sociocultural theory and hermeneutics to complement and challenge conventional theory. To illuminate these concerns and possibilities in a concrete context, the author uses her own classroom experience in teaching a qualitative research methods course. The importance of examining cases of assessment practice in context for developing, teaching, and evaluating validity theory is discussed.
Reconsidering Reliability in Classroom Assessment and Grading
Authors: Smith Jeffrey K.
Published: October 2005

It is argued that classroom assessment evolves from a different set of issues and demands from more traditional measurement concerns and that approaches to reliability developed from traditional concerns are not appropriate for most classroom settings. The assessment and grading issues for high school instruction are examined from the perspective of reliability. An alternative conceptualization of reliability, sufficiency of information, is proposed and explored. This conceptualization is based on the argument that at a rudimentary level, reliability theory is based on the notion of having enough information to make decisions or draw inferences.
Understanding and Improving Teachers' Classroom Assessment Decision Making: Implications for Theory and Practice
Authors: McMillan James H.
Published: October 2005

Teacher assessment decision making is described as a process that may be important in developing a measurement theory for classroom assessment. Teacher beliefs and values about what and how to assess often conflict with the pressures of external demands, especially recent high‐stakes large‐scale testing. With this sense of tension, teachers make assessment decisions that will engage and motivate students and enhance learning. Implications for a measurement theory of classroom assessment, as well as teacher preservice and inservice, are explored. The article suggests that measurement specialists need to adapt technical testing principles to be more relevant to the nature of classroom assessment decision making.
Construct‐Irrelevant Variance in High‐Stakes Testing
Authors: Haladyna Thomas M. , Downing Steven M.
Published: October 2005

There are many threats to validity in high‐stakes achievement testing. One major threat is construct‐irrelevant variance (CIV). This article defines CIV in the context of the contemporary, unitary view of validity and presents logical arguments, hypotheses, and documentation for a variety of CIV sources that commonly threaten interpretations of test scores. A more thorough study of CIV is recommended.
Software Use in Psychometric Research
Authors: Skaggs Gary
Published: October 2005

Research on psychometric methods is heavily dependent on software. The quality, availability, and documentation of such software are critical to the advancement of the field. In 2000, an ad hoc committee of NCME recommended that NCME adopt policies that promote greater availability and better documentation of software. This article follows the ad hoc committee's report by examining the use of software in four top‐tiered journals in recent years. The results indicated that the most frequently cited programs were those written by the articles' authors. The documentation and availability for these programs are often not clear, particularly for software used for simulations. The use of proprietary software was not widespread in the four journals, but there is still room for concern in the future. This article recommends that NCME form a permanent committee to address software issues.
History and Background of Nebraska's School‐based Teacher‐led Assessment and Reporting System (STARS)
Authors: Roschewski Pat
Published: October 2005

Nebraska's approach to standards, assessment, and accountability, the School‐based Teacher‐led Assessment and Reporting System (STARS) is based upon local control and the belief that classrooms and teachers must be at the heart of student learning and accountability. STARS relies on locally‐developed assessment systems to accurately measure and report student performance on state content standards. Each local system in Nebraska's 500+ school districts is reviewed for technical quality, and districts are publicly rated for assessment quality and student performance. The purpose of this article is to establish the historical background.
Technical Quality Criteria for Evaluating District Assessment Portfolios Used in the Nebraska STARS
Authors: Plake Barbara S. , Impara James C. , Buckendahl Chad W.
Published: October 2005

Nebraska districts use different strategies for measuring student performance on the state's content standards. District assessments differ in type and technical quality. Six quality criteria were endorsed by the state. These criteria cover content and curricular validity, fairness, and appropriateness of score interpretations. District assessment portfolios document how well assessments meet these criteria. Districts receive ratings on how well their assessments meet each of the quality criteria and are given a rating from Unacceptable to Exemplary. This article presents these technical quality criteria and explains how they are (a) individually rated and (b) combined for the district's overall quality rating.
A Strategy for Evaluating District Developed Assessments for State Accountability
Authors: Buckendahl Chad W. , Plake Barbara S. , Impara James C.
Published: October 2005

As part of Nebraska's assessment and accountability system, districts' local assessment systems are evaluated for their psychometric quality. This article provides an overview of a two‐stage evaluation strategy, discusses how it was applied in Nebraska, and presents results from the first three years of the evaluation process. Benefits of the method include an emphasis on formative evaluation and promotion of improved assessment quality at the local level. A limitation of the model is the inability to make refined comparisons of student performance across districts on the assessments. Results from the first three years suggest that greater specificity in the review criteria and additional reviewer calibration activities are needed.
Facilitating the Development of Assessment Literacy
Authors: Lukin Leslie E. , Bandalos Deborah L. , Eckhout Teresa J. , Mickelson Kristine
Published: October 2005

When STARS reform efforts were launched in 2000, teacher training in assessment was seen as crucial to the success of the program. The STARS reform efforts focus on both supporting the implementation of quality classroom assessment practices and implementing a district‐based accountability system. The training programs described in this article were developed in response to one or both of these needs. Two of the programs were designed to provide training to experienced teacher. The other two programs were designed to meet the needs of pre‐service teachers. Evidence suggests that the training programs have had a positive impact on teacher confidence, knowledge, and skill in key areas of assessment. In addition, there also appears to be evidence, while somewhat limited, which suggests students also experience positive outcomes.
Can a Teacher‐Led State Assessment System Work?
Authors: Bandalos Deborah L.
Published: October 2005

Recent implementation of Nebraska's Standards‐based Teacher‐led Assessment and Reporting System (STARS) introduced a unique opportunity to examine the benefits and drawbacks of a teacher‐led state assessment system. STARS is unique among state assessment systems in that statewide tests are replaced by locally developed assessments designed by teachers. Benefits of this system are found to be greater teacher use of results for instructional planning, greater diagnostic utility of the assessments, and gains in assessment literacy. Disadvantages include the time‐consuming and stressful nature of creating localized assessments. Compatibility of STARS with the requirements of the No Child Left Behind Act are discussed.
Validity of High‐Stakes Assessment: Are Students Engaged in Complex Thinking?
Authors: Lane Suzanne
Published: October 2005

The validity of high‐stakes assessments and accountability systems is discussed in relation to the requirements of No Child Left Behind (NCLB). The extent to which content standards and assessments are cognitively rich, the challenges in setting performance standards, and the impact of high‐stakes assessments on instruction and student learning are addressed. The article argues for quality content standards, cognitively rich assessments, and a cohesive, balanced assessment system.
Avoiding Misconception, Misuse, and Missed Opportunities: The Collection of Verbal Reports in Educational Achievement Testing
Authors: Leighton Jacqueline P.
Published: October 2005

The collection of verbal reports is one way in which cognitive and developmental psychologists gather data to formulate and corroborate models of problem solving. The current use of verbal reports to design and validate educational assessments reflects the growing trend to fuse cognitive psychological research and educational measurement. However, doubts about the trustworthiness or accuracy of verbal reports may suggest a potential reversal of this trend. Misconceptions about the trustworthiness of verbal reports could signal misuse of verbal reports and, consequently, waning interest and missed opportunities in the description of cognitive models of test performance. In this article, misconceptions of verbal reports are addressed by (a) discussing the value of cognitive models for educational achievement testing; (b) addressing pertinent issues in the collection of verbal reports from students; and (c) concluding with avenues for a more productive union between cognitive psychological research and educational measurement.
Assessing School Readiness: Validity and Bias in Preschool and Kindergarten Teachers' Ratings
Authors: Mashburn Andrew J. , Henry Gary T.
Published: October 2005

As a part of efforts to evaluate and monitor the increasing public investment in early childhood education, teachers are being asked to assess children's school readiness. In this study, preschool teachers and kindergarten teachers rated children's skills in three areas (kindergarten readiness, academic skills, and communication skills), and these ratings were compared with direct assessments of the children's skills. Ratings by both groups of teachers tended to be more highly related to basic skills, such as counting and number naming, than to abilities such as solving applied problems and using expressive and receptive vocabulary. Preschool teachers' ratings had a lower association with children's observed skills and abilities than kindergarten teachers' ratings. Ratings of children attending Head Start were systematically inflated, but this relationship was mediated to a significant extent by the teachers' levels of education. More educated teachers rated children in a manner consistent with the children's directly assessed skills. Implications of these findings for informing future efforts to assess school readiness by using teacher ratings are discussed.
Setting Performance Standards: Contemporary Methods
Authors: Cizek Gregory J. , Bunch Michael B. , Koons Heather
Published: October 2005

This module describes some common standard‐setting procedures used to derive performance levels for achievement tests in education, licensure, and certification. Upon completing the module, readers will be able to: describe what standard setting is; understand why standard setting is necessary; recognize some of the purposes of standard setting; calculate cut scores using various methods; and identify elements to be considered when evaluating standard‐setting procedures. A self‐test and annotated bibliography are provided at the end of the module. Teaching aids to accompany the module are available through NCME.
Numerical Predictions of Transport Phenomena in a Proton Exchange Membrane Fuel Cell
Authors: Lin Yongming , Beale Steven B.
Published: October 2005

Transport phenomena play an important role in the performance of the proton exchange membrane fuel cell. Water generated by electrochemical reactions and transported by osmotic drag and back diffusion can cause saturation or flooding, preventing oxygen from reaching catalysis sites. Dehydration may also occur, resulting in poor proton conductivity. Balancing water content within the membrane involves judicious water and heat management strategies. In this paper, detailed mathematical models for the prediction of all significant aspects of physicochemical hydrodynamics for a proton exchange membrane fuel cell are employed. Fully coupled heat and mass transfer and electrochemistry are considered, and the dependence of water transport on these factors is taken into account. Two distinct approaches were considered: a fully three-dimensional approach and a hybrid scheme, whereby the electrochemistry and electric fields are treated as locally one dimensional in the membrane assembly. Comparisons between the two approaches are presented and discussed. The numerical results suggest a dependence of the rate-of-water removal on temperature, current density, and inlet humidification levels, and also that the oxygen concentration in the air channels significantly affects current density distribution.
Multiscale Parametric Studies on the Transport Phenomenon of a Solid Oxide Fuel Cell
Authors: Cheng C. H. , Chang Y. W. , Hong C. W.
Published: October 2005

This paper conducts a multiscale parametric study of temperature and composition effects on the transport phenomenon of a solid oxide fuel cell (SOFC). The molecular dynamics technique was employed to study the transport phenomenon of the solid electrolyte, which is made of yttria-stabilized zirconia. The influences of Y2O3 concentration and various operation temperatures on the SOFC were studied. Simulation results show that there exists an optimal concentration of 8mol% of Y2O3 in the composition for oxygen transport. Also higher operation temperature promotes the oxygen ion-hopping process that increases the ionic conductivity. A macroscale parametric study was also conducted in this paper to validate the influence of the temperature uniformity in the solid electrolyte by employing the computational fluid dynamics technique. The temperature distribution maps of a single-cell planar SOFC with coflow, counterflow and cross-flow channel designs are presented. The results conclude that the coflow configuration is the best design of the three.
Artificial Neural Network Modeling of PEM Fuel Cells
Authors: Ou Shaoduan , Achenie Luke E. K.
Published: October 2005

Artificial neural network (ANN) approaches for modeling of proton exchange membrane (PEM) fuel cells have been investigated in this study. This type of data-driven approach is capable of inferring functional relationships among process variables (i.e., cell voltage, current density, feed concentration, airflow rate, etc.) in fuel cell systems. In our simulations, ANN models have shown to be accurate for modeling of fuel cell systems. Specifically, different approaches for ANN, including back-propagation feed-forward networks, and radial basis function networks, were considered. The back-propagation approach with the momentum term gave the best results. A study on the effect of Pt loading on the performance of a PEM fuel cell was conducted, and the simulated results show good agreement with the experimental data. Using the ANN model, an optimization model for determining optimal operating points of a PEM fuel cell has been developed. Results show the ability of the optimizer to capture the optimal operating point. The overall goal is to improve fuel cell system performance through numerical simulations and minimize the trial and error associated with laboratory experiments.
Theoretical Calculation of the Electrical Potential at the Electrode/Electrolyte Interfaces of Solid Oxide Fuel Cells
Authors: Celik Ismail , Pakalapati S. Raju , Salazar-Villalpando Maria D.
Published: October 2005

A new semi-empirical model is formulated to calculate the potential differences at the cathode/electrolyte and electrolyte/anode interfaces separately for solid oxide fuel cells. The new model is based on a reduced reaction mechanism, and it accounts for the oxygen ion concentration at these interfaces. The model also considers the Gibbs free energy for the two electrode interfaces seperately. Results from case studies demonstrate the great potential of the proposed model
Review of the Computational Fluid Dynamics Modeling of Fuel Cells
Authors: Ma L. , Ingham D. B. , Pourkashanian M. , Carcadea E.
Published: October 2005

This paper presents a review of the current situation in the computational fluid dynamics (CFD) modeling of fuel cells and highlights the significant challenges that lie ahead in the development of a comprehensive CFD model for fuel cell applications. The paper focuses on the issues concerned with solid oxide fuel cells and proton exchange membrane fuel cells because these are the two most poplar and probably the most promising types of fuel cells for both stationary and transport applications. However, the general principles presented in this paper are applicable to all types of fuel cells.