ED435708 1999-09-00 Performance

Assessment Links in Science (PALS).

ERIC/AE Digest.

ERIC Development Team

www.eric.ed.gov

Table of Contents

If you're viewing this document online, you can click any of the topics below to link directly to that section.

Performance Assessment Links in Science (PALS). ERIC/AE Digest.... 1

PERFORMANCE ASSESSMENT IN SCHOOL REFORM.............. 2

TECHNOLOGY IN ASSESSMENT REFORM............................. 2

TECHNOLOGY IN PROFESSIONAL DEVELOPMENT................. 3

PERFORMANCE ASSESSMENT LINKS IN SCIENCE(PALS)......... 3

ISSUES AND EXPANSION OF THE SYSTEM............................ 4

REFERENCES.................................................................. 5

ERIC Identifier: ED435708

Publication Date: 1999-09-00

Author: Quellmalz, Edys - Schank, Patricia - Hinojosa, Thomas - Padilla, Christine

Source: ERIC Clearinghouse on Assessment and Evaluation Washington DC.

Performance Assessment Links in Science

(PALS). ERIC/AE Digest.

THIS DIGEST WAS CREATED BY ERIC, THE EDUCATIONAL RESOURCES

INFORMATION CENTER. FOR MORE INFORMATION ABOUT ERIC, CONTACT

ACCESS ERIC 1-800-LET-ERIC

Understanding the central role that performance assessment plays in standards-based

ED435708 1999-09-00 Performance Assessment Links in Science (PALS). ERIC/AE

Digest.

Page 1 of 6
reform, educators are seeking ways to use these assessments to test student learning.

Education agencies need pools of performance tasks to use in their student assessment

programs and in evaluations of state and federally funded programs. Reform projects

need standards-based assessment, too, as do teachers who are trying to implement

reforms. Experience indicates, however, that the level of effort and costs of developing

performance assessment tasks are very high (Quellmalz, 1984).

To meet the need for innovative approaches for sharing exemplary assessment

resources, to facilitate the development of new ones, and to further understand how the

use of standards-based performance assessment can advance educational reform, SRI

International is developing Performance Assessment Links in Science (PALS), an

on-line, standards-based, interactive resource bank of science performance

assessments. Coupled with the development of the resource bank is a program of

research on effective use of these resources.

This digest describes work-in-progress. SRI is currently seeking organizations to

participate in implementation studies involving the use of PALS. The "ideal" professional

development model would be for an organization or group of schools to want to focus

on classroom applications of science performance assessment and to use the PALS

resources in an initial professional development institute, followed by several

school-year meetings where teachers would bring samples of student work produced in

response to PALS investigations, with subsequent work on designing additional

classroom science assessments. For the accountability studies, SRI will work with a few

assessment programs from states, districts, or specially funded programs that are

interested in using the on-line, secure tasks in the Accountability Pool. Assessment

programs might use PALS to search, select, and plan assessment administrations

and/or conduct on-line rater training and scoring sessions.

PERFORMANCE ASSESSMENT IN SCHOOL

REFORM

Reform-minded programs have set out to develop alternative forms of student

assessment that call for students to construct rather than select responses.

Performance assessments are generally valued for testing students' deep

understanding of concepts and inquiry strategies, for making students' thinking visible,

and for measuring skills in communicating about their science knowledge. In addition,

performance assessments can present authentic, real-world problems that help

students to show they can apply academic knowledge to practical situations. On the

other hand, performance assessments are time-consuming and costly to develop,

logistically demanding, and of questionable utility if not developed and scored according

to sound measurement methods.

TECHNOLOGY IN ASSESSMENT REFORM

www.eric.ed.gov ERIC Custom Transformations Team

Page 2 of 6 ED435708 1999-09-00 Performance Assessment Links in Science (PALS). ERIC/AE

Digest.
Technology offers a powerful strategy for increasing the ease with which educators can

access and use standards-based student assessment. Technology can be used to

efficiently archive numerous assessments for ready browsing. Currently, some sets of

performance assessment tasks are distributed on CD-ROM, and there are plans to

place released test items on the Internet, although these resources are not yet

coordinated or easily accessible to programs and schools. Electronic networks can go

beyond storage by supporting the growth of a community of colleagues and leveraging

expertise, regardless of geographic location and institutional base. Networks can offer

templates and guidelines and support collaborative development and on-line

conversations about the alignment of tasks with standards, quality of tasks and student

work, on-line training, scoring, and standards-setting sessions. Technology can also

support simulated investigations and data collection and analysis of student responses.

The exponential advances taking place in technology promise to revolutionize

assessment practices and education reform (Kozma &Schank, 1998; Quellmalz, 1999).

TECHNOLOGY IN PROFESSIONAL

DEVELOPMENT

New professional development models are needed to provide teachers with greater

opportunity to access, discuss, incorporate, and co-construct assessment resources

and other reform-based materials(Little, 1993).Current efforts find it difficult to reach

many teachers and to maintain discourse, and teachers have difficulty implementing

new ideas back at their schools. Technology can help provide mechanisms for teachers

to overcome their isolation and make more effective use of their time spent on

professional growth.

PERFORMANCE ASSESSMENT LINKS IN

SCIENCE(PALS)

PALS provides an on-line assessment resource library designed to serve two purposes

and user groups:(1) the accountability requirements os state education agencies and

specially funded programs, and (2) the professional development needs of classroom

teachers. SRI is developing PALS under a grant from the Instructional Materials

Development (IMD) program within the National Science Foundation with two primary

goals:

(1) To develop a two-tiered on-line performance assessment resource library composed

of performance assessment tasks for elementary, middle, and secondary levels. One

tier will be a password-protected, secure Accountability Pool of science performance

assessments for use by state assessment programs and systemic reforms. The second

tier is for use by teachers and professional development organizations. The

Professional Development Pool provides performance-based science assessments that

have been used successfully in large-scale (state or national)assessment programs and

have been released.

(2) To evaluate the effectiveness of policies, implementation models, and technical

ERIC Resource Center www.eric.ed.gov

ED435708 1999-09-00 Performance Assessment Links in Science (PALS). ERIC/AE

Digest.

Page 3 of 6
quality requirements for the use of the two tiers of PALS.

In our design for PALS, experienced assessment programs can contribute

standards-based science assessment tasks with documented technical quality to the

PALS on-line library. The Accountability Pool will be composed of password-protected,

secure tasks accessible only to approved assessment program staff. Assessment

programs can thus share their resources and enjoy a much larger pool of science

performance assessments to use or adapt for their testing administrations. The PALS

resource library can provide large, continually updated collections that can support

efficient searching, selection, and printing.

The Professional Development Pool contains resources that are of documented

technical quality and have been released for access by teachers and professional

development groups. Pre-service and in-service programs, for example, can reach

teachers in geographically distributed and remote locales, resulting in great savings of

travel and materials expenses. On-line guidelines and templates can support classroom

use of science performance assessments. Teachers may administer the performance

tasks as part of their classroom assessments, adapt them, or use them as models for

developing similar investigations.

To help users design test administration forms that cover important science standards,

the on-line system provides assessment planning charts (Stiggins,1994).PALS

automatically produces an assessment planning chart to display coverage of standards

by the performance tasks selected by the user.

The science performance assessment library includes the scoring rubrics designed to

judge the quality of student responses to a task or set of tasks. To bring meaning to the

scoring rubrics, a library of exemplars of scored student work is included.

Rater training materials are not routinely published by assessment programs. SRI has

developed specifications for on-line rater training and scoring so that each agency

wishing to take advantage of the PALS system can convert traditional, stand-up training

procedures and calibration to written form, assemble training packets, and test their

effectiveness in on-line delivery.

An essential component of PALS is the documentation of the technical quality of each

task in the library. PALS contains the technical quality indicators provided for the

field-tested science tasks. Since the resource library will be stocked only with tasks that

have survived a systematic development process, the tasks and rubrics will also have

been subjected to content validity and bias reviews.

ISSUES AND EXPANSION OF THE SYSTEM

The project will be addressing a number of issues. One will be the procedures for

www.eric.ed.gov ERIC Custom Transformations Team

Page 4 of 6 ED435708 1999-09-00 Performance Assessment Links in Science (PALS). ERIC/AE

Digest.
identifying the science standards that tasks are designed to test and the nature of the

groups that will make the alignment judgments Another issue relates to policies for

accessing the resources that have been developed with funds from different sources

and that represent assessment materials being distributed by various organizations. A

third issue relates to the criteria and procedures for reviewing tasks submitted for

inclusion in the bank. Finally, the system must develop strategies for operating,

maintaining, and expanding the resources.

PALS uses technology to efficiently archive numerous assessments for ready browsing

by teachers, and goes beyond mere storage of assessments to support cross-links with

standards, tasks, scoring criteria, and annotated student work. These resources can be

shared by assessment programs, allowing them to have access to a large pool of

performance assessments to use or to adapt for their testing administrations. Teachers

can administer the tasks as part of their classroom assessments, adapt them or use

them as models for developing their own investigations, and contribute their adaptations

to the resource bank for other teachers to use.

As an on-line library alone, PALS is a valuable set of resources. However, the growing

body of professional development research and the results of our pilot study suggest

many benefits of integrating more collaboration support into PALS. SRI plans to

integrate an online meeting/discussion component to help the community members

leverage expertise, regardless of geographic location. This component could be used to

support collaborative development and on-line conversations about the alignment of

tasks with standards, the quality of the assessment tasks and student work, rater

training and scoring, and standards setting. The authors believe that, by taking

advantage of new models of professional development that include innovative digital

technologies, PALS will provide excellent professional development opportunities for

teachers.

REFERENCES

Kozma, R., & Schank, P. (1998). Connecting with the twenty-first century: Technology in

support of education reform. In D. Palumbo and C. Dede (Eds.), Association for

Supervision and Curriculum Development 1998 yearbook: Learning with technology.

Alexandria, Virginia: Association for Supervision and Curriculum Development.

Lieberman, A.(1996, November).Creating intentional learning communities. Educational

Leadership, 51-55.

Little, J.(1993).Teachers' professional development in a climate of educational reform.

Educational Evaluation and Policy Analysis, 15(2), 129-151.

Quellmalz, E. S. (1984).Designing writing assessments: Balancing fairness, utility, and

cost. Educational Evaluation and Policy Analysis, 6, 63-72.

Quellmalz, E.S. (1999).The role of technology in advancing performance standards in

ERIC Resource Center www.eric.ed.gov

ED435708 1999-09-00 Performance Assessment Links in Science (PALS). ERIC/AE

Digest.

Page 5 of 6
science and mathematics learning. In Kathleen Comfort (Ed.), Advancing standards for

science and mathematics learning: Views from the field. Washington, DC: AAAS.

Stiggins, R. J. (1994).Student-centered classroom assessment. New York: Macmillan

College Publishing Company.

This publication was prepared with funding from the Office of Educational Research and

Improvement, U.S. Department of Education, under contract ED99CO0032. The

opinions expressed in this report do not necessarily reflect the positions or policies of

OERI or the U.S. Department of Education. Permission is granted to copy and distribute

this ERIC/AE Digest.

Title: Performance Assessment Links in Science (PALS). ERIC/AE Digest.

Note: PALS Web site: http://pals.sri.com.

Document Type: Information Analyses---ERIC Information Analysis Products (IAPs)

(071); Information Analyses---ERIC Digests (Selected) in Full Text (073);

Available From: ERIC Clearinghouse on Assessment and Evaluation, 1129 Shriver,

University of Maryland, College Park, MD 20742. Tel: 800-464-3742 (Toll Free).

Descriptors: Elementary Secondary Education, Faculty Development, Information

Dissemination, Online Systems, Performance Based Assessment, Resources,

Sciences, Standards

Identifiers: ERIC Digests

###

[Return to ERIC Digest Search Page]

www.eric.ed.gov ERIC Custom Transformations Team

Page 6 of 6 ED435708 1999-09-00 Performance Assessment Links in Science (PALS). ERIC/AE

Digest.