Vol. 2 No. 4 August 1998

The NAEP 1997 Arts Education Assessment: An Overview

Abstract: The NAEP 1997 Arts Education Assessment represents the first comprehensive approach on a national level to

the development of methods for assessing student performance in dance, music, theatre, and the visual arts. This Focus

on NAEP, the first in a series of five publications, gives an

overview of that effort.

Introduction. In 1997 the National Assessment of Educational Progress (NAEP) completed the first national

arts assessment in twenty years, and the first ever to go

beyond music and the visual arts to include theatre. Assessments for music, theatre, and the visual arts and also

dance were field tested in the fourth, eighth, and twelfth

grades. The full-scale assessment covered eighth-grade

students only, and did not include dance. In addition to

including a broader range of art forms, the assessment

used a richer mix of art works for students to respond to,

and more diverse tasks, which required students to create

and perform, as well as respond. These tasks required innovative scoring methods to evaluate student artistic

achievement objectively and reliably. This Focus on

NAEP report, along with four others on the individual

arts, will give a preview of the NAEP 1997 Arts Education Report Card, to be released in the fall of 1998.

The Arts Education Report Card will contain an assessment of eighth-grade student performance in music, theatre, and the visual arts, drawing data from both public

and private schools. NAEP assessed performance in music and the visual arts using a nationally representative

sample of all students, regardless of their background in

music or the visual arts. The theatre assessment was a

targeted assessment, limited to students who would have

taken at least 30 class hours of instruction in theatre by

the end of the 1996−97 school year. The samples were

large enough to obtain statistically valid results. Because

of the difficulty of obtaining a sample large enough to be

consistent with NAEP’s standards, no assessment of

dance was conducted, although the Report Card will

contain examples of dance assessment tasks.

Background. In 1992, the Arts Education Consensus

Project, sponsored by the National Assessment Governing

Board (NAGB), began an 18-month effort to establish

objectives for assessing arts education in the nation’s

elementary and secondary schools. The National Endowment for the Arts, in collaboration with the Getty Center

for Education in the Arts, funded the Arts Education

Consensus Project, designed to develop the NAEP Arts

Education Framework. The framework, developed simultaneously with the National Standards for Arts Education, received substantial input from the general public,

along with arts educators, artists, policy makers, and

business representatives. In 1995, NAEP began fieldtesting assessment tasks based on the framework, administering the actual assessment in 1997.

The importance of the arts. The NAEP 1997 Arts Education Assessment Framework reflects its authors’ conviction that the arts are essential for every child’s complete development and education. The expectation is not

that all children will become professional artists, although

some will. What is expected is that all students will experience the joy of creating and the self-confidence that

comes from the development of skills and performance;

that they will come to know the importance of discipline,
2

practice, persistence, and self-criticism; and that they will

apply what they have experienced and learned through the

arts to other aspects of their lives. The benefits of artistic

experience and knowledge are prized not only by educators but also by parents and the business community.

Arts for everyone. The ultimate purpose of the NAEP

1997 Arts Education Assessment Framework was to obtain a comprehensive picture of what arts education is in

the United States and what it ought to be. The framework

emphasized performance, and the assessment of the arts

achievement of all students, not just those with specialized education in the arts. This was not always possible.

Not all American schools offer significant instruction in

all four arts. Some schools sponsor extensive arts education programs, providing instruction in dance and theatre,

as well as music and the visual arts. In some school systems, however, the arts are a marginal experience for students at the elementary and middle school level and an

elective subject in high school.

The Framework Matrix

The Arts Education Consensus Project created a special

matrix to guide the development of the assessment

framework, helping to ensure consistency in assessment

across the arts (see figure 1). The matrix analyzed all

four arts in terms of processes and content. Processes include (1) creating, (2) performing, and (3) responding.

Content includes (1) knowledge and (2) skills, including

perceptual, technical, expressive, and intellectual/-

reflective skills. This section sets forth the framework’s

analysis of the four art forms in terms of process and

content.

As the matrix illustrates, the nature of dance and music

allows students to be assessed both as creators and performers. In theatre, creating and performing blend into

one: even the performance of an existing play requires

students to make creative decisions involving lighting,

staging, costume, direction, and characterization. In the

visual arts, students are assessed as creators, not performers. The process of responding, on the other hand, is

applicable for all four arts. Content, the knowledge and

skills appropriate for each individual art, cuts across all

four disciplines.

Processes

Creating refers to generating original art. This includes

the expression of a student’s unique and personal ideas,

feelings, and responses by creating a visual image, a

dramatic character, a written or improvised dramatic

work, or by composing or improvising a piece of music

or a dance.

Performing means re-creating an existing work, a process that calls upon the abilities of students to reproduce

or interpret such a work. Typically, “performing” does

not apply to the visual arts, where reproducing an artist’s

existing work is not central. A student playing a transcription for solo piano of George Gershwin’s Rhapsody

in Blue is an example of performing, while an improvisation on the same piece would be an example of creating.

Responding varies from the relatively passive response

of an audience member to a work of art or performance to

the interactive response between a creator/performer and

his/her creation. Although a response is often thought of

as verbal (either oral or written), responses can also be

conveyed non-verbally or in the art forms themselves.

Such a response can be emotional, intellectual, or physical, as the work is at once understood and critiqued. Responding requires students to exercise their abilities to

analyze, reflect, make judgments, and generate new ideas.

A student writing a critique of a performance of Rhapsody in Blue is an example of responding, while a student

Figure 1.The Framework Matrix

NOTE: In theatre, creating and performing blend into one, while

in the visual arts, students are assessed as creators, not performers.

SOURCE: 1997 NAEP Arts Education Assessment Framework, National

Assessment Governing Board

Grade

 4

Knowledge and Skills

Based on Specific Content in the Arts

Disciplines

Grade

 8

Grade

 12

Dance Music Theatre Visual Arts

Creating

Performing

Responding
3

choreographing a dance to Rhapsody in Blue would be an

example of creating which shares elements of responding.

Content

There are two major components of learning expected of

students who participate in the study of the arts. Students

should gain (1) knowledge and understanding about the

arts, including the personal, historical, cultural, and social contexts for works, and (2) perceptual, technical,

expressive, and intellectual/reflective skills.

Knowledge and Understanding. When students engage

in the artistic processes of creating, performing, and responding, they draw upon various kinds of knowledge

and understanding about the arts in order to construct

meaning.

Personal knowledge is knowledge drawn from a student’s

own life and experiences. A student looking at a selfportrait by Rembrandt or Chuck Close would draw on

her own self-knowledge to consider how she might portray herself. Historical knowledge can refer to both general historical knowledge and knowledge of the history of

a specific art form. Students performing Eugene

O’Neill’s Long Day’s Journey into Night would benefit

from knowing about the Irish immigrant experience at the

turn of the 20th century and about naturalism in American literature and theatre.

Cultural knowledge includes knowledge about students’

own culture and its past, and knowledge of other cultures.

Students studying or performing in Martha Graham’s

Appalachian Spring should understand the interest in

American folklore, democratic ideals, and patriotism that

characterized the period of the Great Depression and

World War II. Social knowledge includes an understanding of social values, structures, and change. A work

like Duke Ellington’s Black, Brown, and Beige, depicting

the African American’s long struggle to escape oppression and achieve equality, demands social awareness on

the part of the listener to be understood fully.

Skills. The acquisition and application of skills determine

the quality of creating, performing and responding within

the arts. Students require perceptual skills to grasp the

significance of sensory stimuli and to discern nuance

within a given art form. For example, dancers must recognize beats and rhythms to perform correctly. Technical

skills, such as the physical dexterity developed by musicians, are needed to produce a work with quality. Expressive skills allow students to impart a unique and personal

nature to the work and create a distinctive impression on

their audience. This occurs in drama when performers

communicate personal emotion through the dialogue and

action of the play. Finally, intellectual/reflective skills

enable students to test different creative possibilities,

solve artistic problems, refine their work, and push it beyond the immediate, as a painter does when he takes an

outdoor sketch and reworks it in the studio.

Throughout the processes of creating, performing, and responding in the arts, students are called upon to apply

knowledge and skills simultaneously. Knowledge and

skills rarely function in isolation; one implies the other.

Students involved in creating, performing, and responding

gain knowledge “about” the arts, but also learn “through”

and “with” the arts. In turn, students later use such

knowledgein history, for example, or aestheticswhen

creating, performing, or responding.

Assessment Tasks

The NAEP Arts Education Assessment developed innovative tasks for the contents and processes of all four arts

NAEP and Dance. Dance assessment tasks for fourth-,

eighth-, and twelfth-grade students were developed and field

tested, but no assessment was administered in dance, because of the difficulty of obtaining a sample large enough to

be consistent with NAEP’s technical standards. The 1997

Arts Education Report Card will contain dance assessment

tasks for the eighth grade, while a separate Field Test Process Report will contain dance assessment tasks and scoring

guides for all three grades.

NOTE: For reasons of confidentiality, the students shown in this picture

are not NAEP participants.

PHOTO BY JULIE NORA
4

disciplines at the fourth, eighth, and twelfth grades. Assessment tasks were prepared by the Educational Testing

Service, under the guidance of four discipline-specific

committees, composed of arts educators and state officials. NAEP began field testing these tasks in 1995. Field

tests for fourth- and eighth-grade students were completed in 1995 for all four subjects, while the twelfthgrade field tests occurred in 1997, for all four subjects.

The twelfth-grade field tests were held to evaluate the validity of the arts tasks, even though NAEP had determined that it would not conduct an assessment for

twelfth-grade students. The arts tasks tested included

both paper-and-pencil and performance tasks.

Paper-and-pencil tasks. Special paper-and-pencil tasks

were developed to assess students’ understanding of the

arts. For example, the field test of the dance tasks used

videotapes of actual dances as cues, or “stimuli,” for

questions. The field test for music used a special recording of a violin performance that included deliberate errors

to assess students’ ability to detect “right” and “wrong”

ways of playing. In the field test for theatre, students

were shown a scene from a film, first without and then

with the soundtrack, and were asked to discuss the extent

to which the non-verbal behavior of the actors conveyed

the meaning of the scene. In the field test of the visual

arts, students were shown four types of cups, ranging

from plastic to fine china, and were asked questions that

considered the cups from both aesthetic and functional

points of view.

The paper-and-pencil tasks included multiple-choice,

short constructed-response, and extended constructedresponse questions, which required students to write one

or more paragraphs in response to the stimuli. These

tasks required sophisticated scoring guides that could discriminate between, and give credit for, the full range of

possible student answers.

Performance tasks. The performance tasks proved to be

especially complex. Unlike other NAEP assessments,

video and audio recording devices were used to record the

tasks. In the dance field test, professional dancers demonstrated dances to students in person after it was found

that students had difficulty using pre-recorded videos to

learn the dance routines. In the music field test, students

sang, played their own instruments, and, in some cases,

used electronic keyboards. In the visual arts performance

tasks, students worked with Plasticine (a clay-like subNAEP and Theatre. The NAEP 1997 Theatre Assessment

of eighth-grade students is the first national assessment of

student achievement in theatre. The assessment was limited to students who had taken in-school theatre classes.

Results will appear in the NAEP 1997 Arts Education Report Card. NAEP also developed and tested assessment

tasks and scoring guides for fourth- and twelfth-grade students. Materials for all three grades will appear in the Field

Test Process Report.

NOTE: For reasons of confidentiality, the students shown in this picture are not NAEP participants.

Photo of 1990 production of “The Children’s Hour” courtesy Living Library Theatre, Inc., Santa Monica, California

NAEP and Music. The NAEP 1997 Music Assessment of

eighth-grade students is the first national music assessment

since 1978. Students were assessed regardless of whether

they had studied music. Results will appear in the NAEP

1997 Arts Education Report Card. NAEP also developed

and tested assessment tasks and scoring guides for fourthand twelfth-grade students. Materials for all three grades will

appear in the Field Test Process Report.

NOTE: For reasons of confidentiality, the student shown in this picture

is not a NAEP participant.

PHOTO BY CLAIRE FLANDERS, courtesy Levine School of Music,

Washington, DC
5

stance), charcoal, drawing pencils, and numerous construction materials. Scoring guides had to be developed to

provide consistent and accurate assessment of the full

range of student achievement on these tasks. NAEP plans

to make available to the public all the tasks not covered

by copyright.

Assessment Sample

The assessments for music and the visual arts used nationally representative samples of the general student

population of eighth graders, while the theatre assessment

was targeted to eighth graders who had taken in-school

theatre classes. NAEP used a targeted assessment for

theatre both to assess the students fairly and obtain results that resembled the breadth of student achievement

allowed for in the theatre scoring guides. The field tests in

theatre for the fourth and eighth grades, which sampled

the general student population, found that it was difficult

to obtain the full range of performance from the students

sampled. Because of the limited number of schools in the

country offering dance programs, NAEP did not conduct

an assessment for dance.

The music and theatre assessments used samples of about

2,000 students each, while the visual arts sample was

about 2,500. These samples were large enough to support

statistically reliable analyses.

NAEP Arts Education Reports

The 1997 Arts Education Report Card will be published in the fall of 1998 and will give data on eighthgrade student achievement in music, theatre, and the visual arts. It will also give examples of assessment tasks

from the eighth-grade dance field test. In addition, the

Report Card will provide information from a variety of

background questionnaires. Students who participated in

the three assessmentsmusic, theatre, and the visual

artsfilled out questionnaires. School background questionnaires were administered for all three subjects. A

teacher background questionnaire was administered for

theatre only, because the theatre assessment was targeted.

This ensured that the students who took the assessment

had taken classes from the teachers responding to the

questionnaire. Information from these questionnaires can

be used to identify possible relationships between student

background and student achievement or between school

or classroom practices and student achievement.

The Field Test Process Report will be published in the

spring of 1999 and will provide extensive information on

the development, administration, and scoring of assessment tasks for the four arts disciplines at the fourth-,

eighth-, and twelfth-grade levels. Schools, districts, and

states can use the process report to help develop or refine

their own arts assessment programs at all three grade levels. In addition, arts educators may want to adapt specific

assessment tasks and scoring guides for use in their own

classes.

Four Focus on NAEP publications, on dance, music,

theatre, and the visual arts, are currently available. Each

provides additional background and detail on NAEP activities in the particular art covered by the Focus on

NAEP.

Conclusion

The NAEP Arts Education Assessment is the first comprehensive approach to developing an assessment of all

four arts on a national level. The assessment developed

paper-and-pencil and performance tasks for all four arts

disciplines, based on a common understanding of arts

processes and content; it created tasks suitable for a variety of students with a variety of educational backgrounds;

NAEP and the Visual Arts. The NAEP 1997 Visual Arts Assessment of eighth-grade students is the first national visual

arts assessment since 1978. Students were assessed regardless of whether they had studied the visual arts. Results

will appear in the NAEP 1997 Arts Education Report Card.

NAEP also developed and tested assessment tasks and

scoring guides for fourth- and twelfth-grade students. These

materials will appear in the Field Test Process Report

NOTE: For reasons of confidentiality, the students shown in this picture

are not NAEP participants.

PHOTO BY UNIPHOTO
6

and it created effective scoring guides that could recognize and give credit for the full range of student ability.

The findings provide performance data for the general

population of eighth-grade students in music and the visual arts, as well as performance data for a targeted

population of eighth-grade students in theatre.

The authors of the Arts Education Framework argued

that “No child in an American school should be deprived

of the opportunity to see, hear, touch, and understand the

accumulated wisdom of our artistic heritage, and to make

one’s own contributions through productions and performances.” The NAEP Arts Education Assessment will

provide significant information toward determining how

close American schools are to achieving that goal.

For Further Information

Arts Education Assessment Framework for the 1997

National Assessment of Educational Progress. Single

copies are available free from the National Assessment

Governing Board, 800 North Capitol Street, NW, Suite

825, Washington, DC 20002–4233. Copies may also be

obtained at http://www.nagb.org/pub.html.

National Standards for Arts Education. Copies are

available for $20 from the Music Educators National

Conference, 1806 Robert Fulton Drive, Reston VA

20191, 800−336−3768.

Arts Education in Public Elementary and Secondary

Schools, NCES 95−082, provides data obtained from

questionnaires sent to the principals of nationally representative samples of public schools. It is not in print, but

is available over the NAEP web site at

http://nces.ed.gov/NAEP/.

The following Focus on NAEP publications are also

available free from the National Center for Education

Statistics, U.S. Department of Education, Washington,

DC 20208−5653. Copies may also be accessed over the

World Wide Web at http://nces.ed.gov/NAEP/.

NAEP and Dance: Framework, Field Test, and Survey,

NCES 98−459

NAEP and Music: Framework, Field Test, and Assessment, NCES 98−529

NAEP and Theatre: Framework, Field Test, and Assessment, NCES 98−528

NAEP and the Visual Arts: Framework, Field Test, and

Assessment, NCES 98−526

The Focus on NAEP series briefly summarizes information about the ongoing development and implementation

of the National Assessment of Educational Progress

(NAEP). This series is a product of the National Center

for Education Statistics (NCES), Pascal D. Forgione, Jr.,

Commissioner, Gary W. Phillips, Associate Commissioner for Education Assessment. This issue was written

by Sheida White of NCES and by Alan Vanneman of

the Education Statistics Services Institute. For more information, contact Sheida White at 202–219–1675. To

order other NAEP publications, call Bob Clemons at

202–219–1690, or e-mail bob_clemons@ed.gov.

Information on NAEP may also be obtained over the

World Wide Web at http://nces.ed.gov/naep/.