Taking Another Look: Secondary Analysis of the NAEP Report Card in the Visual Arts

In 1998, the National Center for Educational Statistics (NCES) released the long-awaited National Assessment of Educational Progress (NAEP) 1997 Arts Report Card. The Arts Report Card presents aspects of the most recent nationwide study of arts learning. The art education community was involved in this assessment project throughout its development and is now charged with expanding our understanding of the raw data and government reports. The National Art Education Association, through general sessions at the 1999 National Conference in Washington and within the Task Force and Research Commission structure, funded travel for a team of arts researchers to attend a government-sponsored workshop for advanced statistical analysis of NAEP data. The team consisted of the authors of this report. The Arts Report Card Arts Report Card information is available in several formats: as a softcover book, on CD, and within a NCES website. Of particular interest to art educators are the Executive Summary, Chapter Four on Creating and Responding in the Visual Arts, Chapter Six on Results of Selected Subgroups, and Chapter Seven on Contexts for Arts Education. The findings are based on student performances for the Creating and Responding blocks and exercises which made up the test battery. A nationally representative random sample of 2,999 eighth-graders participated in the visual arts assessment. The sample includes eighth-graders who have had little or no art exposure, those who have had art [instruction] recently or frequently, and those who have been experiencing artistic involvement in a variety of settings. Initial reading of the NAEP Report Card was disappointing for some. Graphic presentations and technical narratives characterized student achievement, on the whole, as rather average. Scores fell short of stated NAEP framework goals and art educators' expectations (Eisner, 1999; Stankiewicz, 1999). In 1997, as in the 1974-75 and 1978-79 assessments (U.S. Department of Education, 1998), both racial and ethnic indicators and parental education level related significantly to achievement data. For the 1997 data, subgroup reporting also indicated that eighth grade females outperformed male peers. Though specific assessment results cannot be compared across test administrations because of changes in the 1997 assessment battery1, children's participation in the arts and depth of understanding continue to be associated with privilege (Eisner, 1999; Stankiewicz, 1999). All, however, is not grim. Despite rating the assessment poorly, Stankiewicz (1999) ranked the NAEP potential to influence instruction as an "A" and asked if we can "spin the NAEP results into the tapestry of art education in a way that gives us needed information but also helps us represent art education to the public with integrity" (p. 30). Characterizing Students' Experiences Profiles of student's visual arts experiences (U.S. Department of Education, 1998) appear in the narrative section of the NAEP arts summary report. The term "significant" in the report does not infer educational relevance; rather, it identifies population differences "to help focus subsequent discussion among policy makers, educators, and the public" (p. 8). For example, the spread of points between students in the upper and lower ranges of the scale reveals interesting patterns which are quoted below in (1) and (2) to give a sense of the document's statistical presentation and summarized in (3) through (11): (1) "In-school activity is positively associated with student performance... [ with] a significantly larger percentage of students in the upper level of the Responding scale than in the lower level of the scale" taking art in school and, in another comparison, with participating in drawing or painting at least once each week. Further, students taking art in school and painting or drawing outperformed, on the Creating scale, their peers without similar experiences (p. …