**REFLECTIONS ON STATE EFFORTS TO IMPROVE MATHEMATICS AND SCIENCE EDUCATION IN LIGHT OF FINDINGS FROM TIMSS**
Andrew A. Zucker, Ed.D.
Program Manager, Mathematics and Science Education
With assistance from:- Patrick Shields
- Nancy Adelman
- Daniel Humphrey

April 1997
SRI Project 1817
This paper was commissioned by the National Education Goals Panel (PurchaseOrder #4331FJ700008) for release at its meeting on July 30, 1997.  The opinionsand recommendations expressed in this paper are those of the author and do notnecessarily reflect those of the Goals Panel or its members.
INTRODUCTION
In recent years, many states have articulated clearer, more ambitious standards forwhat students should know and be able to do as they progress through school. However, results of the Third International Mathematics and Science Study(TIMSS), as well as results from periodic administrations of the NationalAssessment of Educational Progress (NAEP), demonstrate the gap between currentstudent achievement in the United States and the higher academic standards thatstates are aiming for.  The National Education Goals Panel (NEGP) would like tosuggest methods for implementing rigorous state standards effectively so that,over time, they will help raise student achievement.  The Goals Panel has thereforeasked SRI International1 to describe how states have been implementing theirstandards.  How are states using standards as they select and fund textbooks orcurricula, provide support for teachers and instruction (including states' policiesrelating to pre- and in-service teacher professional development), and conductassessments of students' learning?  The purpose of the paper is to illustrateselected state efforts in these areas, discuss what we know about how statescurrently support or implement their state standards, and recommend ways tomake those processes more effective.
In preparing this paper, SRI International used an extensive data set collected forprior and ongoing work.  SRI is currently completing the last of 5 years evaluatingthe National Science Foundation's (NSF's) Statewide Systemic Initiatives (SSI)Program.2  That program has supported 25 states and Puerto Rico in ambitiousefforts to improve mathematics and science education in elementary and secondaryschools.  Research conducted for the evaluation has focused on efforts in all theStatewide Systemic Initiatives (SSIs).  However, a particularly intensivedata-gathering effort has taken place in a dozen case study states.  SRI haspublished case studies describing mathematics and science education improvementefforts in three of those states (Connecticut, Delaware, and Montana_see Zuckerand Shields, 1995), and plans to publish case studies about the other nine(Arkansas, California, Kentucky, Louisiana, Maine, Michigan, New York, Vermont,and Virginia) later this year.  Much of the data used to prepare this paper comesfrom the SSI evaluation.  
Other sources of information include SRI's current evaluation of the Dwight D.Eisenhower Mathematics and Science Education State Curriculum FrameworksProjects (a program that has supported 16 states to develop and implementcurriculum frameworks in mathematics, science, or both) and the evaluationconducted last year by SRI of the American Association for the Advancement ofScience's Project 2061 (the first U.S. effort to develop national science educationstandards).  A variety of secondary data sources are also referenced.
TIMSS FINDINGS:  SOME KEY CLAIMS
TIMSS shows (as NAEP does) that American students are performing at levelsbelow those that most people believe are appropriate or those that correspond tothe higher academic standards established in recent years by many states.  TIMSSnot only compared student achievement in mathematics and science at three gradelevels in 50 educational systems around the world; the study also examined thegoals and many characteristics of classroom practices in those education systems. The TIMSS findings are provocative and have stimulated a good deal of discussionabout efforts to raise student achievement.
Some key TIMSS claims about American education can be found in publicationscalled A Splintered Vision (Schmidt et al., 1997), Pursuing Excellence (Peak,1996), and Characterizing Pedagogical Flow (Schmidt, 1996).  Several findings thatare especially pertinent to this paper are identified below.  
Curriculum
Science and math curricula (and our textbooks) include too many topics. That is,the U.S. has a curriculum that is "a mile wide and an inch deep."
Findings for some U.S. subjects/courses differ from the general findings.  Inparticular, U.S. Algebra I classes are far more focused than other 7th- and8th-grade classes.
There are also some important differences between mathematics and science.  Asan example, although the core of mathematics topics is broad, it varies little amongstates.  On the other hand, the number of core science topics is much smaller, andthe overlap among state curricula is also small.  (Also, although 8th-gradeAmerican students achieve scores slightly above the international average inscience, they are below average in mathematics.)
The U.S. science and mathematics curricula (including textbooks) are lesschallenging than in some other nations.  For example, curricula in France, Spainand, to some extent, Japan reportedly pose more varied demands on students thando the U.S. curricula.
Instruction
The way that American teachers organize lessons may be flawed.   Americanteachers act primarily as "information transmitters."  Moreover, typical teachers inthe U.S. use far more activities in a single lesson than seems appropriate.
Standards-based reform
Fundamental changes in various components of the U.S. education system may beneeded to improve student achievement.  The TIMSS reports suggest that changesare needed in teacher preparation, working conditions in schools, the quality ofcurricula and textbooks, and other aspects of American education.
Some comments on these claims
Virtually all the major findings are consistent with or expand on findings from otherresearch, and this consistency adds to the credibility of TIMSS.  For example, SRIconvened a panel of experts to review selected science texts as part of anevaluation of the American Association for the Advancement of Science's Project2061.  Findings from that review are consistent with the TIMSS findings.  Inparticular, the SRI review of two popular high school biology textbooks published in1995 and 1996 found that the glossaries in each contain about 1,000 technicalterms (which are also used in the body of the books) and that the authors "havenot eliminated material extraneous to what is required to meaningfully convey keyconcepts" (Zucker, Young, and Luczak, 1996).
Thus, the TIMSS data, like the SRI research just reported, support some keyprinciples advocated in American national mathematics and science educationstandards documents.  For example, one important tenet of Benchmarks forScience Literacy (written by Project 2061 staff) is that it would be good practice inteaching science to spend more time on fewer important topics.  This approach issometimes referred to as the principle that "less is more" (although a bettersummary might be "do more of less").  That position is supported by TIMSS datashowing that U.S. science and mathematics curricula try to cover a great manytopics but (when compared with curricula in many other nations) sacrifice intensityof coverage, and deeper understanding, to do so.  
Similarly, the National Council of Teachers of Mathematics (NCTM) standards focusattention on the importance of teaching mathematics for understanding, not just sothat students commit key facts and procedures to memory.  The science educationstandards also focus on the importance of conceptual understanding.  This principleis supported by TIMSS data showing that typical U.S. lessons are not as focusedas lessons in many other countries, concentrate on less challenging material, andutilize the teacher primarily as a transmitter of information. 
Currently, however, TIMSS data do have some significant limitations.  For example,TIMSS has not yet released information about high school students.  The absenceof current information from TIMSS about high schools should not lead us to ignoresome of the unique problems at that level.  (For example, one of these problems isthat the majority of high school science teachers are specialists who seem quitesatisfied with their current textbooks, despite the fact that many of the books areladen with an excess of specialized terms [for supporting data, see Weiss, 1994].) This is a topic to which we will return.
The TIMSS reports published to date typically focus on the U.S. as a whole, notindividual states.  This national portrait obscures variations across the states, bothin current practices and in efforts under way to effect change.  We know thatsome of the differences between state education systems are large.3  It is useful toconsider which states have adopted distinctly different approaches to curriculum,instructional materials, support for teachers, or other components of theireducation system.
LESSONS FROM STATE EDUCATION IMPROVEMENT EFFORTS
In this section, examples of distinctive and promising approaches to mathematicsand science education among the states are identified.  In addition, there is adiscussion of the broader lessons learned from SRI's evaluation of NSF's SSIprogram concerning different strategies that states use to support (or "implement")standards-based education reform.
Curricula and textbooks
Instructional materials are basic to what happens in schools.  The TIMSS datachallenge us to consider whether typical American textbooks are likely to beeffective in the hands of typical teachers.  In fact, SRI found that in many of theSSI states instructional materials are still a "weak link," especially in certaindomains (e.g., high school science).  Only six of the SSIs focus on the instructionalmaterials used in mathematics and science classrooms as a major part of theirchange strategy.
To improve student achievement, high-quality textbooks (or other instructionalmaterials, such as kit-based elementary science programs) need to be identified (ordeveloped, if necessary), and decision-makers need to be well informed aboutthem.  Whether decisions are made at the state or the local level, we believe itwould make sense for more states to adopt and/or disseminate information aboutexisting high-quality curriculum materials.  In a few cases, the SSIs have engaged in ambitious efforts to create newinstructional materials.  One successful example can be found in Montana. 
Montana's SIMMS Project:  High-quality materials development.Montana's Systemic Initiative for Montana Mathematics and Science (SIMMS) isproducing six 1-year-long, integrated mathematics courses for grades 9-12.  Thematerials are innovative (e.g., they assume that students have access to powerfulcomputers and graphing calculators throughout, and they focus extensively onapplications of mathematics in real-world settings), they have been highly rated byat least two sets of external reviewers, and both the utility and sustainability of theeffort are reflected in the fact that the materials are being distributed nationwide bya commercial publisher.  Note that schools in Montana, a local-control state, arenot required to use the new SIMMS curriculum, but many do.  The legislature hasprovided millions of dollars to fund technology for SIMMS students and teachersthroughout the state.
Montana is not the only state to focus on using the curriculum to implement a new"vision" for mathematics and science education.  For example, many years agoNew York developed an integrated high school mathematics curriculum forcollege-bound students, eliminating separate courses for algebra, geometry, andother strands of mathematics, and it is still used.  New York is also one of the fewstates to focus on technology (in the sense of engineering) as a topic that allstudents must study in middle schools.  More recently, California developed acourse called Math A as a richer option for 9th-grade students than general math,giving them an option to enroll in Algebra 1 in 10th grade.  The Puerto Rico SSI hasbeen developing a series of year-long curricula in science.  Although suchstate-specific courses or curricula are more the exception than the rule, they dodemonstrate that it is possible to innovate on a large scale and to use thecurriculum as one way of implementing a state's vision of what mathematics orscience education should be.4
Other SSIs, such as New Mexico's, have promoted existing high-qualityinstructional materials.  Full Option Science System (FOSS) is an example of oneset of elementary science materials promoted in New Mexico and in a number ofother states.  (FOSS, distributed by Encyclopaedia Britannica, was developed withNSF funds.)  New Mexico is a textbook adoption state, but it is probably notnecessary that a state be one for state education agency staff to recommendspecific curriculum materials to schools and districts.
Over the past decade, the National Science Foundation, a few private foundations,and some commercial publishers have invested many millions of dollars in thedevelopment of various curriculum alternatives in mathematics and science.5 Increasingly, it should be possible for states, districts, and schools to choose anduse instructional materials that differ from the mainstream in important ways(if theywish to do so.  Ideally, professional development for teachers should be part of theeffort to change curriculum and instruction, because new curricula typically requirethat teachers learn a great deal (both new mathematics or science content and newteaching techniques, such as using technology) before they can use theinstructional materials effectively.
Teaching and professional development
The TIMSS data show not only that American curricula need to be reexamined butthat typical patterns of instruction should change to be better aligned withstandards of good practice.  TIMSS suggests that professional development forteachers is a necessary, but not sufficient, priority if current U.S. studentachievement patterns in mathematics and science education are to be substantiallyimproved. 
SRI found that professional development is the most common SSI strategy (a highpriority in 18 of 25 SSIs), and in our view the majority of the SSIs using thisstrategy can be rated as strong or very strong in the way they conduct professionaldevelopment.  Delivering high-quality professional development is something thatwe as a nation know how to do_at least for those teachers who volunteer andwhen there are sufficient funds.  
The SSIs used three general approaches to professional development:  local humanresource development, local system capacity building, and state system capacitybuilding.  The SSIs typically used a mix of these approaches, and at least nineemployed all three approaches.  California provides an example of a very strongprofessional development strategy.
California's teacher networks.California has established two teacher networks:  the California ScienceImplementation Network for elementary science, and Mathematics Renaissance formiddle school mathematics.  Both networks were created before the state appliedfor an SSI award, and both are aimed at putting into classrooms the kind ofmathematics and science curriculum and instruction envisioned in the Californiacurriculum frameworks.  The quality of the professional development is high; theprograms are open to all teachers in the state; and tens of thousands of teachershave been served.  Funds have been secured to maintain the programs at somelevel after SSI funding ends, and there is clear evidence of positive impacts onclassroom practices.
Nonetheless, in spite of the strong efforts made by most of the SSIs, SRI foundthat the professional development system, such as it is--including state, district,and school policies related to professional development--is in need of restructuringin most states.  Only a few SSI states are making efforts of this kind.
The Michigan Statewide Systemic Initiative:  An example of an effective strategyfor creating an infrastructure for building teachers' capacity.Michigan's SSI has focused on improving the system through which high-qualityprofessional development is available to all local school districts.  The MSSI hasconducted professional development for providers of professional development,developed guidelines and taken action to restructure the state's system ofprofessional development, and helped revise professional development standards.
Massachusetts is trying to change its professional development system.  A newlaw in Massachusetts (not directly tied to the SSI) has radically altered itsrecertification policies by requiring all teachers to write individual professionaldevelopment plans.  The state also specified the types of professional developmentactivities teachers need to participate in to receive credits toward recertification. (The jury is still out in terms of how much impact these changes will have onteachers and teaching.)
These two examples are the exceptions.  As long as the professional developmentsystems in the states remain unchanged, policy-makers will face difficult trade-offsbetween using available funds to work with large numbers of teachers superficiallyor with small numbers (usually volunteers) intensively.  The SSI evaluation teambelieves that professional development should be better embedded in the work lifeof all teachers, as part of their job.  A variety of new technologies (e.g., interactiveCD-ROMs, the World Wide Web) may make this integration more feasible than inthe past, but it is still necessary to find time and money within the system tosupport professional development for teachers and administrators.
Assessment systems
The TIMSS reports (like many earlier ones) suggest that other components of theeducation system need to change, besides curriculum and instruction.  Testing andassessment systems are a component of particular interest to the Goals Panel andto the National Governors' Association. 
One basic premise in designing a system to assess students' learning is that thereshould be a good match between what is being tested and the important goalsestablished for what students should know and be able to do.  (In other words,performance standards established for state tests should be aligned with the goalsfor students that are typically expressed in state standards or curriculumframeworks.)  There are numerous examples of states making significant changesin state tests or assessments to reflect changes in their goals for student learning,some of which began many years ago.
Changing the assessment system in New York.About a decade ago, predating the SSI by many years, the state of New Yorkbegan using a performance-based assessment of elementary science at the fourthgrade.  There is evidence that this approach increased attention to the teaching ofscience in elementary schools, including the use of hands-on science instruction ofthe type promoted in the national science education standards.  New York is alsoproposing that all students, not just those who are college bound, take the Regentsexaminations.  Those examinations are now being revised to better reflect thestate's new curriculum standards.
In spite of examples like this, many state assessment programs still do not includescience, which has not been considered a "basic" subject.  Moreover, in manystates there is a mismatch between the goals established for students in statestandards or curriculum frameworks and the content of state-mandated tests.  In1994, when SRI studied the assessment systems in the 25 SSIs, we were able tofind only 6 state assessment programs in mathematics and 4 in science that metour criteria for suitability in evaluating the progress of the SSIs.  Although anumber of states have improved the alignment of their assessments and standardssince that time, it is clear that much work remains to be done to improve stateassessment systems.  The commitment of Governors and business leaders a yearago to develop appropriate assessments provides a boost to these efforts (NationalGovernors' Association, 1996).
Changing assessment systems is difficult work.  In California, an innovative statetesting system (the California Learning Assessment System, or CLAS) wasabandoned under political pressure.  One underlying reason appears to be that thestandards the state established for mathematics and science education reform werenot widely enough supported to sustain the assessment system (which was alsoexpensive).  There are important lessons here about mobilizing public support forreform to which we will return.
Other strategies for supporting standards-based school improvement
As part of the SSI evaluation, SRI identified eight strategies that account for thegreat majority of the activities supported by the SSIs (see Exhibit 1).  These can beclustered into two groups, loosely corresponding to what are known as "bottomup" and "top down" strategies for school improvement (Zucker, Shields, Adelman,and Powell, 1995; Zucker and Shields, 1997).6  Several of the strategies_includingsupport for teacher professional development; developing, disseminating, oradopting instructional materials; and aligning state policy_have already beendiscussed because they correspond to topics (such as standards, curricula, andtextbooks) emphasized in TIMSS publications. 
 Exhibit 1                     EIGHT STRATEGIES USED BY THE SSIs
Strategies focused on teachers, classrooms, and schools:
1. Supporting teacher professional development
2. Developing, disseminating, or adopting instructional materials
3. Supporting model schools
Strategies focused on districts, regions, and states:
4.   Aligning state policy
5.   Creating an infrastructure for capacity building
6.   Funding local systemic initiatives
7.   Reforming higher education and the preparation of teachers
8.   Mobilizing public and professional opinion
The most common strategy the SSIs have used to support standards-based reform,as noted above, has been to provide teachers with additional professionaldevelopment opportunities, often during the summer.  In addition to professionaldevelopment, two other common strategies used by the SSIs focus resources onactivities at the school and classroom level ("bottom up" change strategies): developing, disseminating, or adopting instructional materials (see the Montanaexample above); and supporting model schools.
In addition, five strategies used by the SSIs aim primarily at effecting change indistricts or at the state level ("top down" change strategies).  These five are: aligning state policies (which includes developing or revising standards, curriculumframeworks, and state assessment systems); creating an infrastructure for buildingthe capacity of local schools; funding local systemic initiatives; reforming highereducation and the preparation of teachers; and mobilizing public and professionalopinion in support of school improvement efforts.  
Examples of effective use of four of the strategies have already been provided inthis paper (supporting teacher professional development in California; developing,disseminating, or adopting instructional materials in Montana; creating aninfrastructure for capacity building in Michigan; and aligning state policies in NewYork and Massachusetts).  Below are examples of effective uses of the remainingfour.
Supporting model schools.
Just five SSIs use a model schools strategy.  Members of the SSI evaluation teambelieve that this is a high-risk strategy.  It requires a very careful plan fordissemination and scaling up.  If such a plan is not well designed and wellimplemented, the result is, at best, improvement in a handful of schools statewide. In the states that we believe have strong model schools strategies, that effortserves as part of a much broader reform strategy.
Puerto Rico:  Successful model schools.  The Puerto Rico SSI established model sites in each of the seven education regionsof the island.  The sites were test beds for Puerto Rico's ambitious curriculumdevelopment efforts and for site-based management ("school empowerment").  Inaddition, they became dissemination centers in the successful process of scaling upto more schools.  These efforts are all closely tied to education reform legislationthat predates the SSI.
Funding local systemic initiatives.
Nine SSIs, mostly in local-control states, chose to support local systemic initiatives. The key factors in building strong local initiatives are sufficient support forparticipants at the local level and some kind of quality control mechanism.  Anexample of a successful use of this strategy can be found in Maine.
Beacon Sites:  Strong local initiatives. The Beacon Sites in Maine (some of which are schools and some districts) havebeen a core element of the SSI.  By keeping the number small (seven or eight), itwas possible to provide substantial support to each.  Support for full-time math andscience specialists at each site has helped assure the effectiveness of the strategy. The specialists have served as classroom-level coaches and have acquired,reviewed, and established libraries of curricula and instructional materials.  Asplanned, the Beacon Sites have also been an important component of scaling upreform, gradually establishing reputations as regional centers for learning abouthigh-quality mathematics and science instructional materials and as places thatprovide models of effective classroom practices.
Reforming higher education. 
Although 14 of the SSIs have adopted this strategy, none of the efforts can berated as very strong for reform of teacher preparation programs.  Changing teacherpreparation has been a challenge, in part because of the difficulty of changinginstitutions of higher education and in part because of the fact that relatively fewof the SSI resources have been used for this purpose.  One fairly strong effort isfound in Arkansas.
The Arkansas "crusades":  Involving higher education.  All of Arkansas' 14 institutions of higher education are involved in the math andscience "crusades" that are the centerpiece of its SSI.  Although progress has notbeen uniform in all 14, there have been changes in liberal arts courses(mathematics and science) in both public and private colleges and universities.
Mobilizing public and professional opinion.
Many surveys and studies provide evidence that public and professional opinion iscritically important for education reform.  A total of 10 SSIs place a significantemphasis on mobilizing opinion.  One that is worth noting is in Louisiana.  
Louisiana:  Mobilizing public opinion in local communities.  One member of the Louisiana SSI staff devotes all his time to helping local districtsinform their communities about reforms supported by the SSI.  On the basis of astrong theory of change that involves altering the generally low opinion ofeducation held by the public in Louisiana, the SSI responds to local media requestsfor information and provides custom audio, video, and written information abouthow the SSI is improving math and science education in the state.  The SSI hashelped local entities to develop high-quality presentations that are reported to havehad a positive impact in their communities.
Each of the eight strategies shows certain strengths and weaknesses (Zucker andShields, 1997).  For example, several of the strategies (especially aligning statepolicy, reforming teacher preparation, and mobilizing public and professionalopinion) typically require political, technical, and financial resources beyond thereach of a 5-year initiative funded mainly with "soft" money.  One result is thatfew SSIs have invested heavily in the reform of teacher preparation, and progressin this area has been limited.
A key point is that the SSIs use several strategies in combination (typically aboutfour of them), basing their selection of strategies to use on the state context (e.g.,whether the state has a strong tradition of local control of schools or morecentralized control), the perceived needs in the state, the strengths of the staff,and other factors.  Good strategic thinking seems to be an essential component ofstandards-based education reform, and knowledge about how to think and actstrategically in the context of state education systems has grown very considerablywithin the past decade.
It is interesting to consider two governance issues with respect to the SSIs.  First,in many of the SSIs, institutions outside of state government have taken onleadership roles.  For example, in Connecticut, the Connecticut Academy forEducation in Science, Mathematics and Technology was formed specifically to leadthe SSI, and similar organizations were formed in Vermont and Maine.  Theinvolvement of strong institutions outside of government can be a considerableasset (e.g., by providing a leverage point that is insulated from state politics andstate agency hiring restrictions or that may be able to work more readily withbusiness groups), but what arrangement is best varies by state. 
On the other hand, with limited exceptions (such as in Arkansas), few schoolsuperintendents, school board members, or other district representatives have beenactively involved with the SSIs.  Since many key policy decisions are made at thedistrict level, this lack of involvement can lead to problems.
To sum up, virtually all the states receiving support from the SSI program have putinto place new or improved standards for what students should know and be ableto do in mathematics and science.  Once that has been done, "implementing thestandards" in a state means designing and effectively carrying out a coherent andstrategic set of activities that support school improvement efforts based on thestate's standards and its traditions.  There is no formula that describes the stepsthat are needed, since no one knows exactly what they are.  The strategies inExhibit 1 identify the varied types of activities that are being used to support stateeducation standards.
REFELECTIONS AND CONCLUSIONS
TIMSS is a massive study; five books are already in press to report on TIMSSfindings, and more are scheduled.  Similarly, the findings from SRI's evaluation ofthe SSI program, including the state case studies, will fill hundreds of pages.  Manyother excellent sources of information about standards-based education reform canbe found (and some are cited here).  Distilling all this information intorecommendations for the Goals Panel would be a daunting task for anyone. Therefore, the title of this paper, as well as of this section, refers to "reflections"about standards-based reform.  These reflections, which are based on research, areintended to stimulate debate and conversation, but certainly do not represent allthe pertinent studies or all points of view.  
The section begins with a discussion of the overall support that TIMSS provides forcontinuing efforts in the U.S. to carry out standards-based education reform.  Thenext two sections discuss components of the education system about whichTIMSS has a great deal to say, namely, curriculum (including textbooks) andinstruction.  Finally, the concluding section discusses a topic that undergirds allefforts at standards-based reform:  mobilizing public and professional support forchanges in the education system.
The need for standards-based reform
By documenting deficiencies in several components of the American educationsystem, TIMSS provides additional reasons for believing that it is important to carryout standards-based reform.  Those who are not satisfied with current levels ofstudent achievement in the U.S. can (again) find evidence that improving thesituation is likely to entail, at the least, a clarification of goals for students'learning, revisions in curricula and textbooks, and better instruction by teachers. By implication, even though TIMSS does not present evidence about stateassessment systems, preservice teacher preparation programs, teacher certificationand recertification requirements, state education infrastructures (such as regionalcenters and teacher networks), or many other features of education systems, onecan conclude that these components, too, should be aligned with goals forstudents' learning.  This is an ambitious agenda.
Research conducted by SRI, and by other groups, suggests that both "top down"and "bottom up" strategies are needed to bring about the kinds of improvementsthat are desired.7  Creative and strategic thinking is necessary to design and carryout a series of linked strategies that support changes in state education systems.
The starting point for standards-based reform in many states is creating a set ofambitious standards for what students should know and be able to do.  Although itis clear that most states have been actively working on their standards, it shouldnot yet be taken for granted that the standards are uniformly of high quality acrossthe states.  Research conducted by SRI and others shows that in some states thestandards are clear and readable, but in others they lack internal coherence, arepoorly formatted, are susceptible to misinterpretation, or are otherwise of lowerquality.  (To monitor the quality of state standards, the Goals Panel has supportedcreation of an independent entity to review them.)
Just as there are variations across nations and states in the nature and the qualityof goals set for students' learning, there are important variations in curriculum,teacher professional development, assessment, and other components of theeducation system.  One can_and should_learn from these differences across thestates, just as one can learn from different nations' practices.  
However, we know that a national education system can best be understood as aconstellation of different features embedded in a particular culture, not just as alogical outgrowth of some particular feature (such as the nature of the textbooks inuse).  Thus, at the same time that we have much to learn from the TIMSS data, wemust be thoughtful in reaching conclusions about what features of other educationsystems we wish to adopt and how those features would work in an Americancontext.  Similarly, state practices don't always travel well.  Some practices aremore particular to local-control states, or small states, or textbook adoption states,or states with many low-enrollment high schools (where teachers more often mustbe generalists), and so on.  Again, this fact underscores the importance of goodstrategic thinking, based on the state context.
What is common across the states is widespread public support for ambitiousstandards and good instruction.  Yet it cannot be taken for granted that educatorsand the public will support specific changes in the schools that state policy-makerssupport.  We will return to this topic in the concluding section of the paper.
Reviewing textbooks and curricula
The TIMSS publications cited at the beginning of this paper heavily emphasizeissues related to mathematics and science textbooks and curriculum.  Althoughmany efforts are under way to change and improve textbooks and otherinstructional materials, the texts used in most classrooms are as described inTIMSS publications.
Some people believe that districts, schools, or even individual teachers should belargely responsible for creating their own instructional materials.  The fact is thatcreating a high-quality curriculum is difficult, specialized work.  Some teachershave a talent for such work, but most do not_nor do they necessarily have aninterest in writing curriculum, or the time to do so.  Expecting most teachers tocreate their own curricula is like expecting most musicians to compose the musicthey play or most actors to write the works that they perform. 
Ambitious plans for having local agencies write their own curricula are usuallydoomed to fail, particularly in the absence of rigorous quality control mechanisms. Such unsuccessful attempts have been a problem that SRI and NSF havedocumented in some of the SSIs.  Other, independent studies have reached similarconclusions.  A corollary of this finding is that it proves unrealistic to expect thatgeneral state standards or curriculum frameworks will provide sufficient guidanceso that district or school staff can "fill in the blanks" to create their own curriculum(Massell, Kirst, and Hoppe, 1997).  In simple terms:  most teachers will continue torely on textbooks most of the time.  
In light of the undisputed importance of textbooks in shaping curriculum andinstruction, it is surprising that there are virtually no publicly available reviews oftextbooks, or other sources of information about their features or their quality_atleast in mathematics and science.  It is far easier to find both objective informationand expert reviews about cars, restaurants, and many other products and servicesthan it is to learn about the variations in coverage or the perceived quality ofdifferent textbooks.  (In at least two SSI states, Ohio and Delaware, informationabout some instructional materials is being made available through the World WideWeb.  However, the focus is on short units and modules more than on fulltextbooks or multiyear text series.)  Although it makes sense to think that expertsshould be involved in researching and creating reviews and product descriptions, italso seems appropriate that the information should be easily available to schoolboard members, teachers, parents, or anyone else with an interest in curriculum. This is an area in which independent, nongovernmental groups might make acontribution.  Increasing the amount of information about products doesn'teliminate variation in preferences, nor does it guarantee that low-quality productswill disappear. Nonetheless, increasing the number of sources and the accessibilityof credible, well-designed information about textbooks could prove useful to manypeople.
Improving instruction
In practice, textbooks, curriculum, and instruction are closely intertwined. Changing textbooks often requires that teachers learn new instructional practices. In any event, TIMSS, NAEP, and many other studies suggest that it is important tochange current patterns of instruction.  
The TIMSS authors claim that there is a "characteristic pedagogical flow" thatvaries from one education system to another.  One TIMSS publication, PursuingExcellence, reports the results of a study that used videotaping to examinemathematics teaching in more than 200 classrooms.  The findings are notencouraging.
American teachers of 8th-grade mathematics rarely developed concepts; instead,they focused their efforts on having students develop and practice individual skills. When experts reviewed the videotapes and rated their quality, 87 percent ofJapanese lessons and 60 percent of German lessons were judged to be of mediumor high quality, compared with only 13 percent of American lessons.  Exhibit 2reproduces a figure from Pursuing Excellence that illustrates the differentapproaches in these nations.
Exhibit 2
COMPARISON OF THE STEPS TYPICAL OF EIGHTH-GRADE MATHEMATICSLESSONS IN JAPAN, THE U.S., AND GERMANY
The emphasis on understanding is evident in the steps typical of Japaneseeighth-grade mathematics lessons:                      
Teacher poses a complex, thought-provoking problem.
Students struggle with the problem.
Various students present ideas or solutions to the class.
Class discusses the various solution methods.
The teacher summarizes the class's conclusions.
Students practice similar problems.                           
In contrast, the emphasis on skill acquisition is evident in the steps common tomost U.S. and German math lessons:                     
Teacher instructs students in a concept or skill.
Teacher solves example problems with class.
Students practice on their own while the teacher assists individual students.
The TIMSS findings about instruction are important, and they provide some usefulguidance on how instruction in U.S. classrooms might be improved.  TIMSSreminds us what a central role the teacher plays in orchestrating instruction. Videotapes of typical lessons in different countries might be quite instructive toteachers_and, in fact, the U.S. Department of Education has produced a CD-ROMshowing 8th-grade mathematics lessons from the U.S., Japan, and Germany (withsubtitles in translation).  However, we must also keep in mind that instructionbalances a great many variables.  Efforts to change instruction to align better withnational and state standards will involve not just focusing on key topics (a majorfocus of research in TIMSS) or increasing the emphasis on teaching forunderstanding of concepts (instead of simply memorizing facts or drilling on skills),but attending to a wide range of other variables, such as using materials (includingcomputer technology) effectively, organizing some instruction so that small groupsof students work together, increasing the number of lessons with aninterdisciplinary focus (e.g., combining mathematics and science), spending moretime on challenging material, making students more active learners (e.g., engagingthem in the classroom discourse), etc.  Teachers need a considerable amount oftime to learn to teach in ways that are significantly different from their currentpractice, particularly when instruction involves topics that the teachers are stilllearning to understand themselves, as is often the case.9  It would be an error tounderestimate the magnitude of the task.
The good news is that there are many successful efforts to provide teachers withhigh-quality professional development.  Policy-makers can increase theopportunities available to teachers and better integrate professional developmentinto teachers' normal work lives.  Also, much is already known about how to makeprofessional development effective (Corcoran, 1995).  But some of the issueshighlighted by TIMSS are seldom addressed in the U.S.  For example, according tothe TIMSS authors, 
U.S. teachers lack the long and carefully mentored introduction to teaching thatJapanese and German teachers receive. _  [Also,] U.S. and German teachers do nothave the rich informal opportunities to learn from each other and to share questionsabout teaching-related issues that are enjoyed by their Japanese colleagues. (Peak,1996)
Somehow, time needs to be found for teachers to work together to improveinstruction.  When time and money are available, well-designed professionaldevelopment can have an impact.
A number of the SSIs have been able to demonstrate that classroom practices havechanged as a result of their efforts.  The changes may be more modest than hadbeen hoped for and typically involve a specific set of classrooms heavily involvedwith the SSI, not whole states, but the improvements are real.  Linking professionaldevelopment explicitly to instructional materials that teachers will later use in theclassroom is one practice that many of the SSIs have used successfully.  Efforts tochange instructional practices need to be systemic in other ways, too.  Focusing onone teacher or classroom at a time is unlikely to succeed (because textbooks,tests, state standards, and other factors need to be aligned with and support goodinstruction).  What is more, there needs to be strong public support for the changesin curriculum and instruction about which teachers are learning.  This is an issue ofcontinuing importance.
Mobilizing public and professional support for change
Standards-based reform has motivated thousands of talented individuals and manydozens of different agencies and institutions across the U.S. to improve educationfor elementary and secondary students.  The development of standards, learninggoals, and curriculum frameworks is receiving much greater attention now than adecade ago.  Setting higher standards has also produced tangible gains forstudents.  For example, the percentage of graduating high school studentscompleting the minimum academic program recommended by the NationalCommission on Excellence in Education increased from 2.7 percent in 1982 to29.2 percent a decade later--a 10-fold increase10 (National Center for EducationStatistics, 1995).
At the same time, it is evident from a great deal of research that standards-basedreform is difficult, slow work.  The difficulty of standards-based reform stems fromthe fact that it presents not only technical but also political challenges.
In the political arena, it is critical to obtain substantial public and professionalsupport for change.  The "vision" that TIMSS speaks about must be carefully andpatiently articulated by policy-makers.  Some SSI states have found that it takesyears to create a vision for change and obtain "buy in" from teachers and thepublic.  Yet, without patience, there is a real risk of backlash, as has happened in afew states.
One clear lesson is that policy-makers should not try to swing the pendulum toextremes, because if people have a perception that new standards or classroompractices are too extreme, they won't support the new approaches.  Researchshows that good teachers will integrate old and new practices_and appropriately so(see, for example, Knapp et al., 1995; Massell et al., 1997).  The public needs tounderstand that setting new goals (such as having much higher proportions ofstudents successfully solve challenging problems in mathematics and science) doesnot mean abandoning all the old goals (such as having students memorize certainkey facts).  Some of the improved approaches to instruction that are called for innational and state standards, such as use of educational technology by teachersand students, will be altogether new to many parents.  As older education goals,standards, and practices are replaced by new ones, it is important that the publicnot believe the baby is being thrown out with the bathwater.
Work done in some of the SSI states illustrates one way a synthesis can becreated.  In Montana--the same state that is trying a bold, distinctively differenthigh school mathematics curriculum--policy-makers are aware that public support isvital.  Montana's draft Framework for Improving Mathematics and ScienceEducation states that:
Montanans include inquiry-based learning in their accreditation standards becausethey believe students best acquire scientific and mathematics knowledge whenthey learn in ways that reflect how mathematics and science actually work. However, Montanans are very clear about their belief that basic facts, basic skills,and basic technologies are essential tools in inquiry-based learning.
As noted earlier, many of the SSIs have set about mobilizing public andprofessional support for change.  The SSIs have used many communicationchannels, including newspapers, television, and other media; public relationsprofessionals; public forums; print publications; and even placemats in fast-foodrestaurants.
Public support seems to be more difficult to obtain in some areas than in others. At certain levels in certain disciplines (e.g., elementary mathematics), there aremany schools and classrooms where a new vision for educational practice is beingimplemented, to one degree or another.  However, in other areas there are fewerexamples of a new vision.   Among the SSIs, for example, very little effort isexpended in aggregate on changing curriculum and instruction in high schools,especially in science.  It would be wrong to conclude that there isn't a need forchange in high schools.  It simply appears to be more difficult to effect change atthat level.  Both educators and important segments of the public appear to resistmany proposed ideas for changing high schools.  For example, SRI's researchsuggests that changing the curriculum for students in high schools' lower tracks ismore acceptable to many parents11 than changing the curriculum forcollege-bound and honors students (even though state and national standards callfor curriculum changes that would affect all students).
CONCLUSIONS
The release of the first several of many TIMSS publications is a stimulus toconsider the strengths and weaknesses of standards-based education improvementefforts.  SRI's evaluation of the SSI program provides a rich database of pertinentinformation.
There is evidence that many of the SSIs have been able to have a positive impacton student achievement.  The numbers of students involved and the magnitude ofthe gains are both probably smaller than one would wish.  Still, these data providemore evidence that sustained attention to standards-based reform pays off. Furthermore, many of the SSIs have changed state education systems in ways thatare likely to have lasting impact, even after NSF's funding for them has ended. The legacies will include revised state curriculum frameworks, improvedinstructional materials, continuing teacher networks, new nonprofit institutionsadvocating school improvement, business partnerships, attentive legislatures, andsome creative, strategic thinking.
However, we have not been able to discover any simple recipe for success.  Theevidence shows that standards-based reform requires a variety of strategies,involving both "top down" and "bottom up" activities conducted by many differentagencies and institutions.  Teacher professional development is clearly a necessarypart of the mix. Public and professional support for change is essential, andstrategies for mobilizing such support are therefore important.  The fact thatchanges in many different components of the education system must be addressedwas stated clearly by the Governors and business leaders at last year's NationalEducation Summit:
We recognize that better use of technology, improved curriculum, better-trainededucators, and other changes in the organization and management of schools arenecessary to facilitate improved student performance. (National Governors'Association, 1996)
Both state governments and the federal government have demonstrated remarkablestaying power on the issue of raising education standards.  A number of indicatorsdemonstrate that progress is being made.  However, TIMSS reminds us that whatwe are really talking about is a major cultural change involving the behavior of tensof millions of Americans.  It is not surprising that the work is slow.  Continuedpersistence is the key to success.  
REFERENCES
Blank, R. K., and Gruebel, D.  (1995).  State indicators of science and mathematicseducation, 1995.  Washington, DC:  Council of Chief State School Officers.
Bodilly, S.  (1996).  Lessons from New American Schools DevelopmentCorporation's demonstration phase.  Santa Monica, CA:  RAND Corporation.
Consortium for Policy Research in Education.  (1995, July).  CPRE Policy Briefs. Tracking student achievement in science and math:  The promise of stateassessment systems.  New Brunswick, NJ:  Rutgers (CPRE).
Corcoran, T. C.  (1995).  Transforming professional development for teachers:  Aguide for state policymakers.  Washington, DC:  National Governors' Association.
Knapp, M. S., and Associates.  (1995).  Teaching for meaning in high-povertyclassrooms.  New York:  Teachers College Press.
Massell, D., Kirst, M., and Hoppe, M.  (1997).  CPRE Policy Briefs.  Persistenceand change:  Standards-based systemic reform in nine states.  Philadelphia: University of Pennsylvania (Consortium for Policy Research in Education). 
National Center for Education Statistics.  (1995).  Digest of education statistics,1995.  Washington, DC:  U.S. Department of Education.
National Governors' Association.  (1996).  National Education Summit policystatement.  Washington, DC:  Author.
National Science Foundation (Division of Elementary, Secondary, and InformalEducation).  (1997).  Review of instructional materials for middle school science.(NSF publication #97-54.)  Washington, DC:  Author.  
Peak, L.  (1996).  Pursuing excellence:  A study of U.S. eighth-grade mathematicsand science teaching, learning, curriculum, and achievement in internationalcontext.  Washington, DC:  U.S. Department of Education.
Rowe, M. B., Montgomery, J. E., Midling, M. J., and Keating, T. M.  (1997). ChemCom's evolution:  Development, spread and adaptation.  In:  Raizen, S. A.,and Britton, E. D. (eds.).  Bold ventures, volume 2:  Case studies of U.S.innovations in science education.  Boston:  Kluwer Academic Publishers.
Schmidt, W. H., et al.  (1996).  Characterizing pedagogical flow:  An investigationof mathematics and science teaching in six countries.  Boston:  Kluwer AcademicPublishers.
Schmidt, W. H., McKnight, C. C., and Raizen, S. A. (1997).  A splintered vision: An investigation of U.S. science and mathematics education.  Boston:  KluwerAcademic Publishers.
Weiss, I. R., Matti, M. C., and Smith, P. S.  (1994).  Report of the 1993 NationalSurvey of Science and Mathematics Education.  Chapel Hill, NC:  HorizonResearch, Inc.
Zucker, A. A., and Shields, P. M. (eds.).  (1995).  Evaluation of the NationalScience Foundation's Statewide Systemic Initiatives (SSI) Program, Second-yearcase studies:  Connecticut, Delaware, and Montana.  Menlo Park, CA:  SRIInternational. 
Zucker, A. A., and Shields, P. M.  (1997, February).  SSI strategies for reform: Preliminary findings from the evaluation of NSF's SSI Program.  Menlo Park, CA: SRI International.
Zucker, A. A., Shields, P. M., Adelman, N., and Powell, J.  (1995). Evaluation ofthe National Science Foundation's Statewide Systemic Initiatives (SSI) Program: Second-year report, Cross-cutting themes.  (NSF publication #96-48.)  Washington,DC:  National Science Foundation.  
Zucker, A. A., Young, V. M., and Luczak, J. M.  (1996).  Evaluation of theAmerican Association for the Advancement of Science's Project 2061.  MenloPark, CA:  SRI International. 
NOTES-  SRI International, the former Stanford Research Institute, is one of the world'slargest not-for-profit research and consulting companies.  It is headquartered inMenlo Park, California.
 SRI's partners in this effort include the Consortium for Policy Research inEducation (CPRE), Policy Studies Associates, Woodside Research Consortium, andthe Council of Chief State School Officers.
 For example, in the early 1990s, California's high school science classes(average size 27) were 60 percent larger than Wyoming's (average size 17). States also vary widely in the percentage of teachers certified in the subjects theyteach, as well as in many other areas of education (Blank and Gruebel, 1995).
 The development of a full-year or multiyear curriculum demands a high level ofquality control, and is therefore a task that not all states will want to take on.  InMontana, for example, the development of the SIMMS curriculum has involveddozens of writers, several sets of reviewers, year-long pilot tests with students,revisions, professional editing, and other steps.
 In the area of elementary and secondary education, one of the major goals ofNSF "is to provide the field with high-quality instructional materials that incorporatethe best research on teaching and learning, include accurate science and the activeparticipation of scientists in the development process, and have undergone anextensive pilot and field test process" (NSF, 1997).  As a result, in recent years,NSF has spent tens of millions of dollars supporting the development ofinstructional materials in mathematics and science.  For example, NSF recentlypublished results of an expert-panel review of 19 full-year and multiyear curriculafor middle school science that have been supported with NSF funds (ibid.).  Mostinstructional materials supported by NSF are published and distributed bycommercial publishers. 
 The eight strategies were derived empirically, based on data collected for theSSI evaluation.  This set may or may not be a complete list of possibilities.  Even ifit is not complete, it seems to include the great majority of actions that states havesupported.  (Note that efforts to increase equity in mathematics and scienceeducation cut across the eight strategies; we do not identify those efforts as aseparate strategy.)
 For example, New American Schools_formerly known as the New AmericanSchools Development Corporation_began with the assumption that an improvementstrategy focusing exclusively on individual schools would be powerful enough tochange whole systems.  More recently, however, a set of published lessonslearned about their efforts concluded that "whole-school designs and design-basedassistance are limited vehicles to reform and must be coupled creatively with otherefforts," including district and state policies, public support, testing, and otherelements (Bodilly, 1996).
 The authorizing legislation for the U.S. Department of Education (P.L. 96-88)includes protections for state and local control of education, including prohibition offederal "direction, supervision, or control over _ the selection of _ textbooks." Reviews are not control, but it seems to the author that nongovernmental agenciesare on firmer ground publishing information that could tread on publishers' toes.
 Elementary teachers, for example, often have poor backgrounds in mathematicsand, especially, science.  Also, a significant number of middle and high schoolteachers are teaching out-of-field.  Inadequate teacher preparation is a particularlyserious problem in many poor urban and rural areas.
 The Commission recommended that all students take 4 years of English, 3years of social science or history, 3 years of mathematics, 3 years of science, and� year of computer science.11  A recently published case study of the ChemComchemistry curriculum (developed by the American Chemical Society) also highlightsissues related to its acceptance and use with high school students in differenttracks (Rowe et al., 1997).