EVALUATION OF THE

NUTRIENT

STANDARD MENU

PLANNING

DEMONSTRATION

Findings from the

Formative

Evaluation

February, 1997

Prepared for:

John Endahl

U.S. Dept. of Agriculture

Food and Consumer Service

Office of Analysis and

 Evaluation

3101 Park Center Drive

Alexandria, VA 22302

Prepared by:

Mary Kay Fox

Abt Associates Inc.

55 Wheeler Street

Cambridge, MA 02138
TABLE OF CONTENTS

Page 

List of Exhibits

....................................................................................................................

ii

Executive Summary............................................................................................................

iii

Chapter 1 Introduction.................................................................................................

1-1

The NSMP Demonstration...........................................................................1-1

The Evaluation of the Demonstration.........................................................1-2

Chapter 2 Status of NSMP Implementation in Demonstration SFAs

.....................

2-1

Status of NSMP Implementation in Early February, 1996.........................2-1

Perceived Burden of NSMP Implementation Tasks

....................................

2-4

Implementation of NSMP Components

......................................................

2-8

Chapter 3 Opinions and Recommendations Regarding NSMP ..............................

3-1

SFA Directors' Opinions About NSMP.......................................................3-1

Attitudes and Perceptions of Key Stakeholders

.........................................

3-3

SFA Directors' Opinions about the NSMP Demonstration........................3-6

State Directors' Experiences and Opinions

.................................................

3-10
Abt Associates Inc. Table of Contents

LIST OF EXHIBITS

Page 

Exhibit 1 Implementation Status of NSMP Demonstration SFAs

as of February, 1996.....................................................................................2-2

Exhibit 2 Characteristics of NSMP Demonstration SFAs by Implementation Status

2-5

Exhibit 3 SFA Directors' Perceptions About Relative Burden of NSMP Tasks

.........

2-7

Exhibit 4 Reported Level of Difficulty Experienced in Obtaining Nutrition

Information from Vendors

...........................................................................

2-8

Exhibit 5 Nutrients Most Often Missing from Vendor Information..........................2-10

Exhibit 6 Software Systems Used by NSMP Demonstration SFAs

............................

2-12

Exhibit 7 Primary Reason for Software Selection.......................................................2-13

Exhibit 8 Person Responsible for Using NSMP Software...........................................2-15

Exhibit 9 SFAs' Use of and Satisfaction with Software Tutorials, Training

and Technical Assistance.............................................................................2-16

Exhibit 10 Staff Opinions About NSMP Software .......................................................2-17

Exhibit 11 Proportion of Recipes Not Yet Standardized (as of November, 1995)

......

2-19

Exhibit 12 Changes Implemented by NSMP Menus

....................................................

2-20

Exhibit 13 Meeting Nutrient Standards

........................................................................

2-21

Exhibit 14 SFA Directors' Personal Opinions about NSMP.........................................3-2

Exhibit 15 Financial and Food Service Staff Attitudes Toward NSMP.......................3-4

Exhibit 16 Stakeholders' Reported Perceptions About School Food Service

Since Beginning of NSMP Demonstration..................................................3-5

Exhibit 17 SFA Directors' Opinions About Participation in the NSMP

Demonstration..............................................................................................3-7

Exhibit 18 SFA Directors' Comments About What USDA Could Have

Done to Make NSMP Implementation Easier

.............................................

3-8

Exhibit 19 SFA Directors' Recommendations for Future Improvements

in the NSMP Demonstration .......................................................................3-9

Exhibit 20 SFA-Initiated Contacts with State Agencies

...............................................

3-11

Exhibit 21 State Visits to Demonstration SFAs

.............................................................

3-12

Exhibit 22 State Directors' Expectations for Monitoring Under NSMP......................3-13

Exhibit 23 State Directors' Personal Opinions About NSMP.......................................3-15

Exhibit 24 State Directors' Opinions About Non-Demonstration SFAs' Attitude

Toward NSMP..............................................................................................3-16
Abt Associates Inc. Executive Summary iii

EXECUTIVE SUMMARY

This report presents formative evaluation findings from the ongoing Evaluation of the

Nutrient Standard Menu Planning (NSMP) Demonstration. Data were collected primarily

from directors of School Food Authorities (SFAs) participating in the demonstration; brief

interviews were also conducted with directors of Child Nutrition (CN) programs in cognizant

States.

The NSMP Demonstration began in January, 1994, with the selection of 35 SFAs. One

SFA dropped out of the demonstration immediately, due to lack of school board approval. 

Key staff from each of the remaining 34 SFAs received training from USDA in June, 1994. 

SFAs were expected to begin implementing NSMP during SY 1994-95 and to be fully

operational, with NSMP menus implemented in all schools in the district, by the Spring of

1996. 

SFA directors were interviewed in October-November, 1995 and February, 1996 to

obtain information on implementation status as well as on barriers to NSMP and perceived

strengths and weaknesses of the system. Directors of cognizant State CN offices were also

interviewed in October-November, 1995. Major findings from these interviews are

summarized in this report.

Implementation

As of early February, 1996, only seven of the 34 demonstration SFAs had

implemented NSMP in all schools. Ten districts were partially implemented, with NSMP

menus being used for breakfast (if offered) and/or lunch in some schools. Nine SFAs were

still in the process of collecting and entering data, planning menus, and/or analyzing menus,

and had not yet implemented NSMP in any schools. The remaining eight SFAs had

withdrawn from the demonstration because of concerns about the use of weighted nutrient

analysis, problems with NSMP software, and/or concerns about staff resources required to

implement NSMP. 

Preliminary evidence suggests that SFAs most likely to experience difficulty in

implementing NSMP are very large SFAs located in urban areas and serving largely lowincome populations. SFAs that do not have a registered dietitian on staff are also more likely

to experience difficulties.

NSMP Software

In February 1996, just over one-half of the SFAs still participating in the

demonstration were using the Nutrikids software package. Roughly one-quarter were using

School Nutrition Accountability Program (SNAP) software and about one-eighth were using the

Computer-Assisted Food Service (CAFS) system. Three SFAs were using software systems that

had not yet been approved by USDA.1

 

 

1

Eleven other software systems have been approved by USDA since the time these data were collected, including

the Computrition system which is being used by one of the demonstration SFAs.
Nutrient Standard Menu Planning Demonstration: Formative Evaluation Report

Abt Associates Inc. Executive Summary iv

SFA staff were generally happy with the NSMP system they were using. Ninety

percent rated their NSMP software as either very easy or easy to use. More than 80 percent

rated their NSMP software as either exceptional or satisfactory with regard to time efficiency.

Menu Changes

Among SFAs that were far enough along in the implementation process to have made

menu modifications, the changes reported most frequently were increased use of fresh fruits

and vegetables and increased use of lower-fat products. Some SFAs reported adding foods

high in carbohydrate to elementary school menus and/or increasing portion sizes in middle

and high school menus in order to meet standards for calories.

Meeting Nutrient Standards

Most SFAs reported difficulty meeting the calorie standard for both breakfast and

lunch menus (i.e., menus planned for one or more age groups were low in calories). In

addition, more than one-half of SFAs reported having problems meeting fat and saturated fat

standards at lunch. 

Perceived Burden of NSMP Implementation Tasks

A majority of SFA directors reported that implementing NSMP imposed a minor or

significant burden on SFA staff and resources. Tasks associated with collecting and entering

data were viewed as most burdensome. Planning menus, monitoring purchased foods,

developing purchasing specifications, and marketing healthful meals to students were

perceived as less burdensome tasks.

SFA Directors' Opinions About NSMP

A majority of SFA directors reported feeling very positive or somewhat positive about

NSMP. SFA directors who reported feeling only somewhat positive about NSMP were

generally supportive of the goal of NSMP but were having difficulty with some aspect of

implementation.

Positive aspects of NSMP identified by SFA directors include the fact that NSMP

provides an accurate assessment of the nutrient content of meals offered; that NSMP provides

assurance that meals offered are healthful; and that NSMP provides increased flexibility in

menu planning. Negative aspects of NSMP include the time and labor required to implement

the system and the use of weighted nutrient analysis.

Perceptions and Attitudes of Key Stakeholders

Directors in a majority of demonstration SFAs reported that district financial staff and

school food service staff have very positive or somewhat positive attitudes toward NSMP. 

Moreover, most directors believe that implementation of NSMP has had a positive impact on
Nutrient Standard Menu Planning Demonstration: Formative Evaluation Report

Abt Associates Inc. Executive Summary v

how parents and teachers feel about school food service. Few SFA directors reported changes

in students' attitudes since the implementation of NSMP; however, at the time interviews

were conducted, many SFAs had not yet mounted information campaigns directed at

students.

SFA Directors' Opinions About the NSMP Demonstration

More than one-half of the SFA directors still participating in the demonstration

indicated that they had no regrets about volunteering to participate. Among those who did

report regrets, the primary source of concern was the staff time required to implement NSMP.

Despite the reservations voiced by some SFA directors, 100 percent of directors

indicated that they would continue with NSMP even if given the opportunity to return to the

old menu planning system. About one-third would continue with the current NSMP

protocol. The remaining two-thirds would change one or more aspects of the existing

protocol; most would eliminate use of weighted nutrient analysis.

State Directors' Experiences and Opinions

Demonstration SFAs are located in 19 different States. While almost two-thirds of the

cognizant State directors had made telephone contact with their NSMP demonstration site(s)

by the time the formative evaluation interviews were conducted, fewer than half had actually

visited the district(s).

A common concern voiced by State directors who had visited a demonstration site

was that SFAs would not be able to meet the labor requirements associated with NSMP

implementation. Other concerns included the need for additional staff training and the need

for improved maintenance of food production records.

Although State directors indicated that they are generally supportive of the goal of

NSMP, many had reservations about the ability of SFAs to successfully implement the

program. Most State directors did not expect a substantial number of SFAs to elect NuMenus

in the upcoming school year. Reasons cited for potential SFA resistance to NuMenus included

fear of change; concerns about labor requirements; and fear of/lack of familiarity with

technology and computers.
Abt Associates Inc. Chapter One: Introduction 1-1

Chapter 1

INTRODUCTION

This report summarizes data collected as part of the formative evaluation of the

Nutrient Standard Menu Planning (NSMP) Demonstration. Data were collected primarily

from directors of School Food Authorities (SFAs) participating in the demonstration. Brief

interviews were also completed with State Agency directors in cognizant States. 

This chapter provides an overview of the NSMP Demonstration and the associated

evaluation. The second chapter summarizes the status of NSMP implementation in

demonstration SFAs as of February, 1996 and examines characteristics of SFAs in varying

stages of implementation. SFA directors' perceptions about the relative burden associated

with NSMP implementation are also discussed. The third and final chapter provides a

synopsis of opinions about NSMP in general and the demonstration in particular. Opinions

of SFA directors and State agency directors are featured, along with SFA directors'

assessments of the opinions of key stakeholders (e.g., food service staff, district financial staff,

teachers, parents and students). The chapter concludes with a discussion of

recommendations and suggestions offered by SFA directors for improving NSMP and/or the

demonstration.

The NSMP Demonstration

The NSMP Demonstration was initiated by the U.S. Department of Agriculture

(USDA) in 1993. The purpose of the demonstration is to examine NSMP as an alternative

approach to planning menus in school-based Child Nutrition (CN) programs. Under the

NSMP (now known as NuMenus) system, menus are planned using specialized nutrient

analysis software rather than traditional program meal patterns which specify both the types

and quantities of food to be included in each meal. Menus are required to meet specific

nutrient standards (one-third of the Recommended Dietary Allowances (RDAs) for lunches

and one-quarter for breakfasts) for food energy, protein, vitamin A, vitamin C, iron, and

calcium. Menus must also satisfy Dietary Guidelines for Americans standards for percent of

calories from fat (30 percent or less) and saturated fat (less than 10 percent). The NSMP

system also calculates sodium, cholesterol, dietary fiber, and carbohydrate content and, while

no specific standards have been established for these nutrients, SFAs are encouraged to

monitor them.

To support NSMP, USDA developed the National Nutrient Database for Child

Nutrition Programs (NND-CNP). The database includes nutrient information from USDA's

standard nutrient data base for foods routinely used in school food service. Initially, USDA

planned to augment the main data base with nutrient information for an extensive number of

commercially-prepared food items. USDA staff requested nutrition information, in a

specified format, from major food manufacturers servicing the school food service industry. 

Information supplied by manufacturers was to be reviewed by USDA staff and entered into

the NND-CNP. As discussed later in this report, however, the food service industry was slow

in responding to this request.

In addition to the specialized nutrient data base, USDA developed detailed
Nutrient Standard Menu Planning Demonstration: Formative Evaluation Report

Abt Associates Inc. Chapter One: Introduction 1-2

specifications for companion nutrient analysis software. The Agency published the

specifications and invited companies to develop software which met the performance and

functional criteria. Once developed, software systems were submitted to USDA for review

and approval. Approved software systems could be used by any NSMP site or any school

district interested in implementing NSMP.

In the summer of 1993, USDA invited SFAs to volunteer for the NSMP demonstration.

 Thirty-five SFAs were selected in January, 1994 from a pool of 127 applicants. SFAs were

selected to provide diversity in geographic location, size, CN program participation, student

participation rates, characteristics of food service programs, and staff experience with

computerized nutrient analysis. One of the 35 SFAs dropped out shortly after selection

because school board approval for the project was not obtained. The district was not

replaced. Thus, the demonstration began with a slate of 34 SFAs. 

The demonstration spans three school years, from SY 1994-95 through SY 1996-97. 

Key staff from participating SFAs attended a training session in June, 1994. NSMP

implementation was expected to begin during SY 1994-95 and to be fully operational, with

NSMP menus implemented in all schools in the district, by the Spring of 1996. 

The Evaluation of the Demonstration

The evaluation design includes both formative and summative evaluation

components. The summative evaluation includes comparison of baseline and posttest

measures of nutrient content, meal costs, student participation, plate waste, and

characteristics of food service operations, as well as an assessment of the acceptability of

NSMP to both district and school-level food service staff. Baseline data were collected in the

Spring of 1994, before SFA staff received any training in NSMP. Some posttest data were

collected in the Spring of 1996 in SFAs that had actually implemented NSMP in elementary,

middle, and/or high schools. Additional posttest data will be collected in the Spring of 1997.

The formative evaluation, which this document addresses, was intended to provide

the Agency with information about difficulties encountered by demonstration SFAs during

the implementation process; SFAs' need for clarification and/or guidance; and the potential

need for modifications or adjustments in the implementation protocol. The original timeline

called for the formative evaluation to be completed in April, 1995, at which point SFAs were

expected to be well on their way to implementing NSMP. Unfortunately, a substantial delay

in the approval of NSMP software impeded significantly the progress SFAs were able to make

during the first year of the demonstration. (This issue is discussed in more detail in Chapter

Two.) As a result, few SFAs had a substantial amount of experience with NSMP by April of

1995, making it impractical to conduct the formative evaluation at that time. Instead, SFA

Directors were interviewed briefly to obtain basic information on the software systems being

used, the amount of implementation work completed to date, major obstacles encountered,

and general opinions about NSMP. This information was shared with USDA staff. 

Detailed formative evaluation interviews were completed with SFA directors and

cognizant State CN directors approximately six months after the originally scheduled date, in

October-November, 1995. Information on the status of NSMP implementation was updated

in February, 1996. Findings from these interviews are discussed in the next two chapters. 
Chapter 2

STATUS OF NSMP IMPLEMENTATION IN DEMONSTRATION SFAS

From the outset, NSMP implementation in all SFAs has been complicated by a number

of factors. Most important was a significant delay in the approval of NSMP software systems

and an associated delay in start-up activities in most SFAs. SFA staff were trained in June,

1994. They were unable to begin work for some time afterward, however, because no

approved software systems were available. Software vendors were slow in submitting

products for USDA review and approval, and many of the programs submitted did not

satisfy the established functional criteria.

By January, 1995, six months after SFAs were trained, only two NSMP software

packages had been approved. At that time, USDA contacted all demonstration SFAs and

asked that they select one of the two available systems and make a concerted effort to move

forward with NSMP implementation during the remainder of SY 1994-95. Several SFAs

lobbied for approval of other software systems, and a third package was ultimately

approved.2

While all but three of the demonstration SFAs eventually complied with USDA's

request and selected one of the three approved software systems, many SFAs did not really

begin working with NSMP software until well into the latter half of SY 1994-95 or the

subsequent summer months. This delayed start-up had a substantial impact on the progress

SFAs were able to make during the first year of the demonstration.

Another problem that complicated implementation was the poor response USDA

received from food manufacturers contacted to supply nutrient information for the NNDCNP. Very few manufacturers responded to the request, and much of the data submitted

was found to be incomplete, inaccurate, or otherwise questionable. As a result, the NNDCNP included in the approved software systems did not include many of the commerciallypurchased products used in school food service. This situation placed an unanticipated

burden on each demonstration SFA. SFA staff had to assume responsibility for obtaining

nutrient information for nationally-available commercial products and for entering this data

into their copy of the NND-CNP. This requirement substantially increased labor

requirements for start-up activities which, in most districts, contributed to a protracted

implementation period. 

Status of NSMP Implementation in Early February, 1996

In view of the complications described above, it is not surprising that, by early

February, 1996, only seven of the original 34 SFAs, approximately one in five, had

implemented NSMP fully in all schools (i.e., had implemented NSMP menus for both

breakfast (if offered) and lunch in all schools) (Exhibit 1).

Ten other SFAs (29 percent) had implemented NSMP partially. Most of these SFAs

had implemented NSMP for elementary school menus (lunch and/or breakfast) but had not

yet completed planning and analysis of middle and secondary school menus. Some SFAs

 

 

2

Since January, 1995, eleven other software systems have been approved.
Nutrient Standard Menu Planning Demonstration: Formative Evaluation Report

Abt Associates Inc. Chapter Two: Status of NSMP Implementation in Demonstration SFAs 2-2

were still building data bases for middle and high school menus, i.e., collecting and entering

nutrition information and/or district-wide production information. Also included in the

"partially-implemented" group are SFAs that deviated in
Nutrient Standard Menu Planning Demonstration: Formative Evaluation Report

Abt Associates Inc. Chapter Two: Status of NSMP Implementation in Demonstration SFAs 2-3

Exhibit 1

IMPLEMENTATION STATUS OF NSMP DEMONSTRATION

SFAS AS OF FEBRUARY, 1996

IMPLEMENTATION STATUS NUMBER

(PERCENT) OF SFAS

(n = 34)

Fully Implemented1 7 (21)

Partially Implemented 10 (29)

Lunch menus implemented in elementary, middle and high schools, but

analysis done with unapproved software.2

1 (3)

Lunch and breakfast (if offered) menus implemented in elementary

schools. Still planning/analyzing menus for middle and high schools.3

3 (9)

Lunch and breakfast (if offered) menus implemented in elementary

schools. Still gathering and entering data for middle and high school

menus.

2 (6)

Breakfast menus implemented at all levels. Lunch menus implemented

in elementary schools. Still planning/analyzing lunch menus for

middle and high schools.

1 (3)

Breakfast menus implemented at all levels. Still planning and analyzing

lunch menus for all levels.

1 (3)

Breakfast and lunch menus implemented in a subset of 21 (out of 84)

schools.

1 (3)

Breakfast and lunch menus for one week, plus selected days

throughout cycle, implemented at all levels. Still gathering and entering

data for remainder of cycle.

1 (3)

Not Yet Implemented 9 (26)

Still planning and analyzing lunch and breakfast (if offered) menus for

all levels.

4 (12)

Still planning and analyzing elementary lunch and breakfast menus.

Still gathering/entering data for middle and high schools.

1 (3)

Still planning and analyzing elementary lunch menus. Still

gathering/entering data for middle and high school lunch and for

breakfast at all levels.

1 (3)

Still gathering and entering data (or just ready to begin planning and

analyzing menus) for all levels, all meals.4

3 (9)

Dropped Out 8 (24)

1 Both lunch and breakfast (if offered) menus implemented in elementary, middle, and high school.

2 SFA is using Computrition software (not approved at the time). Staff adjusted nutrient analysis by hand to reflect

weights. SFA does not serve reimbursable breakfast.

3 One SFA is using Practorcare software (not approved). Menus are not fully weighted, but have been adjusted

using historical information on food purchases, student preferences, etc.

4 One SFA is using Practorcare software (not approved); waiting for USDA approval before beginning
Nutrient Standard Menu Planning Demonstration: Formative Evaluation Report

Abt Associates Inc. Chapter Two: Status of NSMP Implementation in Demonstration SFAs 2-4

implementation.

Source: SFA director interviews (February, 1996).
Nutrient Standard Menu Planning Demonstration: Formative Evaluation Report

Abt Associates Inc. Chapter Two: Status of NSMP Implementation in Demonstration SFAs 2-5

some way from the NSMP protocol. For example, one SFA had implemented NSMP in a

subset of schools rather than all schools in the district. Two other SFAs had implemented

nutrient-based menus in all or some schools, but were using software which had not received

USDA approval. In addition to the use of a different nutrient data base, unapproved

software systems do not perform weighted nutrient analysis. Staff in these SFAs attempted to

emulate the weighting function of NSMP software by adjusting the results of traditional

nutrient analyses either by hand or by using companion spreadsheets.

Nine SFAs, approximately one-quarter of the original thirty-four demonstration SFAs,

had not yet implemented NSMP in any schools. Six of these SFAs had gotten to the point of

planning and analyzing menus but had not actually implemented new menus in any schools.

 The remaining three SFAs had not begun any nutrient analysis.

Eight SFAs (24 percent) dropped out of the demonstration prior to February, 1996. 

One SFA withdrew because there had been a change in SFA directors; the new director did

not feel he could devote the necessary time or resources to NSMP. The other seven SFAs

withdrew intentionally. Directors in all but one of these SFAs cited concerns about the use of

weighted nutrient analysis as a reason for withdrawal. With the exception of one director,

whose SFA had a fully networked information system, directors were concerned about the

amount of staff labor involved in maintaining the production records required under NSMP. 

Most were also concerned that weighted nutrient analysis would limit flexibility in menu

planning and decrease the ability of individual schools to cater to students' preferences. 

About half of the directors were specifically concerned that, in order to meet nutrient

standards, popular, high-fat food items would have to be eliminated or offered much less

frequently and that these changes would have a negative impact on lunch participation in

middle and high schools. 

Problems with NSMP software played a major role in the withdrawal of five SFAs. 

Two SFAs terminated because the director did not want to use nutrient analysis software that

was incompatible with software the district was already using for other program operations. 

One SFA could not afford the hardware upgrade required to run NSMP software at the same

time as other routinely-used software packages. Directors in the remaining two SFAs were

frustrated by the numerous problems staff had experienced in working with one of the

approved NSMP software systems.

A final reason for withdrawal, which influenced the decision in five SFAs, was the

implementation timeline. Specifically, SFA directors were unhappy that they were expected

to implement NSMP fully by the Spring of 1996 despite the lengthy delay in the software

approval process. SFA directors reported that they could not dedicate the amount of staff

labor that would be required to implement NSMP within the condensed timeline.

NSMP Implementation in Different Types of SFAs

At this point in the evaluation, it is instructive to examine how implementation status

varies across a range of SFA characteristics as a means of identifying characteristics which

may influence, either positively or negatively, the likelihood that an SFA will be successful in

implementing NSMP. However, the small sample, external influences (e.g., delays in

software approval and complications with the NND-CNP data base), and the fact that most

SFAs are still in the early stages of implementation dictate that findings from this analysis be

interpreted cautiously. Trends noted in this analysis should be interpreted as merely

suggestive of potential relationships between SFA characteristics and the likelihood of
Nutrient Standard Menu Planning Demonstration: Formative Evaluation Report

Abt Associates Inc. Chapter Two: Status of NSMP Implementation in Demonstration SFAs 2-6

successful NSMP implementation. Patterns noted at this stage will be examined in the final

analysis to see if they hold up over time.

With this caveat in mind, the data displayed in Exhibit 2 reveal some interesting

patterns. Overall, the data suggest that the SFAs most likely to experience difficulty in

implementing NSMP include SFAs located in urban areas, very large SFAs (total enrollment

of 25,000 or more), SFAs that serve largely low-income populations, and SFAs that do not

have a registered dietitian on staff.

As of February, 1996, the proportion of urban SFAs that had not successfully

implemented NSMP (SFAs in the "not yet implemented" and "dropped out" categories) was

substantially higher than that noted for suburban and rural SFAs (78 percent vs. 37 and 44

percent, respectively). A similar pattern was noted for SFA size. The proportion of very large

SFAs, those with total enrollments of 25,000 or more students, that had not at least partially

implemented NSMP was notably higher than for SFAs of other sizes (77 percent vs. 38 to 50

percent).

Implementation status also differed dramatically for SFAs located in lower-income

areas. Eleven of 14 (78 percent) SFAs in which more than 50 percent of lunches are served

free of charge had either not implemented NSMP or had dropped out of the demonstration. 

In contrast, only 30 percent of more affluent SFAs, those in which 50 percent or less of all

lunches are served free, fell into these categories.

Finally, implementation status differed, although less strikingly, among SFAs that did

and did not have a registered dietitian on staff, either as the SFA director or NSMP

coordinator. Sixty-four percent of SFAs without dietitians fell into the "not yet implemented"

or "dropped out" categories, compared to 43 percent of SFAs with dietitians.

No discernable pattern was detected for several other SFA characteristics, namely,

average daily participation, prior experience with computerized nutrient analysis, SFA

director experience, use of cycle menus before NSMP, or prevalence of standardized recipes

before NSMP. Other variables examined, but not included in the exhibit, were the level of

centralization in menu planning and food preparation (before NSMP) and the complexity of

baseline food service operations, e.g., number of items offered, use of specialty bars, use of

self-serve foods, etc. Some of these characteristics are factors which one might expect to

influence NSMP implementation. The fact that a relationship is not apparent at this point

does not mean that these characteristics are not influential. It is quite possible that these, or

other SFA characteristics, may influence implementation during the latter stages of the

demonstration, as more SFAs attempt to move toward full implementation.

Perceived Burden of NSMP Implementation Tasks

Directors of all SFAs still active in the demonstration were asked to rate the relative

burden of 11 specific tasks associated with NSMP implementation (Exhibit 3). The results

indicate that, at the time formative evaluation data were collected (October-November, 1995),

the majority of SFA directors believed that several NSMP implementation tasks were

imposing some burden on SFA staff and resources. Of the 11 tasks discussed, all but three

were rated as either a minor burden or a significant burden by more than half of the

demonstration SFA directors.
Nutrient Standard Menu Planning Demonstration: Formative Evaluation Report

Abt Associates Inc. Chapter Two: Status of NSMP Implementation in Demonstration SFAs 2-7

The tasks viewed as most burdensome were those associated with collecting and

entering data -- tasks that SFA staff would presumably not have to deal with in the absence

of NSMP. Entering and analyzing recipes and obtaining food production information for

weighted nutrient analysis were perceived as the most burdensome tasks; 58 percent of SFA

directors indicated that these tasks imposed a significant burden on the SFA. Tasks perceived

as least burdensome were those that would be required under any menu planning system

including planning menus, monitoring purchased foods, developing purchasing

specifications, and marketing healthful meals to students. 
Nutrient Standard Menu Planning Demonstration: Formative Evaluation Report

Abt Associates Inc. Chapter Two: Status of NSMP Implementation in Demonstration SFAs 2-8

Exhibit 2

CHARACTERISTICS OF NSMP DEMONSTRATION SFAS

BY IMPLEMENTATION STATUS

CHARACTERISTIC IMPLEMENTATION STATUS

FULLY

IMPLEMENTED

(n = 7)

PARTIALLY

IMPLEMENTED

(n = 10)

NOT YET

IMPLEMENTED

(n = 9)

DROPPED

OUT

 (n = 8)

ALL SFAS

(n = 34)

n % n % n % n % n %

FCS Region

MARO 1 (25) 2 (50) 1 (25) 0 (0) 4 (100)

MPRO 1 (25) 2 (50) 0 (0) 1 (25) 4 (100)

MWRO 0 (0) 2 (40) 3 (60) 0 (0) 5 (100)

NERO 2 (40) 2 (40) 0 (0) 1 (20) 5 (100)

SERO 0 (0) 0 (0) 1 (20) 4 (80) 5 (100)

SWRO 1 (17) 2 (33) 2 (33) 1 (17) 6 (100)

WRO 2 (40) 0 (0) 2 (40) 1 (20) 5 (100)

Community Type

Urban 0 (0) 2 (22) 5 (56) 2 (22) 9 (100)

Suburban 4 (25) 6 (38) 1 (6) 5 (31) 16 (100)

Rural 3 (33) 2 (22) 3 (33) 1 (11) 9 (100)

Enrollment

Under 2,500 1 (17) 2 (33) 1 (17) 2 (33) 6 (100)

2,500-9,999 3 (33) 2 (22) 3 (33) 1 (11) 9 (100)

10,000-24,999 3 (38) 2 (25) 2 (25) 1 (12) 8 (100)

25,000 or more 0 (0) 4 (36) 3 (27) 4 (36) 11 (100)

Mean 12,723 26,626 27,079 33,383 25,160

Mean Average Daily NSLP Participation

40 percent or less 2 (29) 2 (29) 0 (0) 3 (43) 7 (100)

41-55 percent 2 (22) 2 (22) 4 (44) 1 (11) 9 (100)

56-70 percent 2 (15) 4 (31) 4 (31) 3 (23) 13 (100)

71 percent or more 1 (25) 1 (25) 1 (25) 1 (25) 4 (100)

Missing10 (0) 1 (100) 0 (0) 0 (0) 1 (100)

Mean 53% 54% 56% 51% 54%

Percent of Lunches Served Free

25 percent or less 4 (40) 2 (20) 1 (10) 3 (30) 10 (100)

26-50 percent 2 (20) 6 (60) 1 (10) 1 (10) 10 (100)

51-74 percent 0 (0) 1 (12) 5 (62) 2 (25) 8 (100)

75 percent or more 1 (17) 1 (17) 2 (33) 2 (33) 6 (100)

Mean 31% 38% 59% 49% 44%
Nutrient Standard Menu Planning Demonstration: Formative Evaluation Report

Abt Associates Inc. Chapter Two: Status of NSMP Implementation in Demonstration SFAs 2-9

Exhibit 2

Computer Expertise

SFA had prior experience with

computerized nutrient analysis

Yes 4 (20) 7 (35) 5 (25) 4 (20) 20 (100)

No 3 (21) 3 (21) 4 (29) 4 (29) 14 (100)

NSMP Coordinator had prior

experience with computerized nutrient

analysis

Yes 1 (12) 3 (38) 2 (25) 2 (25) 8 (100)

No 6 (23) 7 (27) 7 (27) 6 (23) 26 (100)

Nutrition/Food Service Experience

SFA director or NSMP Coordinator is

registered dietitian (R.D.)

Yes 6 (26) 7 (30) 4 (17) 6 (26) 23 (100)

No 1 (9) 3 (27) 5 (45) 2 (18) 11 (100)

SFA directors’ total food service

experience (mean years)

16 15 14 14 15

SFA directors’ experience as director

(mean years)

10 7 8 9 8

Use of Cycle Menu Prior to NSMP

Yes 3 (18) 5 (29) 6 (35) 3 (18) 17 (100)

No 4 (24) 5 (29) 3 (18) 5 (29) 17 (100)

Prevalence of Standardized Recipes Prior to NSMP

All or most 4 (44) 2 (22) 3 (33) N/A N/A 9 (100)

Some 3 (27) 3 (27) 5 (45) N/A N/A 11 (100)

Few or none 1 (17) 4 (67) 1 (17) N/A N/A 6 (100)

1Information on meal counts not provided.

N/A: Information not available.

Source: NSMP Demonstration application forms (Fall, 1993) and SFA director interviews (November-October, 1995 and February,

1996).
Nutrient Standard Menu Planning Demonstration: Formative Evaluation Report

Abt Associates Inc. Chapter Two: Status of NSMP Implementation in Demonstration SFAs 2-10

Exhibit 3

SFA DIRECTORS' PERCEPTIONS ABOUT RELATIVE BURDEN OF NSMP TASKS

NSMP TASKS SIGNIFICANT

BURDEN

MINOR

BURDEN

NOT A

BURDEN

NUMBER (PERCENT) OF SFAS

(n = 26)

Entering/analyzing recipes 15 (58) 7 (27) 4 (15)

Entering/analyzing menus 14 (54) 8 (31) 4 (15)

Obtaining nutrient data for foods not in the

database

11 (42) 9 (35) 6 (23)

Obtaining food production information for

weighted analysis

15 (58) 3 (12) 8 (31)

Entering nutrient data for foods not in the

data base

13 (50) 4 (15) 9 (35)

Developing standardized recipes 11 (42) 6 (23) 9 (35)

Training food service staff 12 (46) 4 (15) 10 (38)

Planning menus 5 (19) 10 (38) 11 (42)

Monitoring purchased foods to ensure that

specifications are met

3 (12) 10 (38) 13 (50)

Developing/modifying specifications for

purchased foods

3 (12) 9 (35) 14 (54)

Marketing healthier choices to students 4 (15) 6 (23) 16 (62)

Source: SFA director interviews (October-November, 1995).
Nutrient Standard Menu Planning Demonstration: Formative Evaluation Report

Abt Associates Inc. Chapter Two: Status of NSMP Implementation in Demonstration SFAs 2-11

As mentioned in the introduction to this chapter, USDA experienced major difficulties

in obtaining nutrition information for nationally-available commercial food products to

include in the NND-CNP. Consequently, the task of obtaining this information fell to

individual SFAs. USDA provided districts with guidelines for requesting information from

manufacturers, including specifications for the data to be submitted. Unfortunately, in the

initial stages of the demonstration, individual SFAs were no more successful in acquiring the

necessary information than USDA headquarters had been. In fact, a principal finding from

telephone interviews completed with demonstration SFAs in April, 1995 was that SFAs were

having great difficulty with this task. Many manufacturers were unable or unwilling to

provide the information and, when information was obtained, it was often incomplete.

In response to this early feedback from demonstration SFAs, USDA launched a

concerted effort to communicate with industry representatives and convey the importance of

their participation in this endeavor. Moreover, data reporting requirements were simplified

to more closely match nutrient information summaries routinely prepared by manufacturers.

 For example, requirements to report moisture and ash were dropped and a decision was

made to accept nutrient information prepared for/displayed on package labels. 

Results of the formative evaluation suggest that these changes have had a positive

impact on the ability of SFAs to obtain nutrition information for commercially-prepared

foods. While most SFAs still consider this task to be burdensome (Exhibit 3), the information

appears to be readily available (Exhibit 4). By and large, SFAs that had worked seriously on

this task by the time the formative evaluation interviews were completed had little difficulty

obtaining nutrition information for commercially-prepared foods. The difficulties that were

reported were very idiosyncratic (i.e., small or rural SFAs didn't seem to be having any more

difficulty than large, urban districts) and were generally associated with locally-produced

products. For example, SFAs who reported difficulty obtaining nutrition information for

baked goods were most often dealing with local wholesale bakeries that were unable to

provide the information.

SFAs have encountered problems with missing information for specific nutrients or

food components (Exhibit 5), most notably saturated fat. More than a third of the SFA

directors indicated that information on saturated fat content was missing for some products.

NSMP Software Systems

As discussed in the introduction to this chapter, at the time SFAs were encouraged to

select a software system, three systems had been approved by USDA for use in the NSMP

demonstration. The three systems were: SNAP (School Nutrition Accountability Program);

CAFS (Computer-Assisted Food Service); and NUTRIKIDS (developed and marketed by

LunchByte Systems, Inc.). By the time formative evaluation interviews were conducted, in

(October-November, 1995), a fourth system, Lunch Box (developed and marketed by Horizon

Software) had been approved.3 In late 1995, more than half of the demonstration SFAs were

using the NUTRIKIDS system (Exhibit 6). Only six SFAs (23 percent) were using SNAP and

three SFAs (12 percent) were using CAFS. The remaining three SFAs were using software

 

 

3

To date, ten other software systems (eight of which incorporate the NUTRIKIDS software, under special licensing

agreements) have been approved. Since none of these systems was available prior to the time formative evaluation

interviews were conducted, they are not included in this discussion.
Nutrient Standard Menu Planning Demonstration: Formative Evaluation Report

Abt Associates Inc. Chapter Two: Status of NSMP Implementation in Demonstration SFAs 2-12

systems that had not received USDA approval.4 Reasons for software selection are

summarized

 

 

4One of these systems, Computrition, has since received USDA approval (June 20, 1996).
Nutrient Standard Menu Planning Demonstration: Formative Evaluation Report

Abt Associates Inc. Chapter Two: Status of NSMP Implementation in Demonstration SFAs 2-13

Exhibit 4

REPORTED LEVEL OF DIFFICULTY EXPERIENCED IN

OBTAINING NUTRITION INFORMATION FROM VENDORS

PRODUCT

VERY

EASY EASY DIFFICULT

VERY

DIFFICULT

CAN'T

SAY YET

N/A

PERCENT OF SFAS

 (n = 26)

Baked breads and rolls 35 39 15 0 12 0

Baked desserts 19 19 19 4 12 27

Dairy products 35 46 4 4 12 0

Pizza 39 50 12 0 0 0

Processed chicken products 31 42 23 0 4 0

Processed meat products 23 50 27 0 0 0

Processed fish products 27 42 31 0 0 0

Mixed dishes/ethnic foods 23 27 27 4 15 4

Frozen desserts 19 31 19 8 15 8

Snack foods 31 31 19 4 0 15

N/A: Not applicable. SFA does not offer or purchase the food or product.

SOURCE: SFA director interviews (October-November, 1995).
Nutrient Standard Menu Planning Demonstration: Formative Evaluation Report

Abt Associates Inc. Chapter Two: Status of NSMP Implementation in Demonstration SFAs 2-14

Exhibit 5

NUTRIENTS MOST OFTEN MISSING FROM VENDOR INFORMATION

NUTRIENT

NUMBER

(PERCENT) OF SFAS

(n = 23)1

Saturated fat 9 (39)

Fiber 3 (13)

Iron 3 (13)

Vitamin A 3 (13)

Calcium 2 (9)

Vitamin C 1 (4)

1

Includes only SFAs that had made some effort to collect information from vendors.

SOURCE: SFA director interviews (October-November, 1995).
Nutrient Standard Menu Planning Demonstration: Formative Evaluation Report

Abt Associates Inc. Chapter Two: Status of NSMP Implementation in Demonstration SFAs 2-15

Exhibit 6

SOFTWARE SYSTEMS USED BY NSMP DEMONSTRATION SFAS

SOFTWARE SYSTEMS

NUMBER

(PERCENT) OF SFAS

(n = 26)

USDA Approved

NUTRIKIDS 14 (54)

SNAP 6 (23)

CAFS 3 (12)

Not Approved

Practorcare 2 (8)

Computrition11 (4)

1Computrition received USDA approval in June, 1996. The version of the software in use at the time the formative evaluation

interview was conducted, however, was not approved NSMP software.

SOURCE: SFA director interviews (October-November, 1995).
Nutrient Standard Menu Planning Demonstration: Formative Evaluation Report

Abt Associates Inc. Chapter Two: Status of NSMP Implementation in Demonstration SFAs 2-16

in Exhibit 7. Most SNAP sites began working with SNAP when it was the only NSMP software available.

SNAP was used in the training USDA provided for demonstration sites in June, 1994.5 All SFAs received

a test copy of the program to use in continuing to gain familiarity with the basics of NSMP, but were

encouraged to investigate other software systems as they became available. As previously discussed,

however, it was quite a while before alternative programs were approved. Consequently, many of the

SFAs that got an early start on implementation stayed with the SNAP system. Not surprisingly, the

proportion of SNAP sites that is fully implemented is substantially higher than NUTRIKIDS sites (67

percent vs. 21 percent; data not shown).6

Two of the three SFAs using the CAFS system chose it because they owned the original CAFS

food service management system and simply purchased the NSMP software as an enhancement. The

one SFA that elected to purchase CAFS did so because of the system's other features, i.e., modules that

support a variety of food service administration and operations tasks. The three SFAs using unapproved

software had been using these systems prior to the NSMP demonstration and were reluctant to switch to

another system.

In most SFAs, directors are assuming hands-on responsibility for use of the NSMP software

(Exhibit 8), although clerks or interns have often been used to enter nutrition information into the data base

and to summarize food production information. Directors of very large SFAs (25,000 or more students) as

well as a few of the large SFAs (10,000 - 24,999 students) have delegated day-to-day responsibility for

NSMP implementation to a staff nutritionist or a programmer/data clerk.

SFA staff appear to be taking appropriate advantage of tutorials, training, and technical assistance

opportunities provided by software vendors (Exhibit 9), although the proportion of SNAP sites that received

training (either in person or over the phone) is noticeably lower than the proportion of NUTRIKIDS or

CAFS sites. It is not clear whether this is due to personal choice of SFA staff or because the service is

not readily available. The vast majority of SFAs have made one or more technical assistance calls to their

software vendor (91 percent overall). In general, SFA staff appear to be reasonably happy with the

training and technical support provided by software purveyors. NUTRIKIDS users were noticeably less

satisfied with built-in program tutorials than with personal training or technical assistance.

Overall, SFA directors and/or data managers (the term used to refer to other SFA staff with dayto-day responsibility for use of NSMP software) using approved NSMP software are happy with the

system they are using (Exhibit 10). Eighty-seven percent rated their NSMP software as either very easy

or easy to use. Approximately 83 percent found the software to be either exceptional or satisfactory in

terms of time efficiency.

Two of the SFAs using the SNAP system were noticeably less satisfied with it than the other four

SFAs using that system. The major problem reported by these SFAs is that the program requires too

much duplicative effort and doesn't provide some of the useful, time-saving functions available in other

 

 

5

The software developer won a competitive contract to develop a generic software package for use in the training.

 

6

It should be noted that several SFAs that started out with SNAP ultimately switched to NUTRIKIDS because they

found it easier to use and more time efficient. This is true, in fact, for all three of the NUTRIKIDS sites that are fully

implemented.
Nutrient Standard Menu Planning Demonstration: Formative Evaluation Report

Abt Associates Inc. Chapter Two: Status of NSMP Implementation in Demonstration SFAs 2-17

Exhibit 7

PRIMARY REASON FOR SOFTWARE SELECTION

PRIMARY REASON NUTRIKID

S

(n = 14)

SNAP

(n = 6)

CAFS

(n = 3)

ALL SFAS

1

(n = 23)

NUMBER (PERCENT) OF SFAS

Ease of use 12 (86) 0 (0) 0 (0) 12 (52)

Already owned main program 0 (0) 2 (33) 2 (67) 4 (17)

It was only software available 0 (0) 4 (67) 0 (0) 4 (17)

System’s other (non-NSMP)

capabilities

0 (0) 0 (0) 1 (33) 1 (4)

Price 1 (7) 0 (0) 0 (0) 1 (4)

Recommended by others 1 (7) 0 (0) 0 (0) 1 (4)

1

Includes only SFAs using approved NSMP software at the time of the interview.

SOURCE: SFA director interviews (October-November, 1995).
Nutrient Standard Menu Planning Demonstration: Formative Evaluation Report

Abt Associates Inc. Chapter Two: Status of NSMP Implementation in Demonstration SFAs 2-15

Exhibit 8

PERSON RESPONSIBLE FOR USING NSMP SOFTWARE

NUMBER (PERCENT) OF SFAS

(n = 26)

SFA director 16 (62)

Nutritionist or nutrition education specialist 8 (31)

Programmer/data clerk 2 (8)

SOURCE: SFA director interviews (October-November, 1995).
Nutrient Standard Menu Planning Demonstration: Formative Evaluation Report

Abt Associates Inc. Chapter Two: Status of NSMP Implementation in Demonstration SFAs 2-16

Exhibit 9

SFAS' USE OF AND SATISFACTION WITH SOFTWARE TUTORIALS,

TRAINING AND TECHNICAL ASSISTANCE

NUTRIKIDS

(n = 14)

SNAP

(n = 6)

CAFS

(n = 3)

ALL SFAS

1

(n = 23)

NUMBER (PERCENT) OF SFAS

Use of Support Services

Tutorial 8 (57) 6 (100) N/A N/A 14 (61)

Training 11 (79) 2 (33) 2 (67) 15 (65)

Technical assistance 13 (93) 5 (83) 3 (100) 21 (91)

Satisfied with Support Services2

Tutorial35 (62) 3 (50) N/A N/A 8 (53)

Training49 (82) 1 (50) 2 (100) 12 (80)

Technical assistance512 (92) 3 (60) 2 (67) 17 (81)

Reasons for Technical Assistance Calls6

General operations issues 6 (46) 2 (40) 3 (100) 11 (52)

Recipe analysis 5 (38) 1 (20) 0 (0) 6 (29)

Printing reports 4 (31) 0 (0) 0 (0) 4 (19)

Requesting data base updates 3 (23) 0 (0) 0 (0) 3 (14)

Other 2 (15) 3 (60) 0 (0) 5 (24)

1

Includes only SFAs using approved NSMP software at the time of the interview.

2Base varies for each support service and includes SFAs that reported using the service (see first section of table.)

3Rated tutorial as either very helpful or helpful.

4

Indicated that training was sufficient to allow successful use of the software.

5Rated satisfaction with technical assistance as either very satisfied or satisfied.

6Base includes SFAs that reported using technical assistance provided by software vendor (see first section of table).

Source: SFA director interviews (October-November, 1995).
Nutrient Standard Menu Planning Demonstration: Formative Evaluation Report

Abt Associates Inc. Chapter Two: Status of NSMP Implementation in Demonstration SFAs 2-17

Exhibit 10

STAFF OPINIONS ABOUT NSMP SOFTWARE

NUTRIKID

S

(n = 14)

SNAP

(n = 6)

CAFS

(n = 3)

ALL SFAS

1

(n = 23)

NUMBER (PERCENT) OF SFAS

Ease of Use

Very easy 5 (36) 1 (17) 1 (33) 8 (35)

Easy 9 (64) 3 (50) 1 (33) 13 (57)

Difficult 0 (0) 2 (33) 1 (33) 2 (9)

Very difficult 0 (0) 0 (0) 0 (0) 0 (0)

Time Efficiency

Exceptional 5 (36) 1 (17) 1 (33) 7 (30)

Satisfactory 8 (57) 3 (50) 1 (33) 12 (52)

Somewhat inefficient 1 (7) 2 (33) 1 (33) 4 (17)

Very inefficient 0 (0) 0 (0) 0 (0) 0 (0)

1

 Includes only SFAs using approved NSMP software at the time of the interview.

Source:SFA director interviews (October-November, 1995).
Nutrient Standard Menu Planning Demonstration: Formative Evaluation Report

Abt Associates Inc. Chapter Two: Status of NSMP Implementation in Demonstration SFAs 2-18

software systems, e.g., the copying and quick search features that NUTRIKIDS offers. One of these

SFAs is seriously considering a switch to NUTRIKIDS for next school year.

In addition, one of the three CAFS sites reported being less than pleased with that software

program. Staff had encountered numerous problems with the system's performance, and the director has

been struggling with the fact that the system can only be operated by highly-skilled staff. These problems

have contributed to a substantial delay in implementation.7

Standardized Recipes

Standardized recipes are a critical component of the NSMP system. Standardized recipes are

recipes that have been tested and shown to produce consistent, high-quality products; they include

comprehensive details about specific ingredients and amounts to be used, preparation steps to be followed,

and portioning guidelines. The use of standardized recipes promotes consistency in quality and portion

sizes across schools. Moreover, they help control food service costs and contribute to accurate nutrient

analysis.

NSMP assumes consistent use of standardized recipes in all schools implementing a particular

menu. Before a menu can be analyzed, a standardized recipe must be prepared for each menu item. This

can be a time consuming process involving actual production and taste tests; recipes often go through

many iterations before a satisfactory product (recipe) is developed. 

SFA Directors were asked to estimate the proportion of recipe items for which standardized

recipes had yet to be developed; results are shown in Exhibit 11. Only nine SFAs (35 percent) reported

that all recipe items were standardized. This group was comprised of all seven of the fully-implemented

SFAs and two of the partially-implemented SFAs. More than 40 percent of SFAs still had to standardize

more than 10 percent of their recipes as of November, 1995. In six SFAs (23 percent), all of which fell

into the "not implemented" category, more than a quarter of the recipes were not yet standardized.

Menu Changes

SFA Directors were asked to identify major changes made in menus as a result of NSMP,

including menus that are still in the planning stages. Responses are presented in Exhibit 12. In keeping

with the pattern of implementation described above, some SFAs were not yet far enough along at the time

of the interview to provide meaningful responses; this was especially true for high school menus.

Among SFAs that could provide information, the most commonly reported changes, across all

school types, were increased offerings of fresh fruits and vegetables and greater use of low-fat items,

particularly low-fat entree items. Approximately one quarter of SFAs reported adding additional servings

of foods high in carbohydrates (e.g., breads or grains, fruits, etc.) to elementary school menus and

increasing portion sizes in middle and high school menus in order to meet calorie standards and/or the

standard for percentage of calories from fat.

 

 7During the Spring, 1996 data collection, the other two CAFS sites reported comparable difficulties with

the system.
Nutrient Standard Menu Planning Demonstration: Formative Evaluation Report

Abt Associates Inc. Chapter Two: Status of NSMP Implementation in Demonstration SFAs 2-19

Exhibit 11

PROPORTION OF RECIPES NOT YET STANDARDIZED

NUMBER (PERCENT) OF SFAS

(n = 26)

None (all recipes fully standardized) 9 (35)

1-10 percent 6 (23)

11-25 percent 5 (19)

26 percent or more 6 (23)

Source: SFA director interviews (October-November, 1995).
Nutrient Standard Menu Planning Demonstration: Formative Evaluation Report

Abt Associates Inc. Chapter Two: Status of NSMP Implementation in Demonstration SFAs 2-20

Exhibit 12

CHANGES IMPLEMENTED IN NSMP MENUS

CHANGE

ELEMENTARY

MENUS

MIDDLE SCHOOL

MENUS

HIGH SCHOOL

MENUS

NUMBER (PERCENT) OF SFAS

(n = 26)

More fresh fruits/vegetables 9 (35) 9 (35) 10 (38)

More low-fat items 6 (23) 8 (31) 7 (27)

Added carbohydrate foods to

increase calories

6 (23) 0 (0) 0 (0)

Larger portion sizes 0 (0) 6 (23) 6 (23)

Smaller portions of protein foods 5 (19) 4 (15) 3 (12)

More choices each day 4 (15) 4 (15) 2 (8)

More whole grains 4 (15) 3 (12) 3 (12)

Less added salt 2 (8) 2 (8) 2 (8)

Can't say yet 8 (31) 10 (38) 12 (46)

Source: SFA director interviews (October-November, 1995).
Nutrient Standard Menu Planning Demonstration: Formative Evaluation Report

Abt Associates Inc. Chapter Two: Status of NSMP Implementation in Demonstration SFAs 2-21

Meeting Nutrient Standards

SFA Directors were asked to identify problems experienced in trying to meet nutrient standards. 

All but three SFAs were able to provide some information on experience with lunch menus; all but eight of

the 24 SFAs that offer breakfast were able to comment on breakfast menus. 

The standard for which most SFAs reported problems was food energy (Exhibit 13). This was

true for both breakfast and lunch menus. Fifteen of the 23 SFAs (65 percent) that commented on lunch

and 11 of the 16 (69 percent) that commented on breakfast reported that planned menus failed to satisfy

the standard for food energy (calories) for one or more age groups. In addition, roughly three-quarters of

SFAs reported having problems satisfying the standard for the proportion of calories from fat at lunch;

more than half had problems meeting the standard for saturated fat. Problems with other nutrients were

much less common. The only other nutrients identified as problematic by more than one-quarter of the

SFAs were vitamin A (for both breakfast and lunch menus) and iron (lunch menus only).

NSMP program requirements do not specify standards for carbohydrate, cholesterol, sodium, or

dietary fiber, however, these nutrients are included in the NND-CNP and SFAs are encouraged to monitor

them. Only two SFAs reported doing so.
Nutrient Standard Menu Planning Demonstration: Formative Evaluation Report

Abt Associates Inc. Chapter Two: Status of NSMP Implementation in Demonstration SFAs 2-22

Exhibit 13

MEETING NUTRIENT STANDARDS 1

MEAL/NUTRIENT STANDARD STANDARD MET FOR

ALL AGE GROUPS

STANDARD NOT MET FOR

ONE OR MORE AGE GROUPS

Breakfast NUMBER (PERCENT) OF SFAS

(n = 16)2

Food energy 5 (31) 11 (69)

Protein 15 (94) 1 (6)

Calcium 15 (94) 1 (6)

Iron 13 (81) 3 (19)

Vitamin A 11 (69) 5 (31)

Vitamin C 15 (94) 1 (6)

Percent of calories from fat 12 (75) 4 (25)

Percent of calories from saturated

fat

13 (81) 3 (19)

Lunch NUMBER (PERCENT) OF SFAS

1

n = 233

Food energy 8 (35) 15 (65)

Protein 22 (96) 1 (4)

Calcium 21 (91) 2 (9)

Iron 17 (74) 6 (26)

Vitamin A 15 (65) 8 (35)

Vitamin C 20 (87) 3 (13)

Percent of calories from fat 8 (35) 15 (65)

Percent of calories from saturated

fat

10 (44) 13 (56)

1

Includes data reported by all SFAs that had analyzed menus, even if the menus had not yet been implemented in schools.

2

Twenty-four of the 26 active SFAs offer breakfast in some or all schools. Eight of these SFAs were excluded because they had

 not yet begun to analyze breakfast menus.

3

Three SFAs excluded because they had not yet begun to analyze lunch menus.

Source:SFA director interviews (October-November, 1995).
Abt Associates Inc. Chapter Two: Status of NSMP Implementation in Demonstration SFAs 2-1

Chapter 3

OPINIONS AND RECOMMENDATIONS REGARDING NSMP

In an attempt to gauge the acceptability of the NSMP system and to elicit information on

implementation challenges and barriers that may be amenable to corrective action, SFA Directors were

asked a number of questions that tapped personal opinions about strengths and weaknesses of the NSMP

approach. Directors were asked to share candid opinions about NSMP, in general, and the demonstration

in particular, and to offer suggestions for improvement. Finally, directors were asked to rate opinions of

key stakeholder groups (e.g., parents, teachers, students, food service workers). A comparable series of

questions was posed to cognizant State CN directors. Findings from this series of questions are

summarized in this chapter. 

SFA Directors' Opinions About NSMP

Despite the perceptions that NSMP imposes substantial demands on SFA resources, as discussed

in Chapter 2, most SFA Directors have a positive opinion about NSMP (Exhibit 14). Almost three

quarters of SFA directors reported having a very positive or somewhat positive opinion of NSMP. 

Directors of all seven of the fully-implemented SFAs and four of the partially-implemented SFAs had

very positive feelings about NSMP.

The group of SFA directors that reported feeling only somewhat positive about NSMP was

diverse with regard to implementation status and generally included directors who are very supportive of

the goal of NSMP but are feeling somewhat challenged by the work involved. The five SFA directors

who reported having a neutral, "wait and see" attitude toward NSMP includes both partially-implemented

(n=3) and not-yet-implemented (n=2) SFAs. The two SFA directors who reported having a negative

overall opinion of NSMP manage very large, partially-implemented districts. Both whole-heartedly support

the goal of NSMP but disagree strongly with a key tenet of the system, the use of weighted nutrient

analysis.

The reasons that most SFA directors feel positive about NSMP in the face of delayed

implementation schedules and larger-than-anticipated burdens on SFA staff and resources is apparent in

the list of program "bests" they generated in response to a survey question (Exhibit 14). Directors clearly

support the underlying philosophy of the NSMP system, whether they express their support in terms of

personal assurance ("I get an accurate assessment of the nutritional quality of the meals") or in the context

of protecting students' interests ("Ensures that children are offered healthy meals").

A substantial number of SFA Directors (more than forty percent) mentioned increased menu

planning flexibility as a "best" feature of NSMP. This benefit was mentioned most often by directors of

fully-implemented SFAs who cited examples such as the ability to incorporate new entree items without

having to adjust recipes to ensure that each individual serving provides two ounces of meat; being able to

decrease meat portions in combination items such as pasta dishes and stand-alone items such as peanut

butter or grilled cheese sandwiches; the ability to offer previously non-creditable items such as yogurt; and

the general freedom of not having to live within the limits of a "five component" menu. 

SFA Directors were also asked to identify the negative, or worst, aspects of NSMP. Their

responses are consistent with data discussed in preceding sections. The major negatives identified by SFA
Nutrient Standard Menu Planning Demonstration: Formative Evaluation Report

Abt Associates Inc. Chapter Three: Opinions and Recommendations Regarding NSMP 3-2

Exhibit 14

SFA DIRECTORS' PERSONAL OPINIONS ABOUT NSMP

NUMBER (PERCENT) OF SFAS

(n = 26)

Overall Opinion of NSMP

Very positive 12 (46)

Somewhat positive 7 (27)

Neutral 5 (19)

Somewhat negative 2 (8)

Very negative 0 (0)

Best Aspects of NSMP

Provides accurate assessment of nutrient content of meals 14 (54)

Provides assurance that nutritious/healthy meals are being

served

10 (38)

Increases menu planning flexibility 11 (42)

Provides an opportunity to have a positive impact on

students' eating habits

4 (15)

Allows development of menus based on specific ages of

children

3 (12)

Provides useful information for nutrition education efforts 3 (12)

Don't know 1 (4)

Worst Aspects of NSMP

Time/labor requirement 14 (54)

Weighted averages 11 (42)

Need to meet all nutrient standards 2 (8)

Wastes paper 1 (4)

Need to continually update software 1 (4)

Software can only be used by skilled staff members 1 (4)

Source: SFA director interviews (October- November, 1995).
Nutrient Standard Menu Planning Demonstration: Formative Evaluation Report

Abt Associates Inc. Chapter Three: Opinions and Recommendations Regarding NSMP 3-3

directors include the time and labor required to implement the program and the use of weighted averages. 

Some directors are resistant to the idea of weighted analysis simply because of the work involved in

collecting, organizing, and analyzing data from individual schools. Others are concerned that the use of

weighted averages will decrease flexibility at the middle and high school levels, and will require significant

changes in traditional offerings, which, in turn, may lead to decreased participation.

Attitudes and Perceptions of Key Stakeholders

SFA Directors were asked to rate the attitudes of district financial staff and school food service

staff toward NSMP. For most staff groups, more than half of SFA directors reported very positive or

somewhat positive attitudes toward NSMP (Exhibit 15). There were some reports of negative attitudes,

however.

More than a third of respondents reported that district financial staff were neutral about NSMP. 

In essence, if NSMP can be implemented without adding costs, then the administration will be supportive. 

If the system results in additional costs, however, it will immediately be seen as a problem. Three SFA

directors reported that financial staff have somewhat negative or very negative attitudes about NSMP. In

all cases, the financial authorities were concerned that program costs were increasing under NSMP. One

director reported that her administrator would not sign off on the purchase of 1.5 ounce meat portions

(rather than the 2 ounce portions required under the old meal pattern), because vendors were charging

approximately equivalent prices for both items. Looking strictly at the bottom line, rather than nutritional

considerations, the administrator could not agree to getting "less food" for the same amount of money.

Some SFA Directors also reported negative attitudes among school food service workers. 

Roughly a quarter of the demonstration sites have kitchen managers or cooks who have somewhat

negative or very negative attitudes about NSMP. SFA directors attribute these negative attitudes to two

factors: 1) increased paperwork associated with NSMP program requirements (e.g., production records,

records of leftovers and substitutions, etc.) and 2) worries that student participation will be adversely

affected because the new meals will not (or do not) taste as good as pre-NSMP meals.

Finally, SFA directors were asked whether NSMP had affected perceptions about school food

service among key stakeholders (teachers, parents, students). More than a quarter of the respondents

could not provide answers because they were not far enough along in the implementation process to have

communicated with these groups about NSMP or about changes in school food service (Exhibit 16). 

Among those who were able to respond, the majority indicated that both teachers and parents felt more

positive about school food service as a result of NSMP. Four SFA directors indicated that NSMP had had

a negative impact on teacher's attitudes about school food service. In all cases, the negative attitude was

attributed to teachers' desire for high-fat menu items that had been eliminated, modified, or offered less

frequently.

Roughly one-third of SFA directors reported that students felt more positive about school food

service; about the same number indicated that there had been no change in student attitudes. The latter

response may be influenced by the extent to which SFAs publicized food service changes to students. 

Many SFAs had not launched student information campaigns. Indeed, some were deliberately avoiding

them, believing that student-oriented publicity about healthful menu changes would act as a deterrent to

participation rather than an encouragement.
Nutrient Standard Menu Planning Demonstration: Formative Evaluation Report

Abt Associates Inc. Chapter Three: Opinions and Recommendations Regarding NSMP 3-4

Exhibit 15

FINANCIAL AND FOOD SERVICE STAFF ATTITUDES TOWARD NSMP

STAFF GROUP

VERY

POSITIVE

SOMEWHAT

POSITIVE

NEUTRAL SOMEWHAT

NEGATIVE

VERY

NEGATIVE

DK/

NA

NUMBER (PERCENT) OF SFAS

(n = 26)

District financial staff 7 (27) 7 (27) 9 (35) 1 (4) 2 (8) 0 (0)

Central (food service)

office staff

11 (42) 7 (27) 2 (8) 2 (8) 1 (4) 3 (12)

Kitchen managers 5 (19) 11 (42) 3 (12) 5 (19) 2 (8) 0 (0)

Cooks 4 (15) 11 (42) 4 (15) 5 (19) 1 (14) 1 (14)

Cashiers 3 (12) 9 (35) 7 (27) 3 (12) 2 (8) 2 (8)

DK/NA: Don’t know or not applicable.

Source:SFA director interviews (October-November, 1995).
Nutrient Standard Menu Planning Demonstration: Formative Evaluation Report

Abt Associates Inc. Chapter Three: Opinions and Recommendations Regarding NSMP 3-5

Exhibit 16

STAKEHOLDERS’ REPORTED PERCEPTIONS ABOUT SCHOOL FOOD SERVICE

SINCE BEGINNING OF NSMP DEMONSTRATION1

STAKEHOLDER GROUP

MORE

POSITIVE

MORE

NEGATIVE

NO

CHANGE

CAN'T

SAY YET

NUMBER (PERCENT) OF SFAS

(n = 26)

Teachers 12 (46) 4 (15) 3 (12) 7 (27)

Parents 12 (46) 1 (4) 6 (23) 7 (27)

Students 8 (31) 2 (8) 9 (35) 7 (27)

1

SFA directors’ reported perceptions.

Source:SFA director interviews (October-November, 1995).
Nutrient Standard Menu Planning Demonstration: Formative Evaluation Report

Abt Associates Inc. Chapter Three: Opinions and Recommendations Regarding NSMP 3-6

SFA Directors' Opinions about the NSMP Demonstration

SFA directors were asked to provide candid feedback on the NSMP Demonstration: whether they

regretted volunteering to participate in the demonstration, reasons for regrets (if any), and

recommendations for change. Responses are summarized in Exhibits 17 through 19.

While the majority (58 percent) of SFA directors have no regrets about participating in the

demonstration, 42 percent of directors did admit to regrets about getting involved in the project (Exhibit

17). For all of these directors, the staff time required to implement NSMP was the major reason for their

regret. Other reasons, mentioned by two or more SFAs, include worries that NSMP will result in more

restricted menus; lack of support and guidance from USDA; delay in software approval; and increases in

food and labor costs.

Despite these reservations, none of the demonstration SFAs would elect to return to the old menu

planning system if given the chance to do so (Exhibit 17). Directors of all 26 SFAs still active in the

demonstration would continue with NSMP. Eight directors (31 percent) would continue with the current

protocol; the remaining 18 directors (69 percent), however, would make changes to one or more aspects

of the existing program requirements. The single change mentioned most often was use of weighted

averages. Sixty-one percent of the directors who would make changes in NSMP requirements would

eliminate weighted averages; another 17 percent would use them only as a monitoring tool.

SFA directors were asked to identify ways in which USDA could have facilitated successful

implementation of NSMP for the purposes of the demonstration. Not surprisingly, timely approval of

software systems and provision of a more complete data base were the two factors cited most often, by

46 percent and 42 percent of SFA Directors, respectively (Exhibit 18). More than one third of the

directors indicated that two-way communication between SFAs and USDA could have been better. 

Directors expressed frustration that they "never knew what was going on" with software approval,

availability of new standardized recipes, data base updates, and the potential impact of new program

regulations. Approximately one in five directors indicated a need for more guidance about how to organize

and sequence the large number of tasks involved in getting NSMP started. Several of these respondents

suggested that the software include "blueprints," i.e., ready-made menus and recipes that could be adapted

by users.

SFA directors were also asked to offer suggestions on steps USDA might take to improve the

demonstration during its final year (SY 1996-97). Better communication, mentioned by half of all

directors, topped the list of suggested improvements (Exhibit 19). Those who offered this suggestion

uniformly included both communication from USDA to demonstration SFAs and communication across

demonstration SFAs. Many SFA directors described a sense of isolation and expressed a desire to talk

with others about implementation problems and solutions and to have the opportunity to share both

information and resources. Several directors suggested that USDA organize one or more group meetings

for staff of demonstration SFAs.8 

 

 

8

Since the time these data were originally collected and communicated to USDA, staff from all demonstration SFAs

have attended regional culinary skills workshops sponsored by USDA. 
Nutrient Standard Menu Planning Demonstration: Formative Evaluation Report

Abt Associates Inc. Chapter Three: Opinions and Recommendations Regarding NSMP 3-7

Exhibit 17

SFA DIRECTORS' OPINIONS ABOUT PARTICIPATION

IN THE NSMP DEMONSTRATION

NUMBER (PERCENT)

OF SFAS

(n = 26)

Do you have any regrets about volunteering to participate in the NSMP demonstration?

Yes 11 (42)

No 15 (58)

Reason(s) for regrets1

Labor requirement 11 (100)

Worry that NSMP may result in more restrictive menu 4 (36)

Lack of support/guidance from USDA 2 (18)

Delay in software approval 2 (18)

Increased food costs 2 (18)

Increased labor costs 2 (18)

If you had a choice to return to the old menu planning system or continue with NSMP, which

would you do?

Continue with NSMP (as is) 8 (31)

Continue with NSMP but make changes 18 (69)

Return to old menu planning system 0 (0)

Recommended changes in NSMP2

Eliminate weighted averages 11 (61)

Use weighted averages only as a monitoring tool 3 (17)

Provide start-up data base that includes fully-analyzed recipes and menus 2 (11)

Use less-stringent nutrient standards 2 (11)

Use traditional rules for Offer-versus-Serve (OVS)31 (6)

Require only one menu for each school level 1 (6)

1Base includes SFA directors who have regrets about NSMP participation (n = 11). Percentages total to more than 100 because

 respondents could give more than one response.

2Base includes SFA directors who recommended making changes in NSMP (n = 18). Percentages total to more than 100 because

 respondents could give more than one response.

3OVS rules dictate how many items a meal must include in order to be considered reimbursable.

Source: SFA director interviews (October-November, 1995).
Nutrient Standard Menu Planning Demonstration: Formative Evaluation Report

Abt Associates Inc. Chapter Three: Opinions and Recommendations Regarding NSMP 3-8

Exhibit 18

SFA DIRECTORS' COMMENTS ABOUT WHAT USDA COULD HAVE

DONE TO MAKE NSMP IMPLEMENTATION EASIER

NUMBER (PERCENT) OF SFAS

(n = 26)

Approve software earlier or adjust implementation schedule to reflect

delay in software approval

12 (46)

Provide better/more complete database 11 (42)

Communicate with sites more often and in a two-way fashion 9 (35)

Provide more specific step-by-step implementation guidelines/start-up

materials

5 (19)

Facilitate networking and sharing among demonstration sites 4 (15)

Provide software free-of-charge 2 (8)

Nothing 2 (8)

Source: SFA director interviews (October-November, 1995).
Nutrient Standard Menu Planning Demonstration: Formative Evaluation Report

Abt Associates Inc. Chapter Three: Opinions and Recommendations Regarding NSMP 3-9

Exhibit 19

SFA DIRECTORS’ RECOMMENDATIONS FOR FUTURE

IMPROVEMENTS IN THE NSMP DEMONSTRATION

NUMBER (PERCENT) OF SFAS

(n = 26)

Improve communication 13 (50)

Improve data base 9 (35)

Eliminate or simplify weighted averages 9 (35)

Provide funds to cover labor, food, software costs 4 (15)

Organize group meeting(s) of demonstration sites 3 (12)

Approve (currently unapproved) software 3 (12)

Provide in-depth training/technical assistance to SFAs that need it 3 (12)

Extend timeline for implementation 2 (8)

Provide more standardized recipes 2 (8)

Increase saturated fat standard 1 (4)

Be more aggressive in efforts to avert demonstration drop-outs 1 (4)

Don't know 7 (27)

Source: SFA director interviews (October-November, 1995).
Nutrient Standard Menu Planning Demonstration: Formative Evaluation Report

Abt Associates Inc. Chapter Three: Opinions and Recommendations Regarding NSMP 3-10

Improving the nutrient database and dropping or simplifying weighted averages were the next

most common suggestions, each offered by more than a third of SFA directors. With regard to the

nutrient database, a primary concern was the need to add nutrient information for nationally available

commercial products at the local level. This process results in a substantial amount of redundant effort

because SFAs across the country have to enter the same foods. Moreover, it invites data entry errors and

other data quality problems. Many NUTRIKIDS users were confused about whether it was acceptable to

use the commercial products nutrient data base the software manufacturer has added to the NND-CNP. 

Several respondents who were particularly knowledgeable about nutrition and dietary data analysis

expressed concerns about the accuracy of the nutrition data supplied by vendors and, in some cases,

included in the NUTRIKIDS data base. A related concern is that many SFA directors are not able to

detect errors in nutrient values provided by vendors because of their lack of experience with nutrition data.

 The general feeling among these SFA directors is that USDA should be building the data base and

maintaining its integrity and that only a minimal number of items should be added at the local level.

State Directors' Experiences and Opinions

The 26 SFAs still active in the demonstration are spread across 19 different States. Most States

have one demonstration district, none has more than two. CN Directors in each of these States completed

a brief interview that queried their experiences with NSMP, acquired through site visits or telephone

communication with staff in demonstration SFAs. The interview also tapped State agency directors'

perceptions about monitoring under NSMP and potential impacts at the State level. Finally, CN Directors

were asked to provide candid information on their personal opinions about NSMP as well as their

assessment of attitudes toward NSMP among non-demonstration SFAs.

Sixty-three percent of State directors reported having SFA-initiated telephone contact with

demonstration SFAs (Exhibit 20). Issues discussed during these conversations were almost universally

related to implementation, ranging from questions about software approval and acquisition (42 percent) to

the need for general reassurance that the district would be able to implement NSMP successfully (10

percent). Four State directors indicated that they, or staff in their office, were very involved in assisting

SFAs with implementation.

At the time formative evaluation interviews were conducted, fewer than half of the State agencies

had actually visited the NSMP districts(s) in their State (Exhibit 21). Of the eight States that had visited

an NSMP site, all but three (62 percent) had at least one concern about what was observed during the

visit. The most common concern was that SFAs were going to have trouble dedicating the amount of

staff time and resources required to implement NSMP within the demonstration time frame. This

observation is certainly consistent with views expressed by SFA directors. Other concerns mentioned by

State CN directors included the need for additional staff training and the apparent inadequacy of food

production records maintained by SFA staff.

State directors have their own concerns about NSMP implementation. Virtually all Directors

indicated some uncertainty about how SFAs would be monitored under NSMP. Eight-four percent

believe, however, that monitoring procedures will have to be changed for NSMP sites (Exhibit 22). 

Moreover, they believe that these changes will increase the time required for monitoring visits and, in most

cases, the number of staff required to complete monitoring visits on a regular cycle. In addition,
Nutrient Standard Menu Planning Demonstration: Formative Evaluation Report

Abt Associates Inc. Chapter Three: Opinions and Recommendations Regarding NSMP 3-11

Exhibit 20

SFA-INITIATED CONTACTS WITH STATE AGENCIES

NUMBER (PERCENT) OF

STATE OFFICES

(n=19)

Contacted by one or more demonstration SFAs 12 (63)

Issues discussed1

Software approval/demonstration time line 8 (42)

Implementation/protocol requirements 6 (32)

“Everything”; State office very involved in implementation 4 (21)

Data base content/update 3 (16)

Concerns re: ability to meet staff/resource requirements 3 (16)

General reassurance 2 (10)

1

Base includes State offices that reported contact with one or more demonstration SFAs (n = 12). Percentages total to more than

 100 because respondent could report more than one discussion issue.

SOURCE: State director interviews (October - November 1995).
Nutrient Standard Menu Planning Demonstration: Formative Evaluation Report

Abt Associates Inc. Chapter Three: Opinions and Recommendations Regarding NSMP 3-12

Exhibit 21

STATE VISITS TO DEMONSTRATION SFAS

NUMBER (PERCENT) OF

STATE OFFICES

(n = 19)

Visited one or more demonstration SFAs 8 (42)

Concerns raised during site visits1

Potential problem meeting staffing requirements 5 (62)

Need for increased staff training 3 (38)

Inadequate/incomplete production records 4 (50)

Questions about appropriateness of implementation 2 (25)

None (no concerns) 2 (25)

 1 Base includes State offices in which staff had visited one or more demonstration SFAs (n = 8). Percentages total to more than

100 because respondents could report more than one concern.

Source:State director interviews (October - November, 1995).
Nutrient Standard Menu Planning Demonstration: Formative Evaluation Report

Abt Associates Inc. Chapter Three: Opinions and Recommendations Regarding NSMP 3-13

Exhibit 22

STATE DIRECTORS' EXPECTATIONS FOR MONITORING UNDER NSMP

NUMBER (PERCENT) OF STATE

OFFICES

(N=19)

Do you expect a need for change in current monitoring procedures?

Yes 16 (84)

No 1 (5)

Don't know 2 (11)

Expected procedural changes1

Need to conduct nutrient analysis 5 (31)

Need to check accuracy of data input 6 (38)

Need to examine nutrient analysis 6 (38)

Need to spend more time on-site 2 (12)

Other 2 (12)

Don't know 2 (12)

Are your current resources sufficient for monitoring under

NSMP?

Yes 6 (32)

No 10 (53)

Don't know 3 (16)

Expectation re: time requirement for monitoring under NSMP

Will need more time 16 (84)

Will need same amount of time 1 (5)

Don't know 2 (11)

Expectation re: staffing requirement for monitoring under NSMP

Will need more staff 14 (74)

Will need same number of staff 4 (21)

Don't know 1 (5)

Expectation re: staff with requirement for nutrition/computer

expertise

Will need more staff with this expertise 16 (84)

Will need same number of staff 2 (11)

Don't know 1 (5)

1

Base includes State directors who expect a need for change in SFA monitoring procedures under NSMP (n = 16). Percentages

 total to more than 100 because respondents could cite more than one change.

Source: State director interviews (October-November, 1995).
Nutrient Standard Menu Planning Demonstration: Formative Evaluation Report

Abt Associates Inc. Chapter Three: Opinions and Recommendations Regarding NSMP 3-14

most State CN Directors envision a need to use different types of staff for monitoring visits: specifically,

staff who have nutrition and/or computer expertise.9

As a group, State CN directors are somewhat less enthusiastic about NSMP than directors of

SFAs participating in the demonstration. While 46 percent of SFA directors feel very positive about

NSMP (see Exhibit 14), only 16 percent of State directors hold this opinion (Exhibit 23). Half of the State

directors rated their opinions as somewhat positive and another 20 percent reported a neutral opinion. The

general feeling conveyed by both of these groups was support of the NSMP concept but reservations

about the ability of SFAs to successfully implement the program. 

Almost 70 percent of State directors believe that SFAs will find NSMP implementation to be

challenging or extremely challenging. Primary reasons cited for concerns were paperwork and time

requirements (58 percent), and lack of communication from USDA (37 percent).

The final questions posed to State CN directors tapped their assessment of attitudes toward NSMP

among other SFAs in their State. According to State CN directors, other SFAs had a fairly negative

attitude toward NSMP in November, 1995 (Exhibit 24). Fifty-eight percent of surveyed State directors

said most of the SFAs in their States had a somewhat negative attitude toward NSMP; another 21 percent

of State directors reported a very negative attitude. A majority of State directors estimated that 5 percent

or less of SFAs would elect the NSMP menu planning option. Only three State directors (16 percent)

predicted that more than 20 percent of SFAs would adopt NSMP.10

Reasons cited by State directors for SFA resistance to NSMP include, in descending order, fear of

change, concerns that staff lack required skills, concerns about time/staffing requirements, "fear" of

computer technology, concerns about student acceptance, and general disagreement with the NSMP

philosophy and rationale. Some SFA directors have reportedly adopted a neutral, "wait and see" attitude

toward NSMP.

 

 

9

The Food and Consumer Service convened a task group of State and Federal personnel to prepare optional forms

and guidance to assist States in monitoring any of the updated menu planning systems, including NSMP. The draft

guidance has been provided to States for comment and will be distributed for use near the beginning of the school

year. The material has been well received, and will relieve States of the necessity for developing their own review

materials, though they may do so if they wish.

 

10Informal survey information collected by FCS Regional Offices indicates that a larger number of SFAs, in the

range of 15 to 20 percent in most States, are considering use of NSMP.
Nutrient Standard Menu Planning Demonstration: Formative Evaluation Report

Abt Associates Inc. Chapter Three: Opinions and Recommendations Regarding NSMP 3-15

Exhibit 23

STATE DIRECTORS' PERSONAL OPINIONS ABOUT NSMP

NUMBER (PERCENT)

OF STATE DIRECTORS

(N = 19)

Overall Opinion of NSMP

Very positive 3 (16)

Somewhat positive 10 (53)

Neutral 4 (21)

Somewhat negative 2 (11)

Very negative 0 (0)

Assessment of Inherent Challenge in NSMP Implementation

Extremely challenging 6 (32)

Challenging 7 (37)

Somewhat challenging 3 (16)

Not at all challenging 0 (0)

Don't know 3 (16)

Reservations/Concerns

Paperwork/time requirement unrealistic 11 (58)

Better communication needed from USDA to all levels 7 (37)

Poor timing/planning 5 (26)

Delay in software approval 5 (26)

Demo sites not representative of most SFAs 3 (16)

Worry that NSMP will affect nutritional integrity 3 (16)

Worry that NSMP will cost more 3 (16)

Other 5 (26)

Source: State director interviews (October-November, 1995).
Nutrient Standard Menu Planning Demonstration: Formative Evaluation Report

Abt Associates Inc. Chapter Three: Opinions and Recommendations Regarding NSMP 3-16

Exhibit 24

STATE DIRECTORS' OPINIONS ABOUT ATTITUDES TOWARD

 NSMP AMONG DIRECTORS OF NON-DEMONSTRATION SFAS

1

NUMBER (PERCENT)

OF STATE DIRECTORS

(N = 19)

General Attitude About NSMP

Very positive 0 (0)

Somewhat positive 1 (5)

Neutral 3 (16)

Somewhat negative 11 (58)

Very negative 4 (21)

Reasons for Less Than Very Positive Attitude

Fear of change 6 (32)

Worry that staff skill-level is inadequate 6 (32)

Worry about time/staff requirement 4 (21)

Fear of technology/not computer literate 4 (21)

Have adopted "wait and see" attitude 3 (16)

Worried about student acceptance 3 (16)

Don't support philosophy/rationale 2 (11)

Estimated Proportion Who Will Implement NSMP

5 percent or less 11 (58)

6-20 percent 3 (16)

More than 20 percent 3 (16)

Don't know 2 (11)

1

State directors’ reported perceptions about directors of non-demonstration SFAs.

Source: State director interviews (October - November, 1995).