# Presentation: 481975

## School Meal ProgramPerformance:What Do We Know?

- Alberta C. Frost, Director
- Office of Analysis, Nutrition and Evaluation
-           December 13, 2005

## What Do Policy Makers Care About?

- Program access and participation rates
- Compliance with nutrition standards
- Program integrity/erroneous payments
- Operational issues
- Finances

## Sources of Data

- Administrative Data/Reports
- Management Evaluations/CRE Reviews
- National Survey Data
- Special Research Studies

## Participation Rates and Participant Characteristics:

In SY 04/05, NSLP was operating in 94,622 public and private schools with enrollments of almost 49 million students
Over 90 percent of all public schools operate NSLP
80 percent of the NSLP schools offer SBP
Over 29 million lunches and 9 million breakfasts are served each day
About half of all lunches and three-fourths of all breakfasts are served free

- In SY 04/05, NSLP was operating in 94,622 public and private schools with enrollments of almost 49 million students
- Over 90 percent of all public schools operate NSLP
- 80 percent of the NSLP schools offer SBP
- Over 29 million lunches and 9 million breakfasts are served each day
- About half of all lunches and three-fourths of all breakfasts are served free

**Assessing Program Performance:****Access and Coverage**

**Notes:**
In SY04/05 NSLP was operating in 94,622 public and private schools with enrollment of 48,879,162.  SBP was operating in 76,290 schools with enrollment of 40,764,945.
In SY01/02, NSLP was operating in 92.5% of all public schools; SBP was operating in 73.2% of all public schools.  
In FY 2004, the average daily participation in NSLP was 29 million. (48.8% Free; 9.6% RP: 41.5% Paid).  The average daily participation in SBP was 8.9 million (73.2% Free; 9.0% RP; 17.8% Paid)

## Assessing Program Performance:Access and Coverage

| Certifications as a Percentage of Eligibles Based on SIPP Data |  |  |  |
| --- | --- | --- | --- |
| Two contiguous months, 
5-18 years of age | Ave. of July or Aug. and Aug. or Sept. |  |  |
|  | 1999 | 2001 | 2002 |
| Children Certified for Free Lunch as a Percentage of Income Eligible Children | 103% | 102% | 103% |
| Children Certified for Free Lunch as a Percentage of those Income and Categorically Eligible | 98% | 97% | 98% |
| Two contiguous months, 
5-18 years of age | Ave. of July or Aug. and Aug. or Sept. |  |  |
|  | 1999 | 2001 | 2002 |
| Children Certified for Reduced Price Lunch as a Percentage of those Income Eligible | 48% | 53% | 54% |

**Notes:**
Estimates of school-age income and adjunctively eligible children from the Census’ Survey of Income and Program Participation (SIPP) were compared to NSLP certification in order to analyze trends in free and reduced-price certifications as a percentage of eligibles.

Based on CPS data between FY93-03, the total number of free certifications as a percentage of low-income children nationally increased from 97 to 131 percent and the total number of reduced-price lunch certifications as a percentage of low-income children nationally increased from 46 to 61 percent.

## Assessing Program Performance:Access and Coverage

- Source:  Characteristics of National School Lunch and School Breakfast Program Participants (1996)

- Participation rates decrease as the student’s grade level increases:

## Students certified for free/reduce-price meals differ markedly from uncertified students in terms of age, race, and place of residence: 
Students certified for free/reduced-price meals tend to be younger and in lower grades. 
Black, Hispanic, and American Indian or Alaskan Natives are disproportionately represented in the free group. 
Most certified students live in urban or rural areas than in suburban areas, and they disproportionately resided in the Southeast and Southwest.

- Students certified for free/reduced-price meals tend to be younger and in lower grades.
- Black, Hispanic, and American Indian or Alaskan Natives are disproportionately represented in the free group.
- Most certified students live in urban or rural areas than in suburban areas, and they disproportionately resided in the Southeast and Southwest.

**Assessing Program Performance:****Access and Coverage**

**Notes:**
Based on Characteristics of National School Lunch and School Breakfast Program Participants (MPR/FNS) 1996 - uses SNDA-I data (1991-92)

Note:  We haven’t collected characteristics of program participants since SNDA-I (1991-91).  We will be updating this information through SNDA-III.

## In School Year 2004/05 an estimated 27 percent of all free approved children were directly certified.
In School Year 2004/05, enrollment in Provision 2/3 schools accounted for about 5 percent of all free approved children. 

- In School Year 2004/05, enrollment in Provision 2/3 schools accounted for about 5 percent of all free approved children.

**Assessing Program Performance:****Access and Coverage**

**Notes:**
Source: Preliminary analysis of Verification Summary Data, School Year 2004-05

## Assessing Program Performance:Nutrition Standards

**NSLP Lunches Provide 1/3 or More of Daily Nutrient Requirements**

- Source:  School Nutrition Dietary Assessment Study-II (School Year 1998-99)

**Assessing Program Performance:****Nutrition Standards**

**Target **
**For**
**Lunches:**
**33%**

**Notes:**
The average NSLP lunch served to students in SY 1998-99 exceeded the program standard of one-third of the RDA for all key nutrients, except for calories in secondary schools (30%)

## Assessing Program Performance:Nutrition Standards

**Assessing Program Performance:****Nutrition Standards**

**Notes:**
SNDA-II Results
There has been significant improvement in the levels of fat in school meals.  Schools reduced the fat in school lunches offered to students from 38 percent of calories in 91-92 to 34 percent of calories in 98-99.  Fat in school breakfasts was reduced from 31 percent to 26 percent of calories. The goal is no more than 30 percent of calories from total fat.
Schools have also reduced the levels of saturated fat in meals.  Saturated fat in school lunches dropped from 15 percent of calories in 91-92 to 12 percent in 98-99.  The percent of calories from saturated fat in school breakfasts dropped from 14 percent to 9.8 percent.  The overall goal for this indicator is less than 10 percent of calories from saturated fat.
These reductions in fat and saturated fat have been achieved while maintaining the overall nutrient contribution of school meals.  School lunches provide more than one-third of the Recommended Dietary Allowances (RDAs) for all targeted nutrients.  Meals served in secondary schools provide slightly less than one-third of the calorie recommendations (30%) while meals in elementary schools provide slightly more than one-third (35%).  Breakfasts met or exceeded the target of providing one-fourth of the RDA for targeted nutrients , with the exception of calories.
Lunches and breakfasts offered in 98-99 were significantly lower in cholesterol and sodium than those offered in 91-92.  Both lunches and breakfasts met the National Research Council recommendation for cholesterol.  The recommendation for sodium was met by breakfasts, but not by lunches.
The number of schools meeting standards in all lunches offered has increased dramatically.   Only one percent of schools offered lunches that met the 30 percent DGA guideline for fat and none met the less than 10 percent guideline for saturated fat in 91-92.  In 98-99, 18 percent of elementary schools and 22 percent of secondary schools met the standard for total fat while 15 percent of elementary schools and 17 percent of secondary schools met the standard for saturated fat.
Students did not always take every type of food offered to them.  However, meals offered and selected exceeded the RDA standards for key nutrients.   Lunches in secondary schools and breakfasts in both elementary and secondary schools contained fewer calories than the standard. 

**Assessing Program Performance:****Nutrition Standards**

**School Nutrition Dietary Assessment III**
- Agency’s periodic assessment of the nutritional effects of school meals.
- Study examines:
- SFA characteristics/operations
- Nutritional quality of meals offered/served
- Participant characteristics
- Student dietary intakes and the contribution of school meals to these intakes
- Data was collected during SY 2004/05 from 135 public SFAs, about 400 schools, and 2,400 students
- Final report expected in Fall 2006

## 	Analysis of data from the 1994-1996 Continuing Survey of Food Intake by Individuals (CSFII) found that NSLP Participants:
have higher intakes of food energy, vitamin B6, vitamin B12, thiamin, riboflavin, calcium, phosphorous, magnesium,  and zinc, but also higher mean intakes of total fat, saturated fat and sodium, both at lunch and through the day.
are more likely than nonparticipants to consume vegetables, milk and milk products, and meat and meat substitutes, both at lunch and through the day.
consume less soda and fruit drinks and fruit flavored drinks at lunch than nonparticipants.
have lower intakes of added sugars at lunch and through the day than nonparticipants.

- 	Analysis of data from the 1994-1996 Continuing Survey of Food Intake by Individuals (CSFII) found that NSLP Participants:
- have higher intakes of food energy, vitamin B6, vitamin B12, thiamin, riboflavin, calcium, phosphorous, magnesium,  and zinc, but also higher mean intakes of total fat, saturated fat and sodium, both at lunch and through the day.
- are more likely than nonparticipants to consume vegetables, milk and milk products, and meat and meat substitutes, both at lunch and through the day.
- consume less soda and fruit drinks and fruit flavored drinks at lunch than nonparticipants.
- have lower intakes of added sugars at lunch and through the day than nonparticipants.

**Notes:**
Note: NHANES 1999 and beyond are not representative of school days.

##   Analysis of data from the 1994-1996 Continuing Survey of Food Intake by Individuals (CSFII) found that SBP Participants:
have higher intakes of food energy, calcium, phosphorous, and vitamin C at breakfast and through the day.
are more likely than nonparticipants to consume fruit and milk both at breakfast and through the day. 

-   Analysis of data from the 1994-1996 Continuing Survey of Food Intake by Individuals (CSFII) found that SBP Participants:
- have higher intakes of food energy, calcium, phosphorous, and vitamin C at breakfast and through the day.
- are more likely than nonparticipants to consume fruit and milk both at breakfast and through the day.

**Assessing Program Performance:****Program Integrity/Erroneous Payments**

**Certification Accuracy Research Conclusions:**
- Certification of ineligible children is a problem – 18% of all students certified for free meals were found ineligible in the Pilot Project.
- The NSLP verification process selects about 93,000 students who appropriately should have their benefits reduced or terminated – 25% of the total group verified.
- Errors are found throughout the administrative process in these small studies (SY2001/02)
- Initial certification (6% error)
- Verification (8% error)
- Change in meal ticket status (17% of verified applications in error)
- Certification of reapplying non-responders (5% re-applications in error)

**Notes:**
Based on small non nationally representative studies:
Evaluation of NSLP Application/Verification Pilot Projects
Case Study of Metro Area Verification Outcomes
Administration of NSLP F/RP Eligibility Determination (ROVA Study)

**Assessing Program Performance:****Program Integrity/Erroneous Payments**

**NSLP/SBP Access, Participation, Eligibility and Certification Study (APEC) – (MPR/FNS)**
- The Improper Payments Act of 2002 requires USDA to identify and reduce erroneous payments in NSLP, SBP, etc.
- This nationally-representative study will examine erroneous payments attributable to misclassification of students (administrative error, household misreporting) and meal counting and claiming errors.
- On-site data collection will occur in SY 2005/06
- A final report is expected in 2007

**Assessing Program Performance:****Program Integrity/Erroneous Payments**

**Regional Office Review of Applications (RORA)**
- Objective: to estimate the rate of administrative error in SFA eligibility determinations for free/reduce-price meal benefits.
- Nationally representative sample of 56 SFAs
-    (8 SFAs per FNS region)
- Random sample of about 50 applications per SFA collected by Regional staff in SY 2004/05
- Report expected in December 2005.
- Similar data collection to continue in future years

## Assessing Program Performance:Program Integrity/Erroneous Payments

- SFAs made incorrect eligibility determinations on 3.5% of approved/denied applications at the time of certification.
- The percentage of applications in error was slightly higher (4.2%) for income-based applications only.
- 83 percent of the incorrectly certified applications resulted in students being certified for more benefits than were justified.

**Preliminary RORA 2005 Results**

**Assessing Program Performance:****Program Integrity/Erroneous Payments**

**SFA Verification Summary Report (FNS-742)**
- State Agencies must submit an annual report to FNS on the results of verification activities for each SFA under its jurisdiction by April 15th.
- FNS-742 data elements include enrollment, application and eligibility information as well as results of verification by application type (categorical application, income/household size application)
- Preliminary data analysis conducted on SY 2004/05 data received from 47 of 57 Child Nutrition State Agencies.

**Notes:**
Preliminary results based on data submitted from 47 CN State Agencies.  Data from Delaware, Massachusetts, New York, North Carolina, Rhode Island as well as Guam, Puerto Rico, and the Virgin Islands were not included in the analysis.  

Note: data from all States except North Carolina have been submitted to CN. 

## Preliminary SFA Verification Summary ResultsSchool Year 2004/05

**Notes:**
The results of the analysis of the verification summary data for SY 2004-2005 indicate the following: 
Of the free and reduced price students certified by application based on categorical or income eligibility:
Students receiving free meal benefits based on categorical eligibility were least likely to have a change in their meal status (of those that responded).
Students receiving reduced price meal benefits were most likely to have a change in their meal status (of those that responded).
Reduced price meal applications selected for verification had the highest rate of non-response; categorically eligible free meal applications had the lowest.

Almost 24 percent of children approved for free meals through categorical eligibility who did not respond to verification requests reapplied and were re-approved; 17 percent of income eligible children reapplied and were re-approved, and only 7 percent of reduced price non-responders reapplied and were re-approved.  The total number of students who reapplied, regardless of whether they were re-approved or not is not known. 

## Preliminary SFA Verification Summary ResultsSchool Year 2004/05

| Summary of Verification Outcomes by Sampling Method Used |  |  |  |  |
| --- | --- | --- | --- | --- |
| Verification Outcomes | All Districts | Districts That Used Random Sampling | Districts That Used Focused Sampling | Districts That Used All Applications |
| No Change | 67.2%      | 70.0% | 41.1% | 87.0% |
| Changed to Reduced Price | 4.5% | 3.8% | 9.4% | 2.0% |
| Changed to Paid | 5.4% | 4.8% | 9.4% | 3.1% |
| Changed to Free | 1.5% | 1.5% | 1.8% | 0.7% |
| No Response | 21.5% | 19.9% | 38.2% | 7.2% |

**Notes:**
 As shown in Table 1, across all the districts, about 11 percent of all the certified applicants who responded to the verification process had their benefit status changed. Among SFAs utilizing random sampling and those that verified all applications, there was a change in meal status for 10.1 percent and 5.8 percent of the certified applications, respectively.  Those SFAs utilizing focused sampling had the highest rate of change in meal status at 20.6 percent.  These rates of change do not take in to account those who did not respond to verification requests regardless of whether they reapplied and were subsequently re-approved.  Non-responders accounted for almost 20 percent of all randomly selected applications, 38.2 percent of the focused applications, and 7.2 percent of applications in which all were selected for verification. 

Applications selected for verification through focused sampling had the highest rate of non-response (38 percent) and the highest rate of change in meal status (21 percent).

## Assessing Program Performance:Operational Issues- Competitive Foods

**	****School Venues Where Food is Sold or Offered**:
- School dining room
- Vending machines and school stores
- Parties and classroom snacks
- Concession stands
- After school programs
- Fundraising activities
- Staff and parent meetings

**Notes:**
SNDA-II Data indicates that:
92% of all schools have a la carte foods available at lunch and 36% of all schools have it available at breakfast.
Vending machines are much more common in middle schools (55%) and high schools (76%) than in elementary schools (15%).  The same is true with school stores and canteens.
Aside from milk, the most common a la carte offerings were: juice (44%); cookies (41%); pizza (33%); ice cream (36%); crackers, granola bars, and pretzels (33%); Fresh fruit (32%); Snack chips (32%); juice drinks (30%); bagels (27%); baked desserts (27%); French fries (27%) 

School Health Policies and Programs Study (SHPPS 2000) – CDC
20% of States require and 20% of States recommend schools prohibit junk foods as a la carte items during lunch or breakfast.
26% of elementary schools, 62% of middle schools and 95% of high schools have one or more vending machines at the school from which students can purchase food or beverages.
27% of elementary schools, 39% of middle schools, and 59% of high schools have a school store, canteen, or snack bar where students can purchase food or beverage.

## Direct Certification:
In School Year 2003-04, an estimated 75 percent of public SFAs used direct certification.  
Provision 2/3:
In School Year 2003-04, less than 10 percent of public SFAs had Provision 2 or 3 schools;  

**Direct Certification:**
- In School Year 2003-04, an estimated 75 percent of public SFAs used direct certification.
**Provision 2/3:**
- In School Year 2003-04, less than 10 percent of public SFAs had Provision 2 or 3 schools;

**Notes:**
Source:  Unreleased SFA Characteristics Report tables from NSLP Sample Frame Study

Provision 2/3

SMI Implementation Study:  Third Year Report
In SY 1999/00 about 517 (3.9%) of public school districts reported schools within their district operating under Provision 2 or 3.  Nationwide there were an estimated 3,154 public schools (3.8%) using these approaches with most of them (89.1%) using Provision 2.

CN Survey of State Agencies – Fiscal Year 2000 (voluntary reporting)
548 SFAs with provision 2 or 3 schools;  4,046 schools (88% using Provision 2)

Note:  Based on unreleased SFA Characteristics Report from NSLP Sample Frame Study

Preliminary analysis of the SFA Verification Summary data suggest that in School Year 2004/05 about 27% of the students certified for free meals were directly certified, and enrollment in Provision 2/3 schools accounted for about 4.5 percent of the students approved for free meals.

## Universal School Breakfast:

Availability of universal-free school breakfast significantly increased school breakfast participation but had little impact on other outcomes measures over the course of the evaluation including academic achievement test scores, attendance, tardiness, health, and discipline.
Offering free school breakfast to all elementary school students would not, on average, be expected to improve academic or behavior outcomes beyond what occurs in schools already offering SBP.

**Universal School Breakfast:**
- Availability of universal-free school breakfast significantly increased school breakfast participation but had little impact on other outcomes measures over the course of the evaluation including academic achievement test scores, attendance, tardiness, health, and discipline.
- Offering free school breakfast to all elementary school students would not, on average, be expected to improve academic or behavior outcomes beyond what occurs in schools already offering SBP.

## Nutrition Education:
Team Nutrition Training Grants were awarded to 21 States in 2005.
54 Gold and 3 Silver Awards given in the first year of the HealthierUS School Challenge
Food Stamp Nutrition Education often coordinated with NSLP/SBP and TN
Public schools are the primary site location for direct nutrition education

**Nutrition Education:**
- Team Nutrition Training Grants were awarded to 21 States in 2005.
- 54 Gold and 3 Silver Awards given in the first year of the HealthierUS School Challenge
- Food Stamp Nutrition Education often coordinated with NSLP/SBP and TN
- Public schools are the primary site location for direct nutrition education

**Notes:**
Clear links between FSNE and other USDA-funded programs: 65% of the implementing agencies coordinated with NSLP; 58% with Team Nutrition and 49% with SBP. 

The vast majority of implementing agencies used direct education as a delivery method (91%).  81% of the implementing agencies provided direct education in public schools, generally using multiple group sessions with an educator.

Source: Implementing Agency Survey – FSNE Evaluation

## Assessing Program Performance:Finances

- Last study to examine the cost to produce reimbursable meals in NSLP/SBP was conducted in SY 1992/93
- Major findings of the School Lunch and Breakfast Meal Cost Study include:
- On average, SFAs operate at the break-even level, with total revenues about equal to total reported costs.
- Revenues from reimbursable meals exceed the cost of producing those meals.  Reimbursable lunches generate a revenue surplus that is used to offset losses from reimbursable breakfasts
- SFAs also subsidize non-program food service (e.g. a la carte) with surplus revenues from reimbursable lunches.
- Revenues from reimbursable meals (including government subsidies and student payments) accounted for an average of 85 percent of total SFA revenues.

## Food and Labor Account for Most Program Costs

**Food Costs**
**Local Food Purchase**
**USDA Donated Commodities**
**Labor Costs**
**Production and Food Service Labor**
**Administrative Labor**
**Other Costs**
**Supplies**
**Capital Expenditures/Depreciation**
**Contracted Services**
**Indirect Charges**

- Source: School Lunch and Breakfast Cost Study, October 1994

**Food**
**$0.79**

**Labor**
** ****$0.71**

**Other Costs**
**$0.13**

**Mean Cost**
**Per Lunch - $1.63**

**Assessing Program Performance:****Finances**

**Notes:**

The following chart provides information on the reported costs attributable to the production of a reimbursable lunch.  These reported costs include only those costs that SFAs are expected to cover from revenues generated from food service sales and government reimbursements. However, reported costs often do not reflect the cost of all resources attributable to food service operations.  Nearly all school districts incur some costs in support of food service operations that are not charged to the School Food Service Account.

Reported costs account for about 81 percent of the full cost of food service operations in SY 1992-93.
Three line items accounted for an average of 90 percent of unreported costs in SY 1992-93.
labor (44%) (administrative support, custodial services, teacher supervision in the cafeteria, etc.)
unreported indirect costs (27%) 
and equipment depreciation (16%)

On a full cost basis, food costs accounted for an average of 38% of food service costs, labor costs accounted for 46% of food service costs, and other costs accounted for the remaining 16 percent of food service costs.

## Student Payments and USDA SubsidiesMake Up Most Program Revenue

**Assessing Program Performance:****Finances**

**Notes:**
Revenues from reimbursable meals (including government subsidies, donated commodities and student payments) accounted for an average of 85 percent of total SFA revenues.

## Assessing Program Performance:Finances

**School Lunch and Breakfast Meal Cost Study - II**
- FNS is in the process of awarding a contract to update the School Lunch and Breakfast Meal Cost Study using the same methodology used in the 1992/93 Study.
- Data collection is expected to be conducted in Spring 2006 and Fall 2006.
- A final report is expected in 2007

## Fresh Fruit and Vegetable Program

- Became a permanent program through the Child Nutrition and WIC Reauthorization Act of 2004.
- $9 million authorized each year
- 25 schools in each of 8 States and 25 schools among 3 ITOs – 225 schools total
- In FY 2005/06 schools allocated $81 per student (smaller schools received $100 per student)

## States and ITOs Participating in the

**States and ITOs Participating in the**
**Fruit and Vegetable Program**

- 2002-03 States
- 2002/03 ITO
- 2003/04 CDC State
- 2005/06 States
- 2005/06 ITOs

- WA

- AZ

- NM

- SD

- IA

- IN

- OH

- MI

- PA

- NC

- MS

**Notes:**

2002 Farm Bill States:
Original 2002/03 States – Indiana, Iowa, Michigan and Ohio
Original 2002/03 ITO – Zuni Pueblo Indian Tribal Organization of New Mexico

2003/04 CDC State – Mississippi

States added with the CN and WIC Reauthorization Act of 2004
2004/05 States – North Carolina, Pennsylvania, Washington
2004/05 ITOs – Intertribal Council of Arizona, Oglala Sioux of the Pine Ridge Reservation in South Dakota

## Fresh Fruit and Vegetable Program

- Interim reports submitted to Congress in each of fiscal years 2005 through 2008.
- Reports describe activities carried out during the fiscal year (acceptability, delivery methods, timing of service delivery, educational activities, most popular fruits and vegetables, etc.)
- CDC’s evaluation of Mississippi’s fruit and vegetable pilot program examined the impact of the program on fruit and vegetable consumption.

**Notes:**
CDC’s evaluation featured a one-group (no control) pretest-posttest design involving students in grades 5, 8, and 10 from 5 of the 25 schools participating in the pilot.  The draft report summarizes data from a self-report student survey (n=725) and 24 hour dietary recall interviews (administered to 8th and 10th grade students only, n=207)

Results show:
Greater levels of exposure to fruit and vegetables at all grade levels
The development of specific preferences fro fruit among 8th and 10th grade students;
Positive attitudes toward eating fruit among 8th grade students;
Increased consumption of fruit, but not vegetables among 8th and 10th grade students (.34 servings in school; .61 servings overall);  vegetable consumption in school actually decreased by .62 servings but vegetable consumption overall remained the same.

## USDA NEEDS YOU

**Assessing Program Performance:****Participation in Research Studies**

## What are the burning questions for you?
What information would help you run a better program?

- What are the burning questions for you?
- What information would help you run a better program?

- Questions and Comments