Economic

Research

Service

Electronic Publications from the Food Assistance & Nutrition Research Program

Direct Certification in the National

School Lunch Program—Impacts 

on Program Access and Integrity

Final Report

By Philip Gleason, Tania Tasse, Kenneth Jackson, 

and Patricia Nemeth

ERS project representative: Linda Ghelfi

Abstract

About 61 percent of school districts used direct certification in the National School Lunch

Program (NSLP) in the 2001-02 school year, the same share as in 1996. Direct certification

increased the number of children certified for free meals by about 400,000 and slightly

increased overall NSLP participation. Under direct certification, school districts use information from State welfare or food stamp offices to certify children to receive free meals. To

qualify, the children’s families must receive food stamps, Temporary Assistance for Needy

Families, or assistance from the Food Distribution Program on Indian Reservations. Directly

certified children’s families do not have to complete certification applications. Direct certification was designed primarily to improve NSLP access and administrative efficiency. This

report presents the findings of a study on direct certification’s prevalence, its implementation

methods, and its effects on NSLP access and integrity.

About the Authors

Gleason, Tasse, and Nemeth are with Mathematica Policy Research, Inc. Jackson is with

Decision Information Resources, Inc.

E-FAN-03-009

October 2003

This report was prepared by Mathematica Policy Research, Inc., under

a research contract from the Economic Research Service. The views

expressed are those of the authors and not necessarily those of ERS

or USDA.
ACKNOWLEDGMENTS 

Many people contributed in significant ways to the preparation of this report. Linda Ghelfi has 

been the project officer throughout the stages of the project in which most of the analysis and 

report writing has been conducted. Her comments and insights throughout this process have 

been valuable. Joshua Winicki was the project officer at the Economic Research Service (ERS) 

throughout the study design and data collection stages of the study as well as during the early 

stages of analysis, and also provided guidance and helpful comments throughout this period. In 

addition, David Smallwood, Dean Jolliffe, and others at ERS and Paul Strasberg, Todd Barrett, 

and Matthew Sinn of the Food and Nutrition Service all provided useful input to the study design

and analysis. John Burghardt of Mathematica Policy Research (MPR) has been an invaluable 

consultant on the project, giving useful guidance on the study design and comments on the 

analysis and report drafts that have clarified many of the trickiest issues.

A number of additional MPR staff provided valuable assistance on the study. John Hall and 

Bidisha Ghosh developed an efficient sample design and carefully selected the sample and 

constructed sample weights. We are grateful to Amy Raduzycki, who supervised the telephone 

center interviewing, along with Kristin Quitoni, Walter Williams, and Rob Samardyck, who 

assisted with survey development and operations. The survey could not have been conducted 

successfully without the outstanding work of the many telephone interviewers who worked on it. 

We received assistance with the collection of State administrative data from Julius Clark. 

Shephali Patel skillfully provided most of the necessary programming support help in analyzing 

the survey and State administrative data. Josh Hart and Nazmul Khan also helped with the

programming. Roy Grisham carefully edited the report, and Marjorie Mitchell skillfully

produced it. 

Finally, we would like to thank the many school food authority (SFA) directors who participated 

in our survey and patiently answered our numerous questions. Many of these SFA directors had 

conscientiously prepared in advance for the interviews by completing the fact sheet we had sent

them prior to our call. We would also like to thank the State food stamp administrators and their 

staff who provided useful State-level data. We gratefully acknowledge these many contributions 

and accept sole responsibility for any remaining errors or omissions in the report. 

 Philip Gleason 

 Tania Tasse

 Kenneth Jackson

 Patricia Nemeth

ii
CONTENTS

Chapter Page 

 EXECUTIVE SUMMARY..........................................................................................v 

I INTRODUCTION........................................................................................................1 

 

A. DIRECT CERTIFICATION AND CERTIFICATION ERRORS: 

 POLICY BACKGROUND............................................................................................. 2 

 B. STUDY METHODS....................................................................................................... 8 

1. Direct Certification Prevalence and Implementation ............................................ 9 

2. The Impact of Direct Certification on Program Access........................................ 9 

3. Effects of Direct Certification on Program Integrity........................................... 10 

II DIRECT CERTIFICATION PREVALENCE AND IMPLEMENTATION.............11 

 

A. PREVALENCE OF DIRECT CERTIFICATION........................................................ 12 

B. DIRECT CERTIFICATION IMPLEMENTATION TYPES ....................................... 20 

C. KEY OPERATIONAL ASPECTS OF THE DIRECT 

 CERTIFICATION PROCESS...................................................................................... 26 

1. Districts’ Perceptions of Implementation Problems............................................... 26 

2. Do Districts Directly Certify as Many Students as Possible? ................................ 30 

3. Do Districts Feel that Direct Certification Has Benefits? ...................................... 34 

 D. SUMMARY.................................................................................................................. 34 

III CERTIFICATION AND PARTICIPATION EFFECTS OF 

 DIRECT CERTIFICATION ......................................................................................36 

 

 A. METHODS ................................................................................................................... 36 

1. District-Level Analysis ............................................................................................ 36 

2. State-Level Analysis ................................................................................................ 45 

B. EFFECTS OF DIRECT CERTIFICATION ON CERTIFICATION AND 

 PARTICIPATION RATES........................................................................................... 50 

1. District-Level Analysis ............................................................................................ 50 

2. State-Level Model.................................................................................................... 63 

3. Summary .................................................................................................................. 71 

iii
CONTENTS (continued) 

Chapter Page 

IV ESTIMATED INELIGIBILITY AMONG CERTIFIED STUDENTS......................74 

 A. INELIGIBILITY MEASUREMENT ........................................................................... 75 

1. Ineligibility Among Students Certiffied by Application.......................................... 76 

2. Ineligibility Among Students Directly Certified Students ....................................... 81 

3. Ineligibility Among All Certified Students.............................................................. 83 

B. THE VERIFICATION PROCESS AND RESULTS.................................................... 84 

1. The Income Verification Process............................................................................. 84 

2. Results of Income Verification ................................................................................ 87 

3. Alternative Estimates of Ineligibility Among Students Certified by Application.... 92 

 

C. ESTIMATING INELIGIBILITY AMONG 

DIRECTLY CERTIFIED STUDENTS...................................................................... 100 

1. Limitations ............................................................................................................. 100 

2. Estimated Rates of Ineligibility Among Directly Certified Students..................... 103 

D. ESTIMATED INELIGIBILITY AMONG ALL CERTIFIED STUDENTS.............. 106 

V EFFECTS OF DIRECT CERTIFICATION ON INELIGIBILITY RATES ...........110 

A. DISTRICT-LEVEL MODEL OF THE EFFECTS OF DIRECT 

CERTIFICATION ON INELIGIBILITY................................................................... 112 

B. ESTIMATED EFFECTS ON INELIGIBILITY AMONG 

ALL CERTIFIED STUDENTS.................................................................................. 113 

C. ESTIMATED EFFECTS ON INELIGIBILITY AMONG STUDENTS 

 CERTIFIED BY APPLICATION .............................................................................. 123 

 VI SUMMARY .............................................................................................................128 

 REFERENCES.........................................................................................................133 

 

APP. A DESCRIPTION OF DATA COLLECTION............................................................A.1 

 

APP. B ANALYSIS OF NON-RESPONSE ON THE SFA SURVEY 

AND STATE ADMINISTRATIVE DATA COLLECTION EFFORTS ................B.1 

 

APP. C SUPPLEMENTAL TABLES FOR ANALYSIS OF THE EFFECTS OF 

 DIRECT CERTIFICATION ON RATES OF CERTIFICATION, 

PARTICIPATION, AND INELIGIBILITY AMONG CERTIFIED STUDENTS .C.1 

APP. D ANALYSIS OF DATA FROM THE SURVEY OF INCOME 

AND PROGRAM PARTICIPATION (SIPP)..........................................................D.1 

iv
EXECUTIVE SUMMARY 

An important aspect of the National School Lunch Program (NSLP) is that low-income children 

can receive lunches free or at a reduced price. Those living in families with incomes of 130 

percent or less of the Federal poverty guideline—or who receive food stamps, Temporary 

Assistance for Needy Families (TANF), or assistance from the Food Distribution Program on

Indian Reservations (FDPIR)—qualify for free meals. Those living in families with incomes 

between 131 and 185 percent of poverty qualify for reduced-price meals. 

The U.S. Department of Agriculture (USDA) introduced the policy of direct certification for free 

meals in the late 1980s. Previously, all families who wished their children to receive free or 

reduced-price meals had to complete an application and provide data on either family size and 

income or receipt of food stamp, TANF, or FDPIR (FS/TANF/FDPIR) benefits. School officials 

then determined whether families met eligibility requirements. Under direct certification,

information from the State food stamp or welfare agency is used to directly certify children 

receiving FS/TANF/FDPIR benefits without requiring them to complete certification 

applications.

Direct certification was designed primarily to improve program access and administrative 

efficiency. If existing data from State food stamp or welfare offices were used to directly certify 

children, fewer eligible children may fail to become certified for free meals. And if the need for 

these children to complete applications and for district officials to process these applications

were eliminated, administrative costs could be reduced. Finally, direct certification might also

improve program integrity. Promoting program access among this group could increase the 

proportion of certified students eligible for the level of benefits they are receiving because 

FS/TANF/FDPIR recipients are eligible for free meals by definition. 

This report summarizes the findings of the Direct Certification Study. The primary objectives of 

the study involved examining the prevalence and consequences of direct certification. In 

particular, we set out to estimate the prevalence of direct certification and describe its methods of 

implementation. The study was also designed to estimate the impact of direct certification on 

program access and program integrity. To examine program access, we estimated the impact of 

direct certification on rates of NSLP participation and certification for free/reduced-price meals. 

To examine program integrity, we first estimated the rate of ineligibility among certified 

students, and then estimated the impact of direct certification on the ineligibility rate. 

The study relied on two major data sources to address these objectives: a survey of school 

districts and administrative data collected from State food stamp/welfare offices. The survey 

collected information on district/foodservice characteristics from a nationally representative 

sample of 1,223 public school districts offering NSLP lunches. Administrative data were 

collected from 37 States on the FS/TANF status of school-aged children from these same 

districts at two points in time during 2001. 

v
How Prevalent Is Direct Certification? 

As of the 2001-02 school year, 61 percent of public school districts used direct certification. 

Because larger districts were more likely to use direct certification than smaller districts, just

over two-thirds of students attended districts using direct certification. This prevalence of direct 

certification among districts was about the same in 2001-02 as it was in 1996, at the time of a 

previous study of direct certification (Jackson, et al. 1999). 

The prevalence of direct certification can also be examined by estimating the percentage of 

students who are directly certified. Among students certified for free meals in the average direct 

certification district, one in four is directly certified. Among all students certified for free meals 

(including those in districts not using direct certification), 18 percent are directly certified.

Among all students (including those not certified for free meals) in all districts, 6 percent are

directly certified. 

In addition to those students who are directly certified, some students certified for free meals are 

“categorically eligible,” meaning that they became certified by application on the basis of a 

reported FS/TANF/FDPIR case number. In the average direct certification district, about 18 

percent of students certified for free meals fall into this category (in addition to the 25 percent 

directly certified). It is possible that many of these categorically eligible students could have 

been directly certified but were missed by the system for some reason. To better understand how 

this type of gap in direct certification coverage could have arisen, the Direct Certification Study 

examined districts’ implementation of direct certification. 

Means by Which Students in Direct Certification Districts Were

Certified for Free Meals

25%

18%

58%

Directly

Certified

Categorically

Eligible

Certified on

Basis of

Income

vi
How Do Districts Implement Direct Certification? 

The 1996 Study of Direct Certification developed a typology of direct certification in which 

districts using the policy were categorized on the basis of how they identified students to be

directly certified and how these students’ direct certification status was triggered. This typology 

included three main types of direct certification districts—(1) non-matching districts, (2) districtlevel matching districts, and (3) State-level matching districts—along with districts with mixed 

type characteristics (Jackson, et al. 1999). 

Direct certification is implemented through some sort of matching in most districts—41 percent 

use district-level matching and 27 percent use State-level matching. The prevalence of matching 

has increased since 1996, when 34 percent used district-level matching and 19 percent used 

State-level matching (Jackson et al., 1999). The previous study found that nearly all matching

districts used passive consent—that is, FS/TANF/FDPIR students were automatically directly 

certified unless they explicitly declined the benefit. We found that by 2001 some matching 

districts had begun to require active consent—that is, with FS/TANF/FDPIR students directly 

certified only if they explicitly accepted the benefit. 

Districts’ use of matching has led to some implementation problems. Overall, nearly half of 

direct certification districts reported having households with children who were directly certified 

while their siblings were not. Districts that relied on matching with passive consent were 

particularly likely to report this problem. Nearly one-third of districts reported problems in 

matching children’s names with their parents. Such problems could be part of the reason that 

substantial numbers of eligible children are “missed” by the direct certification system. 

DIRECT CERTIFICATION TYPOLOGY

• Non-Matching Districts: FS/TANF/FDPIR households are sent letters (typically by a

State agency) notifying them of their children’s potential eligibility for direct

certification. Active consent is required, whereby students must bring these 

notification letters to school to trigger direct certification. 

• District-Level Matching Districts: A list of enrolled children is matched by the 

district against a list of children on FS/TANF/FDPIR to determine who is eligible for 

direct certification. Typically, passive consent is used, whereby no further action is 

required for these children to be directly certified. 

• State-Level Matching Districts: A list of enrolled children is matched by a State 

agency against a list of children on FS/TANF/FDPIR to determine who is eligible for 

direct certification. Typically, passive consent is used, whereby no further action is 

required for these children to be directly certified. 

• Mixed Type Districts: These districts share characteristics of more than one of the

above direct certification implementation types.

vii
Does Direct Certification Affect Program Access? 

A key aim of direct certification is to improve access to the program among students eligible for 

free meals. To assess the success of direct certification in achieving this aim, we estimated the 

impact of direct certification on rates of certification and NSLP participation. We used 

regression techniques to estimate these impacts. In particular, we estimated a district-level 

ordinary least squares (OLS) and a State-level fixed effects model to determine the policy’s

impacts on certification and participation. The findings from this analysis follow: 

• Direct certification leads to an increase in the percentage of students certified for

free meals. Both the district-level and State-level models indicate that direct 

certification leads to a positive and significant impact on the free certification rate. 

The size of the effect is 1.3-1.4 percentage points, implying that the policy leads to an

increase of about 400,000 students certified for free meals. 

• Direct certification appears to lead to a small increase in NSLP participation. Both

models indicate that direct certification has a positive effect on the participation rate,

though only the State-level estimate is statistically significant. The increase in this

rate arises from an increase in participation among students certified for free meals. 

Estimated Effects of Direct Certification

on Certification and Participation Rates

1.3

0.5

1.4 1.5

0

0.5

1

1.5

2

Free Certification Rate NSLP Participation Rate

Percentage Points

District-level model State-level model

** * *

* Statistically significant at the 0.10 level.

** Statistically significant at the 0.05 level.

viii
Does Direct Certification Affect Program Integrity? 

To estimate the effects of direct certification on program integrity, we first had to come up with 

some way of estimating program integrity. To get at program integrity, we estimated the rate of

income ineligibility among certified students—the proportion of children certified for free and 

reduced-price meals in the fall whose family circumstances in December (income, household 

size, and FS/TANF/FDPIR status) made them ineligible for the benefits they were receiving.

Students were considered income ineligible for free meals, for example, if their income exceeded

130 percent of poverty and they did not receive FS/TANF/FDPIR. Students from Provision 2 or 

3 schools were excluded from this analysis, so the estimated rates of ineligibility among certified 

students apply only to those in non-Provision 2 or 3 schools. 

A key challenge in estimating ineligibility among certified students was that we had to rely on 

different sources of information and methods of estimating ineligibility among two groups of 

certified students. For students certified by application, we relied on each district’s report of the

results of its verification process to estimate ineligibility. For directly certified students, who are

not covered by the verification system, we used state administrative data coupled with a

supplemental analysis of data from the Survey of Income and Program Participation (SIPP) to 

estimate ineligibility. We then combined the resulting estimates of ineligibility among these two

groups to estimate the proportion of all certified students ineligible for benefits. 

Understanding the limitations of these data is important to interpreting our findings. First, the 

verification process itself is likely to be subject to errors from several sources. One problem is 

that districts using random sampling to select cases for verification may not have selected a truly 

random sample, although the available evidence suggests that districts’ procedures approximate 

PROVISION 2 AND 3 SCHOOLS 

Provisions 2 and 3 are alternatives to the traditional procedures for determining the 

numbers of and reimbursements for free, reduced-price, and paid meals served by a 

school. Provision 2 is currently the more common arrangement and has been an option

for schools since 1980, though it has become much more commonly used in recent years. 

Under Provision 2, schools must serve meals at no charge to all students for a four-year 

period. During the first, or base, year, a Provision 2 school determines meal price 

eligibility status as usual and counts the number of meals served, by meal type. The

school may or may not use direct certification during the base year. During the 

subsequent three years, the school makes no new eligibility determinations, counting 

only the total number of meals served. Reimbursements during these years are

determined by applying the percentages of free, reduced-price, and paid meals served

during the base year to the number of total meals served during the current year.

These schools were excluded from our analysis of direct certification's effects on 

program integrity because they do not assess the eligibility of students for free and

reduced price meals during non-base years. 

ix
the results of a scientifically random sample.1

 It is also possible that districts may not have 

accurately determined the income eligibility of the households selected for verification. And,

verification information provided by households may in some cases misrepresent income. 

Another limitation of using verification results to estimate ineligibility is that many households 

selected for verification do not respond to the request for income or FS/TANF/FDPIR 

documentation. Districts conducting verification are required to terminate the benefits of these 

households. However, the actual circumstances of the households are not known: They may or

may not be income eligible for benefits. Thus, rather than estimating a single income ineligibility 

rate, we estimated upper and lower bounds of income ineligibility. The lower bound estimate 

assumed that all nonresponding households remain eligible for free or reduced-price meal 

benefits. The upper bound estimate assumed that nonresponding households were ineligible 

unless they reapplied and were approved for benefits subsequent to the verification process. We

believe that the true rate of income ineligibility lies between these bounds.2

 

Not withstanding these limitations, data from the verification process (along with administrative 

data on directly certified students) offer useful insights into issues of program integrity and the 

potential effects of direct certification on integrity. 

Among free/reduced price students in the average district, we estimated a lower bound of 12 

percent to be income ineligible as of December for the level of benefits they are receiving. This 

lower bound estimate assumes that those who fail to respond to the verification request are 

income eligible. We estimated an upper bound of 20 percent income ineligible, assuming that 

those who fail to respond to the verification request are income ineligible unless they reapplied

and were approved for benefits. 

Ineligibility is more common in larger districts, so estimated rates of ineligibility among all

certified students are higher than estimated rates of ineligibility among certified students in the 

average district. Among free/reduced price students, regardless of what district they attend, we 

estimated a lower bound of income ineligibility as of December of 12 percent, again assuming 

that all nonresponders were income eligible. We estimated an upper bound of 33 percent, again 

assuming that those who fail to respond to the verification request are income ineligible unless 

they reapplied and were approved for benefits. 

 1

 This was the conclusion of the most recent national study of income verification in the NSLP 

(USDA 1990). 

2

 Abt Associates conducted the most recent national study of income verification in the NSLP during

the 1986-87 school year for FNS. That study provides some insight about ineligibility among families that 

do not respond to the verification request (USDA 1990). In that study, 10 percent of those selected for

verification did not respond. Household audits of these nonresponders found that 67 percent remained

income eligible for the benefits they were approved for; 18.7 percent were income ineligible for NSLP

benefits; and 14.3 percent were eligible for reduced-price meals even though they had been certified for

free meals. The districts from which these estimates were computed all used random sampling to conduct

verification. While this study was conducted in a small sample of districts and before direct certification

was available to districts, it suggests that some share of the nonresponders to verification requests in 2001

remained income eligible. 

x
Ineligibility is rare among directly certified students, with only 7 percent estimated to be

income ineligible. While over 20 percent of directly certified students have stopped 

receiving FS/TANF by December of the school year, most of these FS/TANF leavers remain 

eligible for free meals on the basis of their income. 

To determine how direct certification affects program integrity, we estimated the impact of direct

certification on those estimates of income ineligibility among certified students in non-Provision 

2 and 3 schools. We found that: 

• Direct certification leads to a decrease in the rate of ineligibility among certified

students. Estimates from each specification of the model suggested that the income

ineligibility rate is lower in direct certification districts than in districts that do not use 

direct certification. 

• The magnitude of the estimated effect of direct certification on ineligibility varies in

alternative specifications. The specification of the model in which districts were 

weighted equally showed a large negative effect of direct certification on the rate of 

income ineligibility. In an alternative specification in which districts were weighted 

by their numbers of certified students, direct certification was estimated to have a 

much smaller negative effect on income ineligibility. 

• Despite leading to a decrease in income ineligibility among certified students, direct 

certification is related to higher benefit reduction/termination rates. The benefit 

reduction/termination rate—the percentage of verified applications in which benefits

are reduced or terminated—tends to be higher in direct certification districts than in

non-direct certification districts. This effect arises because the benefit 

reduction/termination rate is based on a verification sample limited to students 

certified by application—thus excluding directly certified students, who are less

likely than students certified by application to be ineligible. 

Should Direct Certification Be Expanded? 

Given the evidence indicating that direct certification improves both program access and 

program integrity, it is not surprising that a large proportion of districts use the policy. However, 

nearly 40 percent of districts do not use direct certification, and the prevalence of direct 

certification has not grown since 1996. Furthermore, even in districts that use the policy, a 

substantial number of students who could be directly certified seem to be missed by the system. 

Thus, evidence from this study suggests that expanding direct certification would have benefits 

and that there is room for such expansion. 

So, how could such expansion be promoted? Improving districts’ ability to implement direct 

certification could lead to an increase in the number of students directly certified within districts 

using the policy and might also make direct certification a more attractive option for districts not 

currently using it. A key part of improving this implementation will involve improving the 

process by which districts match information on which households are receiving

FS/TANF/FDPIR with lists of students enrolled in the district. Although this study identified 

problems with the matching process as a key implementation issue, it did not address ways of 

xi
successfully addressing this issue. Further research on “best practices” for conducting this

matching would be useful. 

Other implementation issues cited by some districts involved resource constraints and working 

with FS/TANF agencies. Thus, additional policy options for making direct certification more 

attractive to districts and expanding its coverage within districts might involve improving 

interagency coordination and communication and providing small grants or technical assistance

to districts that implement direct certification. 

xii
I. INTRODUCTION 

The U.S. Department of Agriculture (USDA) enacted the National School Lunch Program

(NSLP) in 1946 to “safeguard the health and well-being of the nation’s children and to 

encourage the domestic consumption of nutritious commodities and other foods.” The program 

provides Federal financial assistance and commodities to schools serving lunches that meet 

required nutrition standards. Children living in families whose incomes are 130 percent or less 

of the Federal poverty guideline—or whose families receive food stamps, Temporary Assistance 

for Needy Families (TANF), or assistance from the Food Distribution Program on Indian 

Reservations (FDPIR)—qualify for free meals. Those living in families whose incomes are 

between 131 and 185 percent of the Federal poverty guideline qualify for reduced-price meals. 

All other children pay full price, although full-price lunches are also Federally subsidized by a 

small amount. 

In the early 1990s, the USDA introduced the policy of direct certification to streamline the 

process of determining the eligibility of some children for free school meals. Previously, outside 

of Provision 2 and 3 schools (described below), all families who wished their children to receive

these benefits—that is, to be certified for free or reduced-price meals—had to complete an 

application and provide data on either family size and income or receipt of food stamp, TANF, or 

FDPIR (FS/TANF/FDPIR) benefits. School officials then determined whether families met 

eligibility requirements and certified for benefits those students who qualified for free or

reduced-price meals. Under direct certification, information from the welfare/food stamp 

agencies in a State is used to directly certify FS/TANF/FDPIR recipients, without requiring these 

students’ families to complete certification applications. 

1
This report describes the findings of the Direct Certification Study, which examines the 

practice of direct certification in public school districts nationally. This study has three major 

objectives, each addressed in this report. The first is to estimate the prevalence of direct 

certification and describe how districts implement the policy. The second is to estimate the 

effects of direct certification on program access—its effects on certification and participation in

the NSLP. And, the third is to estimate the effects of direct certification on program integrity--its 

effects on the extent to which students approved for free or reduced-price meals—either by 

application or through direct certification—are ineligible for the benefits they are receiving. 

This last objective requires that we first estimate rates of ineligibility. 

The remainder of this chapter describes the policy background that provides a context for

the study (Section A) and gives a brief overview of the study’s methodology for addressing each 

of the objectives (Section B). Chapter II reports on the prevalence of direct certification and 

methods school districts use to implement it. Chapter III presents estimates of the effects of

direct certification on certification and participation. Chapter IV explains our estimates of 

administrative and income ineligibility. Finally, Chapter V presents our estimates of the effects 

of direct certification on ineligibility of certified students for the benefits they are receiving. 

A. DIRECT CERTIFICATION AND CERTIFICATION ERRORS: POLICY 

BACKGROUND 

The NSLP is a Federal program that benefits all children who eat school lunches meeting 

nutritional requirements in participating schools. The program subsidizes, in the form of cash 

reimbursements and commodities, all school lunches served to children of all income levels. The 

level of Federal subsidy, however, depends upon the income levels of the children who consume

the meals. The largest subsidy goes for meals served to children certified for free meals—those 

living in families whose incomes are 130 percent or less of Federal poverty guidelines. The 

2
subsidy is slightly lower for meals served to children certified for reduced-price meals—those 

living in families whose incomes are between 131 and 185 percent of the guidelines. Finally, a 

small subsidy is provided for meals served to all other children—those who pay “full price” for 

their meals. 

The NSLP is administered by the USDA’s Food and Nutrition Service (FNS) through its 

regional offices. The national FNS office also provides technical assistance to State agencies. 

State agencies administer fiscal elements of the program, provide technical assistance to local 

school food authorities (SFAs), and monitor their performance. The SFAs administer the 

program locally, typically at the district level. In addition to ensuring that program meals meet 

nutritional requirements, SFAs are responsible for processing applications, conducting 

verifications, counting meals served for purposes of claiming reimbursement, and maintaining 

records. 

Like all programs that use means tests to direct government benefits to low-income

households, the NSLP must balance several competing objectives: (1) targeting, or providing 

free or reduced-price meals to intended recipients while not providing them to children who do 

not qualify; (2) administrative efficiency, or holding down the costs of gathering and processing 

data to administer the program; and (3) access, or ensuring ease of access for intended recipients. 

Meeting the first objective by better targeting of benefits to only the intended recipients may 

compromise the other two objectives by raising administrative costs and creating barriers to

access among intended recipients. On the other hand, efforts to promote access to the program

among intended recipients may also lead to greater access among those not eligible for program 

benefits. 

Various NSLP policy initiatives that have been implemented over the years have been 

developed as a response to research findings or anecdotal evidence that the program was

3
deficient in meeting one or more of the above program objectives. Thus, the initiatives have

attempted to achieve one or more of these objectives without greatly compromising on the other 

objectives. For example, a 1980-1981 study by the USDA’s Office of the Inspector General 

(OIG 1981) found that a large fraction (29 percent) of students receiving free or reduced-price

meals were in families whose incomes did not qualify them for the benefits they were receiving. 

Partially in response to this finding, Congress enacted Public Law 97-35 in August 1981, which 

required households to provide more detailed information about household members and 

household income on their application for free or reduced-price meals, and led to the requirement 

that SFAs verify the eligibility of a fraction of their approved free or reduced-price applications. 

These verification requirements remain in place today. SFAs must verify either (1) a 

random sample of the lesser of 3 percent or 3,000 approved applications, or (2) a focused sample 

of both (a) the lesser of 1,000 or 1 percent of all approved applications (selected from those with 

reported monthly incomes within $100 of the income threshold for receiving free or reducedprice meals), plus (b) the lesser of 500 or 0.5 percent of applications among those approved on 

the basis of FS/TANF/FDPIR benefit receipt (USDA 2001).3

 To verify the selected applications, 

SFAs request documentation from the households that submitted these applications to confirm

their eligibility for free or reduced-price meals. This documentation may cover all current 

sources of household income and/or the household’s current FS/TANF/FDPIR status. If the 

household does not provide sufficient documentation by a specified date, their free or reducedprice benefits are terminated.4

 Similarly, if the income or FS/TANF/FDPIR documentation

provided by the household indicates that they are not eligible for the level of benefits they are

 3

SFAs also have the option of verifying a random sample larger than the minimum requirements or 

even verifying all approved applications. 

4

These households are free to reapply for benefits and provide documentation at a later date. 

4
currently receiving, their benefit level is adjusted accordingly. SFAs are required to complete 

the verification process by December 15 of each school year.5

 They also are required to 

maintain a description of their verification efforts and results, but are not required to report these 

results to State or Federal authorities on a regular basis.6

In the mid-1980s, the USDA’s Food and Nutrition Service (FNS) contracted for the Study of 

Income Verification in the NSLP to determine how SFAs were implementing the congressional 

mandate for conducting verification and to examine the results of verification during the 1986-

1987 school year. The study found that about 20 percent of verified cases had benefits reduced 

or terminated, roughly half for not providing the required information and half because the 

verified income was too high to qualify for benefits (USDA, FNS 1990). The study noted that 

the latter situation could arise either because a family had misreported its circumstances on their 

original application or because the family had experienced, but not reported, a change in its 

circumstances between the time they completed the application and the time their application

was verified. Subsequent analysis suggested that most of the observed error was due to changes 

in circumstances rather than misreporting on the initial application (St. Pierre and Puma 1992). 

The policy of direct certification was introduced in the late 1980s, primarily to improve 

program access and administrative efficiency. By using existing data from State food stamp or

welfare offices to directly certify students from FS/TANF/FDPIR households, it was thought that 

fewer eligible children would fail to become certified for free meal benefits. And by eliminating 

the need for these children to complete applications and for these applications to be processed by 

 5

Waivers are occasionally granted exempting districts from this December deadline. In the 2001-

2002 school year, for example, New York City was exempted from the December deadline due to the

disruptions caused by the September 11 terrorist attacks. 

6

However, this information may be requested by State or Federal authorities as part of a review of 

SFA activities. 

5
SFAs, it was thought that administrative costs could be reduced. Because direct certification 

relies on information provided by food stamp or welfare agencies rather than on information 

provided by families, directly certified families are excluded from verification requirements. 

Thus, the policy of direct certification was designed to improve the accuracy of targeting, in

addition to the goals of promoting access and efficiency. 

After initial testing in a few pilot sites, all districts offering school meals were given the 

option of using direct certification. In the mid-1990s, FNS contracted for a study of direct 

certification, for the purpose of determining how widespread its use had become, assessing its 

implementation at the State and district levels, and estimating its impact on program access. The

study found that direct certification was being used in just under two-thirds of public school 

districts serving nearly three-fourths of the nation’s students (Jackson et al. 1999). Moreover, 

State and local officials in districts using direct certification reported doing so because it

simplified program administration and made the program more accessible to students. The study 

also found that direct certification had significant positive effects on districts’ rates of free meal 

certification and NSLP participation.

The 1996 study of direct certification did not address the issue of how direct certification has 

influenced the targeting of free and reduced-price meal benefits to intended recipients. This is an 

important issue because direct certification has led to a situation in which the income verification

system no longer covers all certified students in direct certification districts, since directly 

certified students are not subject to verification. Further, while regulations call for directly 

certified students to report any changes in their FS/TANF/FDPIR status to district authorities, the

regulations did not set up a system by which information from State food stamp or welfare 

agencies would automatically be used to notify districts when children leave assistance. 

6
So direct certification may have affected targeting in two ways. First, it may have led to 

changes in the results of districts’ verification efforts even without any change in the underlying 

eligibility of students, because direct certification changes the pool of students in a given district 

from which the verification sample is selected. Second, at a given point during the school year, 

directly certified students themselves could be ineligible for the free meal benefits they are 

receiving. This would happen, for example, if they remained directly certified even after they 

stopped receiving FS/TANF/FDPIR and if the income of their household rose above 130 percent 

of Federal poverty guidelines. No previous study, however, has generated a national estimate of 

this error rate among directly certified students. 

The issue of children being certified in error—that is, being certified even though they are 

ineligible for benefits—has received much recent attention. A 1997 study by the OIG found 

substantial errors in certification in an audit of SFAs in Illinois covering the 1994-1995 and 

1995-1996 school years. Tordella (2001) found evidence of a large and growing problem of 

overcertification. He found, for example, that by 1999 the number of children certified for free

meals was 28 percent larger than an estimate (based on data from the Current Population Survey) 

of the number whose annual income was 130 percent or less of Federal poverty guidelines.7

 By

contrast, the number of certified children was only 18 percent greater than the number whose 

annual income was 130 percent or less of poverty in 1997 and 5 percent greater in 1994. 

Another relevant policy initiative affecting the administration of the NSLP and certification

for free and reduced-price meals was the initiation of Provisions 2 and 3, which are alternatives 

to the traditional procedures for determining the numbers of and reimbursements for free, 

 7

This measure is an imprecise indicator of overcertification, however, since it is likely that some 

households whose annual incomes were greater than 130 percent of poverty may had incomes less than 

130 percent of poverty in the month in which they applied for benefits. 

7
reduced-price, and paid meals served by a school. These provisions are designed to reduce 

administrative burden. Provision 2 is currently the more common arrangement and has been an 

option for schools since 1980, though it has become much more commonly used in recent years. 

Under Provision 2, schools must serve meals at no charge to all students for a four-year period. 

During the first, or base, year, a Provision 2 school determines meal price eligibility status as 

usual and counts the number of meals served, by meal type. The school may or may not use 

direct certification during the base year. During the subsequent three years, the school makes no

new eligibility determinations, counting only the total number of meals served. Reimbursements 

during these years are determined by applying the percentages of free, reduced-price, and paid 

meals served during the base year to the number of total meals served during the current year. 

B. STUDY METHODS 

The two major sources of information used in this study are a survey of SFAs and 

administrative data collected from State food stamp and welfare agencies. Conducted in early 

2002, the SFA survey was based on a nationally representative sample of 1,223 SFAs that offer 

NSLP lunches. The survey collected data on districts’ enrollment, certification and participation 

rates, implementation of direct certification, and information on the features and results of their 

income verification system. The overall response rate to the survey was 81 percent. State 

administrative data were collected from 37 States on the FS/TANF status of school-age children 

at two points during 2001. Appendix A provides additional information on the collection of 

these two sources of data for the study. An overview of the methodology used to address each 

study objective is provided below. 

8
1. Direct Certification Prevalence and Implementation 

This report provides estimates of the proportion of public school districts nationwide (that

offer NSLP lunches) that were using direct certification as of the 2001-2002 school year, as well 

as the proportion of students nationwide that attended direct certification districts. Another

perspective on the prevalence of direct certification (as well as its implementation) involves 

estimating the proportion of those students attending direct certification districts who actually are 

directly certified, as well as the proportion who are certified by application. The report also 

examines the methods by which districts implement direct certification and estimates the

proportions using each of the implementation types identified as part of the 1996 Study of Direct 

Certification. Each of these issues was addressed using data from the SFA survey. To explore 

changes in the prevalence and implementation of direct certification, these survey results were 

compared with a similar set of results from the 1996 study. 

2. The Impact of Direct Certification on Program Access 

The second objective addressed in this report is to estimate the impact of direct certification 

on rates of certification for free and reduced-price meals and NSLP participation. Estimates of 

the impact of direct certification on certification and participation rates address the issue of

whether the policy of direct certification improves access to program benefits. 

We used regression techniques to estimate each of these impacts of direct certification.

Methods from the 1996 study were replicated to estimate impacts on certification and 

participation. In particular, we used two alternative models to generate these estimates—a 

district-level ordinary least squares (OLS) regression model and a State-level fixed effects 

model. 

9
3. Effects of Direct Certification on Program Integrity 

The third objective is to estimate the effects of direct certification on program integrity, as

estimated by its impact on rates of ineligibility of free and reduced-price certified students for the 

benefit levels they are receiving. These estimates will inform the discussion of how the policy of 

direct certification influences the extent to which free and reduced-price meal benefits go to

intended recipients. To be able to estimate this effect of direct certification, we first had to 

estimate rates of ineligibility among free and reduced-price certified students. 

We also used the district-level OLS model to estimate the impact of direct certification on 

certification error rates. No State-level data on error rates were available with which to estimate 

a State-level model of error rates. Further details on procedures used to estimate ineligibility and

estimate direct certification's effects on ineligibility are provided in Chapters IV and V. 

10
II. DIRECT CERTIFICATION PREVALENCE AND IMPLEMENTATION 

Direct certification was first implemented in a few pilot sites during the late 1980s. Since 

then, regulations have permitted all districts offering NSLP lunches to use direct certification if 

they so desire. As described in Chapter I, the use of direct certification expanded rapidly through 

1996, when nearly two-thirds of districts used the policy (Jackson et al. 1999). In Section A of

this chapter, we show that the prevalence of direct certification has remained roughly constant 

since that time. We also examine the prevalence of direct certification by measuring how widely 

direct certification actually is used to certify students for free meals in those districts that use the

policy. 

The chapter also discusses how districts conduct direct certification. The 1996 study 

developed a typology that described three major ways in which districts and States implement 

direct certification. We use a revised version of that typology in Section B to classify direct 

certification districts according to how they identify, notify, and certify those students who 

ultimately are directly certified. This revised typology allows for comparisons between the 

findings of this study with those of the 1996 study. One key trend that emerged over this period 

was that districts became more likely to use some sort of matching technology to identify 

students eligible for direct certification; however, where the 1996 study found that these 

matching districts generally used passive consent—in which parents were notified and given the 

option of refusing the benefit—we found that, by 2001, some matching districts had begun using 

an active consent process. The active consent process in these districts required parents to 

respond affirmatively in order to have their children approved for free meals by direct 

certification.

11
The final set of issues discussed in the chapter involves several key operational aspects of

direct certification. One indication of problems with the implementation of direct certification 

uncovered by this analysis is that many students in direct certification districts who are not 

themselves directly certified end up being certified by application on the basis of a food stamp or

TANF case number (a group we refer to as being categorically eligible). It is likely that many of 

these categorically eligible students could have been directly certified but for some reason were

missed by the system. We explore this issue in Section C and present districts’ responses to

direct questions regarding implementation problems and the operations of direct certification in

their districts. 

A. PREVALENCE OF DIRECT CERTIFICATION 

In the 2001-2002 school year, direct certification was used by 61 percent of public school

districts offering the NSLP (Table II.1).8

 These direct certification districts served about 68 

percent of all students enrolled in public school districts, as well as the same proportion of 

students certified for free meals. The prevalence of direct certification changed little between the

1996-1997 and 2001-2002 school years. In fall 1996, an estimated 63 percent of districts serving 

72 percent of students used direct certification, estimates that are not statistically significantly

different from the 2001 estimates. 

 8

A few districts classified as non-direct certification districts may have actually used the policy. For

example, nine districts in the Northeast region of the United States reported that they did not use direct

certification even though they were mandated to use the policy by their states. If these nine districts are

re-classified as direct certification districts, the prevalence of direct certification increases to 62.6 percent

among districts, 67.8 percent among enrolled students, and 68.5 percent among certified students. In

addition, New York City reported that they did not use direct certification in 2001-2002 (even though

they had used it in previous years and have used it in subsequent years) because of problems in 

identifying children who were eligible to be directly certified. If New York had used direct certification 

in that school year, the percentage of enrolled students in direct certification districts would have 

increased from 67.8 to 70.2 percent. 

12
TABLE II.1

PREVALENCE OF DIRECT CERTIFICATION AMONG PUBLIC SCHOOL DISTRICTS 

PARTICIPATING IN THE NSLP AND AMONG STUDENTS AT THESE SCHOOLS 

(Standard Errors in Parentheses)

1996 Public

School Districts 

2001 Public 

School Districts 

Percentage of Districts Using Direct 

Certification 63.0 61.0

 (1.81) (1.73) 

Percentage of Students in Districts Using 

Direct Certification 

 Percentage of All Enrolled Students 71.9 67.5 

 (2.94) (1.44) 

 Percentage of Free Certified Students 71.5 68.2 

(4.33) (2.07)

Sample Size 996 1,218 

Source: 2001 Direct Certification Study SFA Survey; Jackson et al. (1999). 

Note: Significance tests were conducted; none of the differences in the percentages presented above

between 1996 and 2001 were statistically significant at the 0.10 level. 

13
Districts in which some or all schools received NSLP reimbursements on the basis of 

Provision 2 or 3 were considered direct certification districts if they used direct certification in

their base year. Overall, 8 percent of districts included Provision 2 or 3 schools in 2001-2002—5 

percent of districts consisted entirely of Provision 2 or 3 schools, and 3 percent had Provision 2 

or 3 in some but not all schools (Table II.2). Among districts with Provision 2 or 3 schools, just 

over one-fourth reported 2001-2002 as the base year; the remainder reported an earlier base 

year.9

 Nationally, an estimated 2.6 million students, or 6 percent of all public school students in

NSLP schools, were in Provision 2 or 3 schools during the 2001-2002 school year. Overall, the 

prevalence of direct certification in Provision 2 or 3 districts was 51 percent during 2001-2002, 

although among districts whose Provision 2 or 3 schools had a base year in 2001-2002, the 

percentage using direct certification of 59 percent was much closer to the national rate.10

The prevalence of direct certification (across all districts) can also be examined through an 

estimate of the percentage of students who are directly certified. Among students certified for 

free meals in the average direct certification district, one in four was directly certified, with the

rest certified by application (Table II.3). Among all enrolled students in the average direct 

certification district, 8 percent were directly certified. The numbers are similar when calculated

across students, rather than for the average district. Among public school students in direct 

 9

If the Provision 2 or 3 schools within a district had different base years, districts were asked to 

report the most common base year used. 

10 Since no individual students in Provision 2 or 3 schools are actually certified for free or reducedprice meals (though they all receive free meals), students in these schools are excluded from our estimates 

of rates of ineligibility among certified students presented in Chapter IV. 

14
TABLE II.2

DISTRICTS’ USE OF PROVISION 2 OR 3 AND DIRECT CERTIFICATION 

(Standard Errors in Parentheses)

Percentage or 

Number 

Percentage of Districts that Include: 

 Provision 2 or 3 in any schools in district 8.4 (1.00)

 Provision 2 or 3 in all schools in district 4.9 (0.78)

Percentage of Provision 2 or 3 Districts with a Base Year in: 

 2001-2002 28.8 (5.31)

 2000-2001 or earlier 71.2 (5.31)

Number of Public School Students Nationally in Provision 2 or 3 Schools 

 (millions of students) 2.60 *

Percentage of Public School Students Nationally in Provision 2 or 3 Schools 5.6 *

Percentage of Districts Using Direct Certification Among: 

 Districts with Provision 2 or 3 in any schools in district 50.6 (5.83)

 Districts with Provision 2 or 3 in all schools in district 42.1 (7.97)

 Provision 2 or 3 districts with a base year in 2001-2002 58.5 (10.97)

 Provision 2 or 3 districts with a base year prior to 2001-2002 47.4 (6.84)

Sample Size 1,218

Source: 2001 Direct Certification Study SFA Survey. 

*Standard errors were not calculated for the student estimates. 

15
TABLE II.3

PERCENTAGE OF STUDENTS WHO ARE DIRECTLY CERTIFIED 

(Standard Errors in Parentheses)

Direct 

Certification

Districts 

Non-Direct 

Certification

Districts 

 All 

 Districts 

In Average District: 

Percentage of certified free students 

who are directly certified 24.9 0.0 15.1 

(0.92) (0.0) (0.77)

Percentage of enrolled students who 

are directly certified 7.5 0.0 4.5 

(0.41) (0.0) (0.30)

Among Students: 

Percentage of certified free students 

who are directly certified 27.3 0.0 17.9 

(0.87) (0.0) (0.71)

Percentage of enrolled students who 

are directly certified 8.9 0.0 5.8 

(0.39) (0.0) (0.32)

Sample Size 760 426 1,191 

SOURCE: 2001 Direct Certification Study SFA Survey. 

certification districts nationally, 27 percent of those certified for free meals and 9 percent of all 

those enrolled were directly certified. Among public school students in all NSLP districts

nationally (regardless of the district’s direct certification status), 18 percent of those certified for 

free meals, and 6 percent of all those enrolled, were directly certified.11

 11In districts that include Provision 2 or 3 schools, we collected information on the direct certification

status of students as of the base year. Thus, the counts of directly certified students include some who

were directly certified prior to the 2001-2002 school year. We estimated that about 275,000 students (or 

about 10.2 percent of all directly certified students nationally) were counted as directly certified and were 

also enrolled in a Provision 2 or 3 school with a base year prior to 2001-2002. 

16
In both 1996 and 2001, more than one-third of districts did not use direct certification. Why 

have some districts implemented direct certification, while others have not? Do particular 

problems impede the implementation of direct certification, or have all districts that wish to 

adopt the policy already done so? One factor in answering the above questions involves

knowing whether the non-direct certification districts have ever used the policy. While some (17 

percent) of these districts do have previous experience with direct certification, most (83 percent) 

have never used the policy (Figure II.1). 

The most common set of reasons districts give for not using direct certification have to do 

with not really needing to use the policy. More than four-fifths of non-direct certification 

districts reported that an important reason for not using direct certification was that they were

satisfied with the procedures they were currently using to determine student eligibility (Table 

II.4). In addition, 52 percent reported that the percentage of eligible students in their district was 

too small to make direct certification worthwhile.12 Overall, 86 percent of non-direct 

certification districts cited one of these two reasons. This finding suggests that there is little 

reason to expect the majority of non-direct-certification districts to adopt the policy in the future. 

With little need for direct certification, these will likely remain non-direct certification districts

in the future unless their circumstances change. 

17

 12This reason was more commonly cited by districts that had never used direct certification (55 

percent) than by districts that had stopped using the policy (39 percent). 
Figure II.1 

Prior Use Of Direct Certification 

Among Non-Direct Certification 

Districts 

17%

83%

Used DC in Past Never Used DC

18
TABLE II.4 

REASONS DISTRICTS REPORT FOR NOT USING DIRECT CERTIFICATION 

(Standard Errors in Parentheses)

It is too difficult to get cooperation from TANF or food

stamp agencies

28.1 

(3.57) 

25.3 

(5.00) 

28.4 

(3.04) 

Either of the “TANF/FS Agency” Reasons 34.9 

(3.64) 

42.9 

(6.08) 

37.8 

(3.13) 

Concerns about how direct certification would 

compromise student confidentiality 

29.0 

(3.48) 

28.1 

(5.44) 

29.7 

(2.99) 

Sample Size

a 301 98 417

Percentage of Non-Direct Certification Districts 

Reporting the Reason as Important or Very Important

Reasons for Not Using Direct Certification

Never Used Direct 

Certification

Used Direct 

Certification in the 

Past Total 

No Need for Direct Certification

Satisfied with current procedures to determine student

eligibility 

81.3 

(2.72) 

76.2 

(5.26) 

81.1 

(2.33) 

Percentage of students eligible for free meals is too 

small to make direct certification worthwhile 

 55.4** 

(3.79) 

38.8 

(6.25) 

52.4 

(3.25) 

Either “No Need” Reason 87.4 

(2.22) 

78.8 

(4.97) 

86.2 

(1.96) 

Lack of Resources 

No staff available at district level to perform the work

required for direct certification 

 49.5** 

(3.78) 

32.4 

(5.50) 

47.3 

(3.23) 

Lack of computer resources at the district level to assist 

in matching process 

 40.9** 

(3.88) 

25.4 

(5.02) 

38.8 

(3.28) 

Funds not available for training district staff to do work

required for direct certification 

 38.4*** 

(3.84) 

21.5 

(4.74) 

36.6 

(3.26) 

Any of the “Lack of Resources” Reasons 62.2*** 

(3.60) 

39.1 

(5.88) 

59.7 

(3.12) 

Problems with TANF/Food Stamp Agencies 

TANF or food stamp agencies do not keep records in a 

manner that makes it cost-effective 

 26.9* 

(3.32) 

39.0 

(6.07) 

29.9 

(2.89) 

Source: 2001 Direct Certification Study SFA Survey. 

a

Because of nonresponse on individual survey items, sample sizes vary to a small degree for the items listed in the table. 

The sample size listed in the table is the maximum sample size among items reported. 

 *Difference between districts that have and have not used DC is significantly different from zero at the .10 level, two-tailed test 

 ** Difference between districts that have and have not used DC is significantly different from zero at the .05 level, two-tailed test.

*** Difference between districts that have and have not used DC is significantly different from zero at the .01 level, two-tailed test. 

19
Even if they wished to use direct certification, districts that lack (or perceive that they lack) 

sufficient resources to implement the policy probably will not do so. A substantial proportion of 

districts—especially among those that had never used direct certification—reported that a reason 

for not using the policy was that they lacked some key resource. Among districts that had never 

used direct certification, about half said that they had no staff available at the district level to 

perform the required work, 41 percent reported a lack of computer resources, and 38 percent said 

that funds were not available for training district staff to do the work required for direct 

certification. Overall, 62 percent cited one of these “lack of resources” reasons for not using 

direct certification. 

Less commonly, districts cited as a reason for not using direct certification that they had 

problems dealing with State food stamp or welfare agencies. Thirty percent claimed that TANF 

and food stamp agencies do not keep records in a manner that makes it [direct certification] costeffective.13 A similar percentage reported that concern about student confidentiality was an

important reason in their decision not to use direct certification. 

B. DIRECT CERTIFICATION IMPLEMENTATION TYPES 

A key component of the 1996 study was the development of a direct certification typology. 

Jackson et al. (1999) described three primary direct certification types. The 1996 study 

categorized most—though not all—direct certification districts into one of these three types on 

the basis of the following aspects of their implementation of direct certification: 

20

 13This reason was cited significantly more often by districts that had previously used direct 

certification, perhaps because of negative experiences they had with these agencies when they were using 

direct certification. 
• Whether a process of matching lists of students enrolled in the district with 

households receiving FS/TANF/FDPIR was conducted 

• If matching was conducted, whether it was done at the State or district level 

• Whether the State or district sent the letter notifying households of their eligibility 

• Whether children were required to return the notification letter to the school to trigger 

direct certification 

In nonmatching (Type I) districts, there was no matching of enrollment and 

FS/TANF/FDPIR lists. Instead, a State agency sent letters to all families with school-age

children receiving FS/TANF/FDPIR notifying them of their potential eligibility for free meal 

certification. These nonmatching districts required active consent to directly certify students. To 

become directly certified, in other words, students in these families were required to bring the 

notification letter to school and submit it to the appropriate school meal program administrators; 

without this letter, students would not be directly certified to receive free meals. 

Both district-level matching (Type II) and State-level matching (Type III) districts 

conducted matching of enrollment and FS/TANF/FDPIR lists. These districts also used passive 

consent—students in a given district who were matched on these two lists were automatically

considered directly certified for free meals and did not have to take the additional step of

returning their notification letter to school to trigger direct certification. Families not wanting 

their child to be directly certified could notify the district of this, and the child would be removed 

from the direct certification list. 

Type II and Type III districts differed in the entity that carried out the processes of matching 

and notification. In Type II—district-level matching—districts, matching was done at the district 

level and districts sent the notification letters to students’ households. In Type III—State-level

matching—districts, matching was done at the State level, in which case the letters were sent by 

the State. 

21
In the 1996 study, most direct certification districts conformed to one of these three 

implementation types. About 16 percent of districts, however, implemented direct certification 

in a way that did not conform precisely to one of the three types. These mixed type districts

combined characteristics of more than one type in implementing direct certification. For 

example, some nonmatching districts performed the task of identifying and notifying families 

with school-age children on FS/TANF/FDPIR at the district level, rather than the State level. 

Some matching districts performed some functions at the State level, others at the district level. 

A handful of mixed type districts in 1996 conducted matching but also required active consent in 

order to trigger direct certification. 

Between 1996 and 2001, there was a shift in the implementation of direct certification 

toward districts that conduct matching. We found that in 2001, 41 percent of direct certification 

districts conducted district-level matching and 27 percent conducted State-level matching (Table 

II.5). These levels of matching districts represent an increase since 1996, when 34 percent 

conducted district-level matching and 19 percent conducted State-level matching. Overall, the 

percentage of direct certification districts using matching increased from 53 to 68 percent over 

this period. There was a corresponding reduction over this period in the prevalence of

nonmatching districts, from 32 percent in 1996 to 20 percent in 2001. 

The trends described above are also apparent when examined from the perspective of 

students. Among students enrolled in direct certification districts, about two-thirds were in 

matching districts in 1996 and more than 80 percent were in matching districts in 2001 (Table 

II.5). 

22
TABLE II.5

PREVALENCE OF DIRECT CERTIFICATION IMPLEMENTATION TYPES, FINAL 

CATEGORIZATION 

(Standard Errors in Parentheses) 

1996 Public School Districts 

Type I: 

Non-Matching 

Type II: 

District-Level

Matching 

Type III: 

State-Level 

Matching Mixed

Percentage of Districts 32.1 33.6 18.6 15.6 

(5.0) (5.3) (4.2) (3.9)

Percentage of Students: 

 Enrolled Students 25.0 40.8 26.0 8.2 

(6.3) (7.2) (6.4) (4.0)

 Certified Free Students 28.3 40.3 24.9 6.5 

(10.3) (10.3) (9.0) (5.1)

Sample Size 148 

2001 Public School Districts 

Percentage of Districts 20.1** 41.2 26.5 12.1 

(1.92) (2.38) (2.65) (1.63)

Percentage of Students: 

 Enrolled Students 11.1** 51.7 30.4 6.8 

(0.98) (1.69) (1.60) (0.78)

 Certified Free Students 7.3 55.0 32.7 5.0 

(0.85) (2.12) (2.04) (0.78)

Sample Size 785 

Source: 2001 Direct Certification Study SFA Survey. 

Note: The 2001 categorization replicated the methodology used in the 1996 study, with one exception. After

the initial categorization, which exactly matched the 1996 methodology, a large proportion of districts 

were in the “mixed” category. Many of these districts could be recategorized into one of the three 

primary types on the basis of their use of matching. This recategorization was conducted, resulting in a 

proportion of districts that could be placed into one of the three primary groups much closer to the 

proportion in 1996. 

 *Difference between 1996 and 2001 frequency is significantly different from zero at the .10 level, two-tailed test. 

 ** Difference between 1996 and 2001 frequency is significantly different from zero at the .05 level, two-tailed test. 

*** Difference between 1996 and 2001 frequency is significantly different from zero at the .01 level, two-tailed test. 

23
In addition to the trend toward matching in the implementation of direct certification 

between 1996 and 2001, the way matching districts implement direct certification also changed 

over this period. While the 1996 data suggested that nearly all matching districts used passive 

consent in directly certifying students, a substantial proportion classified as district-level or 

State-level matching districts in 2001 required active consent to trigger direct certification. In 

other words, these districts (or their States) produced a matched list of enrolled students whose 

families received FS/TANF/FDPIR, but they did not automatically directly certify these students. 

Instead, they required them to bring their notification to school in order to become directly 

certified. 

Figure II.2 illustrates this pattern. The 1996 study defined Type II districts as those that

conducted matching and notification at the district level and that used passive consent. In 2001, 

32 percent of districts conformed precisely to that definition of district-level matching. 

However, another 10 percent of districts conducted matching at the district level but used 

active consent to trigger direct certification. Similarly, although 19 percent of districts

conducted State-level matching and notification and used passive consent—thus conforming to 

the 1996 definition of Type III implementation—another 8 percent conducted State-level 

matching and notification but required active consent. Thus, while more districts are using

matching to implement direct certification, there also has been an increase in the use of active 

consent in conjunction with matching, a practice that was rare in 1996. 

The reasons for the increase in the practice of using active consent with matching are not 

clear. One possibility is that districts that formerly were nonmatching districts began conducting 

matching but retained their previous practice of requiring active consent, and their reason for 

using the matching process was for purposes of documentation. These districts may have wished 

24
Figure II.2 

Detailed Breakdown of Direct 

Certification Implementation Types 

12

8

19

10

32

20

0 10 20 30 40

Mixed

State/active

State/passive

District/active

District/passive

Nonmatching

Percentage of Direct Certification Districts

25
to keep track of which students might be expected to bring in a direct certification notification 

letter.14 An alternative possibility is that the districts previously used matching along with 

passive consent, but began requiring active consent to address problems with the matching 

process. In Section C of this chapter, we show the proportion of districts reporting that they

experienced problems with the direct certification process leading to some children in a

household being certified, while their siblings were not. It turns out that this proportion was 

much higher among districts using passive consent, along with matching, than among those

using active consent with matching. Perhaps the latter group of districts turned to active consent 

as a way of avoiding this or other problems related to matching. A final possibility is that these

districts simply misunderstood the set of questions about matching included on the SFA survey 

and should have been coded as nonmatching districts.15 

C. KEY OPERATIONAL ASPECTS OF THE DIRECT CERTIFICATION PROCESS

1. Districts’ Perceptions of Implementation Problems 

In 1996, the use of direct certification was common among public school districts offering 

NSLP lunches, but many districts were inexperienced in the use of this policy. At that time, for 

example, nearly a third of public school districts had been using direct certification for one year 

26

 14Among the districts that reported using both matching and active consent on the SFA survey, we

asked whether the purpose of matching was “to keep track of which students might be expected to bring a

letter of approval for direct certification” and whether matching was conducted “primarily for record 

keeping purposes.” Just under half of the districts that reported using matching and active consent 

answered affirmatively to one of these questions. 

15In the 1996 study, information about direct certification implementation was collected to a greater 

extent through the use of open-ended questions so that the implementation typology could be developed 

based on as much information as possible. Since the typology had already been developed when the 

survey for the current study was designed, much greater use was made of closed-ended questions in

obtaining information about direct certification implementation. While the closed-ended questions allow 

for more efficient collection and analysis of data from a large sample of districts, they are more

susceptible to misinterpretation on the part of respondents. 
(Jackson et al. 1999). By the late 1990s, however, the adoption of direct certification had 

slowed, so that only 7 percent of direct certification districts reported that they had been using 

the policy for one to two years (Figure II.3). A majority of direct certification districts (64

percent), reported that they had been using direct certification for more than five years. 

In other words, direct certification districts now have more experience in operating the 

policy than they did in 1996. As a result, they may be less likely to experience some of the 

difficulties that are typically associated with the initial implementation of any policy. That is to 

say, while the set of problems that districts new to direct certification report might be initial 

“kinks” that will go away as the districts gain experience in operating direct certification, the

problems reported in 2001 are less likely to disappear. Table II.6 shows the problems that direct 

certification districts report having experienced, both overall and by direct certification 

implementation type. We have grouped these problems into four categories: (1) matching issues, 

(2) resource issues, (3) parents not wanting their child to receive free meals, and (4) other issues. 

Direct certification districts are most likely to report experiencing problems related to 

matching. Nearly half reported that the direct certification process resulted in a situation in 

which some children in a household were directly certified while their siblings were not. This 

situation could arise if the matching of enrollment and FS/TANF/FDPIR lists identified one child 

in a household but not his or her sibling(s).16 Not surprisingly, districts that used matching and 

passive consent were most likely to report this problem—two-thirds of these districts cited the 

27

 16This could happen, for example, if children in the same household had different last names or if

one of the lists contained a typo in one of the children’s names. 
Figure II.3 

Districts’ Experience Using Direct 

Certification

21

43

30

7

0 10 20 30 40 50

>10

6 to 10

3 to 5

1 to 2

Number of Years Ago District 

First Started Using DC

Percentage of Direct Certification Districts 

28
TABLE II.6 

PROBLEMS ENCOUNTERED IN CONDUCTING DIRECT CERTIFICATION, BY DIRECT 

CERTIFICATION IMPLEMENTATION TYPE 

(Standard Errors in Parentheses) 

Percentage of Direct Certification Districts Indicating as Problem

District-Level

Matching Districts 

State-Level 

Matching Districts

Problem

NonMatching 

Districts 

Passive 

consent 

Active 

consent 

Passive 

consent 

Active 

consent 

Mixed 

Type 

Districts Total 

Matching Issues

Having some children in a 

household directly certified 

while siblings may not be 

29 

(5.8)

62 

(4.1) 

25 

(7.1)

65 

(5.9) 

62 

(8.3) 

40 

(7.0) 

47.2 

(2.41) 

Not being able to match 

child’s name with parent’s 

name

28 

(5.0)

31 

(3.5) 

23 

(6.9)

27 

(4.4) 

37 

(9.0) 

25 

(5.2) 

28.6 

(1.99) 

Either of the matching issues 42 

(6.6) 

68 

(4.1) 

35 

(8.4) 

72 

(5.9) 

48 

(9.9) 

50 

(7.3) 

56.3 

(2.59) 

Resource Issues

More mailing had to be done 13 

(3.5)

26 

(3.0) 

14 

(6.2)

22 

(3.9) 

19 

(7.9) 

18 

(6.5) 

20.0 

(1.79) 

Current staff do not have time 

for direct certification 

13 

(4.1) 

18 

(2.8) 

16 

(6.6) 

9 

(2.5) 

25 

(8.8) 

17 

(4.5) 

15.8 

(1.64) 

Had to modify computer 

system

4 

(1.8)

14 

(2.1) 

9 

(3.9)

11 

(2.8) 

6 

(2.8) 

7 

(2.6) 

9.7 

(1.08) 

Any of the resource issues 21 

(4.3)

40 

(3.7) 

27 

(7.9)

34 

(4.7) 

32 

(9.1) 

31 

(6.9) 

32.3 

(2.13) 

Parents not wanting their child to

receive free meals 

11 

(5.9) 

9 

(2.0) 

27 

(10.5) 

12 

(3.1) 

23 

(7.9) 

23 

(7.0) 

14.4 

(1.95) 

Other 17 

(3.3)

18 

(2.8) 

5 

(4.6)

17 

(3.9) 

14 

(5.2) 

24 

(7.5) 

15.6 

(1.73) 

Sample Size 149 315 49 168 44 80 790

Source: 2001 Direct Certification Study SFA Survey. 

29
sibling matching problem (Table II.6). By contrast, fewer than one-third of districts requiring 

active consent (whether or not they used matching) reported this problem. As suggested in the 

previous chapter, this finding is consistent with—although not proof of—the argument that the 

sibling matching problem arises from the matching process, but the use of active consent can 

reduce the likelihood of the problem occurring. Another matching-related concern involves 

districts’ difficulty in matching the names of children with their parents—29 percent of direct 

certification districts indicated that this was a problem (Table II.6). The frequency with which 

this problem was cited did not appear to be systematically related to whether or not the district 

used matching or whether it used active or passive consent. Overall, 56 percent of districts cited

at least one of the two matching-related problems. 

Resource issues were less commonly cited by direct certification districts as problems they

had encountered in implementing the process. Fewer than one in three districts cited any of the 

following resource-related problems—(1) the fact that direct certification required more 

mailings, (2) a lack of staff time, or (3) the need for computer modifications as a result of direct 

certification. 

Similarly, few districts (14 percent) reported as a problem the fact that parents of children 

who had been directly certified did not want their children to be certified for free meals (Table 

II.6). Among nonmatching districts, only 11 percent cited this problem. Since these districts 

require active consent—that is, the parents had to agree to direct certification in the first place—

this finding is not surprising. However, a larger proportion (about one-fourth) of both districtlevel and State-level matching districts that use active consent reported this problem. 

2. Do Districts Directly Certify as Many Students as Possible? 

30

The figures reported in Table II.6 describe districts’ own perceptions of the problems with 

implementation of direct certification. An alternative approach to assessing the implementation
of direct certification involves examining the success of districts in directly certifying those 

students whose families are receiving FS/TANF/FDPIR and thus are eligible for direct 

certification. Unfortunately, we do not know the direct certification status and FS/TANF/FDPIR 

status of each student enrolled in our sample districts. However, the SFA survey did ask districts 

to report the number of children directly certified versus certified by application, as well as the 

number certified by application on the basis of a FS/TANF/FDPIR case number (that is, the 

number categorically eligible). Students who were categorically eligible were not necessarily on

FS/TANF/FDPIR at the time that direct certification eligibility was determined (typically during 

a summer month prior to the school year), since they could have begun receiving assistance after 

that time or they may have misreported their status on their application. It seems reasonable to 

assume, though, that a substantial proportion of categorically eligible students were on 

FS/TANF/FDPIR during the summer and could have been directly certified. Thus, if the number 

of categorically eligible students in a district is large relative to the number of directly certified 

students, this constitutes evidence that the direct certification process in the district missed a 

portion of students who could have been (and apparently wanted to be) directly certified. 

Previously, we showed that in the average direct certification district, 25 percent of freeapproved students were directly certified. Table II.7 presents this information along with 

information on the number of categorically eligible students in public school districts nationally. 

In the average direct certification district, 18 percent of students certified for free meals were

categorically eligible in addition to the 25 percent who were directly certified, with the 

remaining 58 percent certified by application on the basis of reported income. Among all 

enrolled students in the average direct certification district, 62 percent were not certified at all,

26 were certified for free or reduced-price meals on the basis of income, 8 percent were directly 

certified, and 5 percent were categorically eligible. 

31
TABLE II.7

PERCENTAGE OF STUDENTS CATEGORICALLY ELIGIBLE 

VERSUS DIRECTLY CERTIFIED 

(Standard Errors in Parentheses)

Direct

Certification

Districts 

Non-Direct 

Certification

Districts 

All 

Districts 

Percentage of Certified Free Students 

Who Are: (means) 

Directly certified 24.9*** 

(0.92) 

0.0 

(0.0) 

15.1 

(0.77) 

Categorically eligible 17.5*** 

(0.62) 

30.4 

(1.44) 

22.4 

(0.70) 

Certified by application/income 57.6*** 

(1.15) 

69.7 

(1.44) 

62.5 

(0.96) 

Percentage of Enrolled Students Who 

Are: (means) 

Directly certified 7.5*** 

(0.41) 

0.0 

(0.0) 

4.5 

(0.30) 

Categorically eligible 5.3*** 

(0.29) 

9.2 

(0.74) 

6.8 

(0.34) 

Certified by application/income 25.6 

(0.68) 

26.7 

(1.49) 

26.0 

(0.71) 

Not certified 61.5 

(0.90) 

64.1 

(1.87) 

62.6 

(0.92) 

Sample Size 623 331 957 

Source: 2001 Direct Certification Study SFA Survey. 

 *DC/Non-DC difference is significantly different from zero at the .10 level, two-tailed test 

 **DC/Non-DC difference is significantly different from zero at the .05 level, two-tailed test. 

***DC/Non-DC difference is significantly different from zero at the .01 level, two-tailed test. 

32
The proportions categorically eligible versus directly certified suggest that many students

who were on FS/TANF/FDPIR either prior to or early in the 2001-2002 school year were not 

directly certified. Among students in the average direct certification district about whom we 

have some evidence that they were on FS/TANF/FDPIR in summer or fall 2001, only 59 percent 

were directly certified. The remainder completed an application to become certified, the step

that direct certification was supposed to eliminate for those on assistance. It is also possible that 

additional students were on FS/TANF/FDPIR in summer 2001, and were neither directly 

certified nor categorically eligible—instead, they may have become certified on the basis of an 

application on which they reported their income or they have not been certified at all.

Why does the direct certification process appear to be missing so many students who may be

eligible to be directly certified? One potential explanation was noted above: that categorically 

eligible students were not actually eligible for direct certification, because they either

misreported FS/TANF/FDPIR receipt or they began receiving it after direct certification 

eligibility was determined. Two other possible explanations involve the direct certification 

process itself. First, students identified as being eligible for direct certification in States that use 

active consent may have been notified of their eligibility but did not return the notification letter

to their school and thus did not trigger direct certification. Second, students receiving 

FS/TANF/FDPIR may not have been identified by the matching process used by the 

State/district.17

33

 17The first of these two explanations would predict lower proportions of directly certified students

relative to those categorically eligible in active consent districts than in passive consent districts.

However, the second explanation would predict lower relative proportions of directly certified students in 

matching districts than in nonmatching ones. Because most matching districts also use active consent and

nonmatching districts use passive consent, these two effects may cancel each other out. We examined 

variation in the proportion directly certified among those either directly certified or categorically eligible,

and found little variation across districts using the different direct certification implementation types. 
3. Do Districts Feel that Direct Certification Has Benefits? 

Despite the evidence described above that there are some problems with the implementation

of direct certification, most districts that use direct certification agree that it has had some

positive consequences. For example, 8 of 10 direct certification districts agreed that direct

certification makes the approval process less costly, and 9 in 10 agreed that it makes the approval 

process more efficient (Table II.8). In addition, 9 in 10 agreed that direct certification helps get 

more children to become certified and receive free meals. 

D. SUMMARY 

Use of direct certification has remained fairly stable since 1996. More than 6 in 10 of the 

nation’s public school districts continue to use this process to certify children for free meals. 

Among districts not using direct certification, most reported that they do not use this practice 

because they are satisfied with their current procedures for certifying students. 

Most districts using direct certification conduct some sort of matching process to identify 

and notify students of their direct certification status, and the percentage of districts conducting 

matching increased between 1996 and 2001. However, there appears to have been a shift in the

way these matching districts implement direct certification. In the 1996 study, nearly all 

matching districts used passive consent, whereby students’ direct certification status was

triggered as soon as they appeared on a matched list of those both enrolled in the district and on 

FS/TANF/FDPIR. No further actions were required of these students to become directly 

certified. By 2001, however, a substantial minority of matching districts began requiring active 

consent on the part of students to trigger direct certification. Although we lack direct evidence 

on the causes of this trend, we suspect that it has multiple causes, including a desire on the part 

of districts to improve their record keeping and a response to past problems they may have had 

with the matching process. A common matching problem cited by districts—although not as

34
TABLE II.8

PERCENTAGE OF DIRECT CERTIFICATION DISTRICTS THAT AGREE OR STRONGLY 

AGREE WITH POSITIVE STATEMENTS ABOUT DIRECT CERTIFICATION 

Percentage of Public 

School Districts Standard Error 

Direct Certification Makes the Approval Process 

More Efficient 93.2 1.07 

Direct Certification Makes the Approval Process 

Less Costly 79.5 1.96 

Direct Certification Helps Get More Children 

Certified for Free Meals 90.6 1.31 

Direct Certification Helps Get More Children to 

Receive Free Meals 89.4 1.42 

Sample Size 787 

SOURCE: 2001 Direct Certification Study SFA Survey.

commonly by districts using active consent—was that the direct certification process sometimes 

resulted in a situation in which some children from a household were directly certified and others 

were not. 

Although direct certification is widely used nationally, there is evidence that many students 

who could be directly certified are not. While about 25 percent of students certified for free 

meals in direct certification districts are directly certified, another 18 percent are certified by 

application on the basis of a FS/TANF/FDPIR case number. At least some of these categorically 

eligible students could have been directly certified but were missed by the system. 

35
III. CERTIFICATION AND PARTICIPATION EFFECTS OF DIRECT 

CERTIFICATION 

In Chapter I, we described direct certification as a policy implemented to improve access to 

the NSLP. In this chapter, we present estimates of the effects of direct certification on two 

measures of program access, certification for free/reduced-price meals and NSLP participation. 

Section A describes the methodology used to estimate the effects of direct certification. The 

estimated effects of direct certification on rates of certification and participation are presented in 

Section B. 

A. METHODS 

We used two empirical approaches for estimating the effects of direct certification: (1) a

district-level analysis, and (2) a State-level analysis. The district-level analysis compares levels

of the outcome measures (rates of certification, participation) in districts using direct certification 

with the levels of the outcome measures in districts not using direct certification at a single point 

in time (October 2001). The State-level analysis compares the change in the outcome measures 

in States that began using direct certification over a given period with the changes in States that

did not begin using direct certification over the same period. 

1. District-Level Analysis 

We first describe the basic district-level model below, then give the details of how the model

was used to estimate the effects of direct certification on certification and participation.

a. General Model 

The district-level model explores the relationship between a district’s direct certification

status and the outcome measures(certification and participation). In the model, each observation 

36
represents information on the characteristics of a given school district as of October 2001.18 The

model can be expressed:

(1) i i i y c = + X b + a DC + ei

 

where: 

yi = outcome of interest for district i 

 Xi = vector of district i’s characteristics hypothesized to influence outcome

 DCi = binary variable indicating whether district i used direct certification 

In addition, ei is an error term, and a, b, and c are parameters to be estimated. The key 

coefficient is a, which represents the effect of direct certification on the outcome of interest. 

This specification assumes that direct certification has a constant effect on the outcome of 

interest: a district’s rate of certification or participation. In other words, the model assumes that 

the effect of the direct certification policy does not vary across districts according to differences 

in the implementation of direct certification, in terms of variables such as the length of time the 

district has been using direct certification or important aspects of the policy’s operational 

features. We tested the assumption implicit in this model by estimating alternative specifications

of the basic model that allowed for differential effects of direct certification depending on its 

characteristics. In particular, we estimated alternative specifications that allowed the effect of 

direct certification to vary by district size and poverty level. We also estimated specifications 

that modeled the influence of direct certification in the following ways: 

18In most of the estimation results presented in this chapter, the observations of the model are not 

weighted according to the size of the district. For selected models, however, we did use sample weights, 

so that the experiences of large districts accounting for a larger proportion of enrolled and certified

students count more heavily than do the experiences of smaller districts with fewer students. 

37
• Direct certification entered into the model as a series of variables indicating how long 

the district had been using the policy (no more than 2 years, 3 to 5 years, 6 to 

10 years, or more than 10 years).19

• Direct certification entered into the model as a series of variables indicating what type 

of direct certification implementation the district used (nonmatching, district-level

matching with passive consent, district-level matching with active consent, Statelevel matching with passive consent, State-level matching with active consent, or 

mixed). 

• Direct certification entered into the model as a single continuous variable indicating 

the proportion of certified free students in the district who had been directly certified 

(set to 0 if the district did not use direct certification). 

The basic model included another feature designed to determine whether the effect of direct 

certification differs in different contexts. In particular, we interacted direct certification status

with a variable indicating whether the district included schools operating under Provision 2 or 3, 

and in which the base year was earlier than the 2001-2002 school year. If direct certification 

were used in the Provision 2 or 3 schools in these districts, it would have been used prior to 

2001-2002, and thus may have had a different effect on the outcome measures of certification 

and participation than in districts without Provision 2 or 3 schools.20

The estimation of these alternative specifications was undertaken to deepen our 

understanding of how, if at all, direct certification influences rates of certification and 

participation. Does the effect of the policy grow stronger over time? Does direct certification 

 19We collected information on how long districts had been using direct certification on the SFA

survey. However, because some districts could not report an exact number of years of direct certification

experience, we allowed them to report their experience within the ranges of years shown above. We also 

included in the model a binary variable indicating whether the district had formerly used direct

certification but no longer did so by the 2001-2002 school year. 

20Since schools operating under Provision 2 or 3 would not have conducted verification, districts 

with these schools were excluded from the analysis of the effects of direct certification on rates of 

ineligibility among certified students. 

38
have stronger effects if it is implemented in a particular way? How do its effects differ if it

involves a large versus small proportion of a district’s certified students? 

b. District-Level Model of the Effect of Direct Certification on Rates of Certification 

The model represented by Equation (1) estimates the effect of direct certification on

certification when the outcome variable yi represents the certification rate in district i. The 

analysis focuses primarily on the effect of direct certification on the free certification rate, 

defined as the proportion of all enrolled students in a district who are certified for free meal

benefits.21 We also estimated versions of the certification models in which the dependent 

variables were the reduced-price certification rate—the proportion of enrolled students certified 

for reduced-price meals—and the total certification rate—the proportion of enrolled students 

certified for free or reduced-price meals. Since these certification rate measures are continuous 

variables, we estimated these models using ordinary least squares (OLS) regression techniques.22

A critical aspect of successfully estimating the effect of direct certification on the free 

certification rate in a district is that the model adequately controls for any district characteristics 

that influence certification and that potentially could differ between direct certification and nondirect certification districts. Failure to control for such characteristics may result in selection 

bias in the estimated effects of direct certification. For example, certification rates are likely to

be higher in districts with larger proportions of low-income students, and districts with large 

low-income populations may be more likely to use direct certification, especially if these low-

 21See Appendix Table C.1 for information on the rates of certification for free and reduced-price

meals, as well as NSLP participation in public school districts nationally that offer NSLP lunches. The 

information in that table was based on data collected as part of the SFA Survey and was used to create the

dependent variables used in the analysis. 

22In estimating the standard errors of the model’s regression coefficients, we took into account the

complex sample design of the SFA survey, using the SUDAAN statistical package, which employs a 

Taylor series expansion to account for design effects. 

39
income populations contain numerous FS/TANF/FDPIR recipients. Thus, it is important that the 

model control for the economic status of a district’s students. 

The district characteristics hypothesized to have an important influence on the free

certification rate are represented by the vector Xi in Equation (1). The variables included in Xi are 

listed in Table III.1,23 and were selected because they fell into one of the following categories:

•

•

•

 

Demographic and Socioeconomic Characteristics of the District’s Students. Since

certification requires that students’ household incomes fall below certain thresholds 

(or that they receive FS/TANF/FDPIR), controlling for the socioeconomic status of 

districts’ students is important. We included two measures of poverty in the model—

an estimate of the poverty rate of students in the district and an estimate of the overall

poverty rate in the county—along with the squares of these measures. Elements of 

socioeconomic status are also measured by the racial/ethnic distribution in the district 

and by the percentage of students who are limited English proficient. Another

demographic characteristic that has been shown to be related to certification is a 

student’s age (Gleason 1996), in which the model included a measure of the 

percentage of the district’s students enrolled in elementary school. 

District Size. The number of enrolled students in a district may be related to 

certification rates for at least two reasons. First, district size may influence the way 

the program is administered, which in turn could influence how easy or difficult it is 

for students to become certified. Second, the size of the district may affect the stigma

of being certified. For example, students who are certified may more easily be 

identifiable and noticeable in smaller districts. 

Administration of the NSLP. As suggested above, the way in which a district

administers the NSLP may influence rates of certification. In particular, students 

may, more or less, be likely to become certified, depending on how the district

distributes and processes applications, encourages or fails to encourage households to 

apply for benefits, monitors the certification status of students over the course of the 

school year, and conducts the verification process. We included several variables in 

the model that proxy for various aspects of the district’s program administration, 

including several features of the verification process, whether they use single child or

multi-child applications, and their use of electronic point-of-sale systems for 

23Table IV.1 also lists variables not included in Xi in the main model but that were included in

alternative specifications whose results are also presented in this chapter. Descriptive statistics on the

variables listed in Table IV.1 are shown in Appendix Table C.2, overall and by direct certification status. 

40
TABLE III.1 

DISTRICT CHARACTERISTICS INCLUDED IN DISTRICT-LEVEL ANALYSIS 

Direct certification Binary indicator of whether district uses direct certification 

Binary indicator of whether direct certification has been in place for 1-2 years 

Binary indicator of whether direct certification has been in place for 3-5 years 

Binary indicator of whether direct certification has been in place for 6-10 years 

Binary indicator of whether direct certification has been in place for more than 10 years 

Binary indicator of whether non-matching direct certification is in place 

Binary indicator of whether district-matching, passive consent direct certification is in place 

Binary indicator of whether district-matching, active consent direct certification is in place 

Binary indicator of whether State-matching, passive consent direct certification is in place 

Binary indicator of whether State-matching, active consent direct certification is in place 

Binary indicator of whether a mixed form of direct certification is in place 

Binary indicator of whether district formerly used direct certification but did not do so in the 

2001-2002 school year

Continuous variable indicating the percentage of certified free students who are directly certified 

Provision 2 or 3 Schools Binary indicator of whether the district includes any schools operating under Provision 2 or 3

and in which the base year for most of these schools prior to the 2001-2002 school year

Size of district Binary indicator of enrollment no more than 500 

Binary indicator of enrollment between 501 and 1,000

Binary indicator of enrollment between 1,001 and 5,000 (excluded from model)

Binary indicator of enrollment between 5,001 and 10,000 

Binary indicator of enrollment between 10,001 and 25,000

Binary indicator of enrollment greater than 25,000 

Proportion of young

students 

Percentage of enrolled students who are in elementary school (typically grades K through 5)a

Urbanicity Binary indicator of urban residence 

Binary indicator of suburban residence (excluded)

Binary indicator of rural residence 

State Set of binary indicators of the State in which the district is located

(Note—States in which fewer than 10 districts were included in the sample were combined with

other nearby States and represented by a single binary variable. In particular, the following sets

of States were represented by these combined binary variables: AK and HI; DE and NJ; NM

and AZ; RI and CT; DC and MD; NH and VT; SD and ND; and MT, ID, and WY. 

Racial/ethnic distribution Proportion of students in the district who are white, non-Hispanic (excluded) 

Proportion of students in the district who are black, non-Hispanic 

Squared proportion of students who are black, non-Hispanic

Proportion of students in the district who are Hispanic

Squared proportion of students who are Hispanic

Proportion of students in the district who are “other” 

Squared proportion of students who are “other” 

41

 
Table III.1 (continued) 

Primary language used

by students

Proportion of students in the district classified as “limited English proficient”

a

Poverty rate Estimated (1997) poverty rate among students attending district

Squared poverty rate among students attending district

Poverty rate (1999) among all who live in county 

Squared poverty rate among all who live in county

Month verification 

process 

completed

Binary indicator of verification process completed in October or earlier 

Binary indicator of verification process completed in November 

Binary indicator of verification process completed in December (excluded) 

Binary indicator of verification process completed in January or later 

Type of verification 

sample selected

Binary indicator that random verification sample selected (excluded)

Binary indicator that focused verification sample selected 

Binary indicator that other type of verification sample selected 

Size of verification 

sample selected

Binary indicator that less than 1 percent of applications selected verification sample

Binary indicator that 1 to 2 percent of applications selected for verification sample 

Binary indicator that 2 to 4 percent of applications selected for verification sample (excluded) 

Binary indicator that 4 to 10 percent of applications selected for verification sample

Binary indicator that >10 percent of applications selected for verification sample 

Type of application used Binary indicator that district uses single-child certification applications (excluded)

Binary indicator that district uses multi-child certification applications

Binary indicator that district uses some other type of certification application 

Use of verification for 

cause 

Binary indicator of whether district uses verification for cause 

Use of electronic pointof-sale system 

Binary indicator of whether district uses an electronic point-of-sale systema

a

This variable has a substantial number of missing values in its original form. These missing values have been imputed

using the mean of all valid observations of the variable. In addition, the model includes a binary variable that indicates 

whether or not the values of the variable have been imputed in this way.

42

 
processing school meals. Districts’ use of electronic systems has been hypothesized 

to promote certification by reducing stigma (USDA 1999). The other measures may 

or may not directly influence certification rates, but even if they have no direct 

effects, they may proxy for important aspects of NSLP administration that do 

influence certification. 

• Geographic Factors. The model includes indicators of whether districts are located

in urban, suburban, or rural areas, as well as the individual States in which districts

are located. Urbanicity may influence certification because of differences between 

urban, suburban, and rural areas in attitudes toward government programs and 

because of differences in available alternatives to school meals. A district’s State 

may influence certification because of differences between States in the 

administration of the NSLP and/or differences in economic conditions.

c. District-Level Model of the Effect of Direct Certification on NSLP Participation

The underlying model that explains how direct certification may influence participation 

differs somewhat from that shown in Equation (1). This underlying model can be represented by 

the following set of equations: 

43

i

i

(2) C Z i i = + 1 1 b a1DCi + u1

(3) P Z i i = + 2 2 b a2Ci + a3DCi + u2

(4) where: 

 Ci = rate of certification in district i

 Pi = rate of NSLP participation in district i

 Z1i = vector of district characteristics hypothesized to influence certification 

 Z2i = vector of district characteristics hypothesized to influence participation 

 DCi = binary variable indicating whether district i used direct certification 

In this model, Equation (2) is analogous to Equation (1) and is basically the model described 

in subsection 1b. Equation (3) represents the determinants of the participation rate in a given 

district. Our main focus in the analysis will be on the overall participation rate—the proportion

 
of all enrolled students who get a school meal on a given school day. However, we will also 

examine free, reduced-price, and paid school lunches among all students and among the students

who qualify for those meals. In the model, the district characteristics represented by Z2i is 

hypothesized to influence participation in the district, and these characteristics may or may not 

be the same characteristics hypothesized to influence certification. In addition, certification itself

has been shown to influence NSLP participation (Gleason 1995; and Maurer 1984) and is

included in the model. Direct certification may affect participation either directly, as represented

by the model parameter a3), or indirectly, through its effect on the certification rate. 

We estimated a reduced-form version of this model. By substituting Equation (2) into

Equation (3), we obtained the following:24

44

)

 

(5) 2 1 2 2 1 3 2 1 2 ( ) ( ) ( Pi i = + Z a b b + a a + a DCi + a u i + u i

This reduced-form model can be estimated by ordinary least squares (OLS) by regressing 

the participation rate on district characteristics and direct certification status. In the reducedform model, it is not possible to distinguish between the direct effect of direct certification on 

participation and its indirect effect (via certification). The model provides a valid estimate of the 

overall effect of direct certification on NSLP participation, which is the impact of primary 

interest. 

The control variables included in the model are the same as those included in the

certification model and listed in Table III.1. Other district characteristics have been found to 

influence NSLP participation that are not included in the model, such as gender, available 

alternatives to school meals in the district, and the price of school meals (Gleason 1995). 

24For simplicity, Equation (4) assumes that the factors that influence participation (Z1i), and the

factors that influence participation (Z2i), are the same and are represented by Zi. 

 
However, this information was not collected as part of the Direct Certification Study. We felt 

that these district characteristics were unlikely to be strongly correlated with a district’s decision 

to use direct certification and thus that their exclusion would not lead to a substantial bias in the 

estimated effect of direct certification on districts’ NSLP participation rate. 

2. State-Level Analysis 

Despite the range of factors controlled for in the district-level model, a potential weakness of 

the model is that there may be important district characteristics that influence districts’ rates of

certification, participation, and/or ineligibility for which we lack data. If these unmeasured

characteristics are also related to whether or not a district uses direct certification, then the

estimated effects of direct certification could suffer from selection bias. For example, suppose

that the intensity of a district’s efforts to promote certification have an important influence on the 

certification rate. Districts that make an extra effort to promote certification might also have

chosen to use direct certification; thus, failure to control for these efforts may lead to a biased

estimate of the effect of direct certification on the certification rate. In this instance, we would 

attribute the results of a district’s efforts to promote certification to the effect of direct 

certification.

45

A related problem is that district-level characteristics may be measured inaccurately. For 

example, the variables measuring the level of poverty in the district may not accurately capture

differences between direct certification and non-direct certification districts in their levels of

economic distress. In other words, two districts with identical measured poverty rates may differ 

in terms of the proportion of students who are economically disadvantaged and would benefit 

from free/reduced-price certification. If the more disadvantaged of these districts are also more 

likely to use direct certification, then the model would detect a spurious positive relationship 

between direct certification and the certification rate. 

 
To address this potential weakness of the model, we would like to control for all relevant

differences between districts, both measured and unmeasured. A “fixed-effects” model offers 

one way to control for unmeasured fixed (unchanging) differences between districts.25 In a

district-level, fixed-effects model, a district would essentially serve as its own control. Changes

in districts’ free certification and participation rates over time could be examined to determined

whether these changes were correlated with whether or not the district had implemented direct 

certification over that period. For example, if the certification rate had increased substantially 

over a given period in districts that implemented direct certification at some point during that 

period, but had remained constant in districts that had not done so, we would conclude that direct 

certification had positively influenced the free certification rate. 

To estimate a fixed-effects model, we need longitudinal data—information at more than one 

point in time on districts’ certification/participation rates, direct certification status, and other 

characteristics included as control variables in the model. Unfortunately, we have no such 

longitudinal data for the school districts in our sample; we do have longitudinal information on 

districts’ rates of certification and participation, direct certification status, and other factors 

defined at the State level. This information allowed us to estimate State-level fixed-effects

models of certification and participation. Since we had no State-level data on rates of 

ineligibility among certified students, we could not estimate State-level models of ineligibility

among certified students. 

46

 25An alternative approach to the issue of selection bias that we considered but rejected would be to 

estimate an instrumental variables or “selection correction” model. In this type of model, districts’ 

decision to use direct certification would be explicitly modeled and a two-stage estimation strategy would

be used to estimate the effect of direct certification on the outcome of interest. This estimation strategy

requires there to be “identifying variables” that are strongly related to a district’s direct certification status 

and have no direct effect on the outcome of interest. We decided against this approach because of the

difficulty of finding appropriate identifying variables in the context of the certification, participation, and 

ineligibility models. 

 
The Food and Nutrition Service’s (FNS’s) administrative records provided the data needed

to construct the key dependent variables of the model—State-by-State rates of certification for 

free and reduced-price meals and NSLP participation. This information was available for all 50 

States and the District of Columbia for the period between 1990 and 2001. For each State, the 

certification and participation rates were measured in October of each year. 

The key independent variable on State use of direct certification over this period came from 

the 1996 Study of Direct Certification (Jackson et al. 1999). This survey included a question on 

whether any district in a State was using direct certification and, if so, when the policy was first 

implemented in the State. All but two States had begun using direct certification by 1996; we

learned whether/when the two remaining States began using the policy by phoning these States’ 

child nutrition directors. This information allowed us to construct a set of variables indicating 

whether each State was using direct certification and the length of time they had been using it as 

of each of the years between 1990 and 2001.26

The other independent variables in the State-level fixed-effects model measure State 

characteristics that change over time. These time-varying characteristics were obtained from a 

47

 26We also have two measures of the penetration of direct certification within each state. The first

source of this information was the 1996 study, which provided information on the percentage of school 

districts in the state using the policy as of 1996. Second, the SFA survey conducted as part of the current 

study provides us with samples of districts within each of the states, from which we estimated the

percentage of the state’s districts using the policy as of 2001-2002. However, since this information was 

available only at these two points in time, we could not include the penetration of direct certification as an 

independent variable in the model. Instead, we used this information to categorize states into highpenetration and low-penetration ones and estimated differential effects of direct certification across the

two types of states. 

 
variety of Federal government sources and cover the period 1990 to 2001. These variables 

include:27

• Percentage of the State’s residents receiving Aid to Families with Dependent 

Children (AFDC) or TANF 

• Percentage of the State’s residents receiving food stamps 

• Median income

• Poverty rate 

• Unemployment rate 

• Mean hourly wage in the manufacturing industry 

We estimated the following State-level, fixed-effects model to determine the impact of 

direct certification use on a State’s certification and participation rates. In the model, each

observation represented conditions in a given State in a given year (from 1990 to 2001), so that 

each State contributed 12 State-year observations. 

(6) 1 2 1 2 3 _ it it it t i it y c c DC YRS d Z d YEAR d STATE u ′ ′ ′ = + + + + +

where:

yit = outcome of interest in State i in year t (certification or participation rate) 

DC_YRSit = number of years direct certification has been used in the State 

 Zit = vector of time-varying characteristics of State i in year t

 YEARt = vector of binary variables representing the year (1990-2001) 

 STATEi = vector of binary variables representing the State 

 uit = random error term

48

 27Descriptive statistics on these independent variables, as well as on the variables representing states’ 

direct certification status and certification and participation rates over the 1990 to 2001 period, are

provided in Appendix Table C.3. 

 
The dependent variables in the model included the free and reduced-price certification rates,

as well as the free, reduced-price, paid, and overall participation rates. The key coefficient 

estimate from the model is c2, which represents the effect of one year of using direct certification 

in the State on the dependent variable. 

This specification of the State-level model differs from the specification of the district-level 

model in that the key independent variable of the model that captures direct certification use is 

measured as the number of years of direct certification use, rather than as a single binary variable

indicating whether or not the State used direct certification in a given year. The rationale for this 

specification was our hypothesis that a State’s use of direct certification in a given year will have

a different effect depending on the number of years the policy has been in use in the State. One 

reason for this hypothesis is that when direct certification was first implemented in a State, it 

may not have been implemented in many districts within the State. The longer the policy has 

been in place anywhere in the State, however, the more likely it is that its use has spread to a

larger proportion of districts in the State. Since we would expect the effect of direct certification 

in a State to be larger if the percentage of districts in the State that use direct certification were

larger, we also would expect the estimated effect of direct certification to be larger if it had been

used in the State for a longer period. The previous study of direct certification provides 

empirical support for this hypothesis; (Jackson et al. 1999) found that the estimated impact of 

direct certification did indeed grow stronger over time in States, and that this effect was roughly 

linear. 

Given that by 2001, direct certification had been used in many States for more than 10 years, 

it is possible that this growing effect of direct certification reached a maximum at some point.

Thus, we estimated an alternative specification of the State-level model that allowed for a 

nonlinear effect of direct certification. This nonlinear specification involved entering the State’s 

49

 
experience with direct certification as a set of binary variables indicating the length of time the 

policy had been used in the State. Finally, we estimated a model in which the effect of direct 

certification was allowed to differ in States with a high direct certification penetration rate versus 

in States with a low penetration rate. The penetration rate is the proportion of districts in the 

State that used direct certification at a given point in time.

28

B. EFFECTS OF DIRECT CERTIFICATION ON CERTIFICATION AND 

PARTICIPATION RATES 

This section presents estimates of the effects of direct certification on districts’ rates of 

certification and participation, based on district-level and State-level analyses. In both types of 

analysis, we find that direct certification has a small, but statistically significant, positive effect 

on the percentage of a district’s students who are certified for free meals. Evidence on the effect 

of direct certification on participation is mixed, with the State-level analysis suggesting a small 

positive effect and the district-level analysis indicating that the effect of direct certification is 

statistically insignificant. These results are generally consistent with estimates of the effect of 

direct certification on certification and participation from the previous study of direct

certification.

1. District-Level Analysis 

The district-level analysis tells us whether districts using direct certification have higher 

rates of certification and participation than districts not using direct certification, controlling for a 

wide variety of measurable district characteristics. The two key outcomes we used to measure

certification and participation are the percentage of enrolled students certified for free meals—

50

 28We estimated this specification of the state-level model in two ways, using the two versions of the

direct certification penetration rate described above. 

 
the free certification rate—and the percentage of enrolled students who get a school lunch on a 

typical school day—the overall participation rate. Table III.2 presents estimates of the effects of 

direct certification and other district characteristics on these two outcomes. 

a. Estimated Effects on Certification 

Since direct certification is designed to certify students for free meals, the hypothesized 

impact of direct certification on the free certification rate is a positive one. All else equal, we 

hypothesize that the free certification rate will be higher in districts using direct certification than 

in districts not using this policy. 

Estimation of the basic model suggests that direct certification has a positive, statistically

significant effect on a district’s free certification rate, although the effect is relatively small in 

magnitude. The estimates indicate that direct certification leads to an increase of about

1.3 percentage points in a district’s free certification rate (Table III.2). For example, since the

average non-direct certification district has a free certification rate of 29.7 percent (see Appendix 

Table C.1), the estimates suggest that if this average district began using direct certification, its 

free certification rate would increase to 31.0 percent. Since there are about 46 million students in

public school districts offering NSLP lunches and 31 million in districts that use direct 

certification, this 1.3 percentage point effect implies that direct certification leads to about 

400,000 additional students being approved for free meals who would not have been approved in

the absence of direct certification. This estimated effect is roughly in line with the estimated 

effect of direct certification found in earlier research. For example, the previous Direct 

Certification Study found that one year of direct certification experience led to an increase in the 

free certification rate of 0.56 percentage points, and that five or more years of direct certification 

51

 
TABLE III.2 

COEFFICIENT ESTIMATES FROM DISTRICT-LEVEL MODELS OF THE IMPACT OF DIRECT CERTIFICATION 

ON THE FREE CERTIFICATION RATE AND OVERALL PARTICIPATION RATE 

(Rates are shown as percentages) 

Variable 

(1) 

Free-Certification 

Rate Model 

(2) 

Overall ParticipationRate Model 

Intercept -4.78* 

(2.69) 

46.5*** 

(4.56) 

District Uses Direct Certification 1.32** 

(0.67) 

0.48 

(1.00) 

Provision 2 or 3 Schools 1.65 

(2.00) 

5.10** 

(2.15) 

Provision 2 or 3 Schools Interacted with Direct 

Certification 

0.90 

(2.11) 

-3.79 

(2.36) 

District Formerly Used Direct Certification 2.40*** 

(0.85) 

1.46 

(1.27) 

Size of District

500 or less 4.72*** 

(1.19) 

9.08*** 

(1.70) 

501 to 1,000 1.17 

(0.76) 

3.62*** 

(1.39) 

1,001 to 5,000 — — 

5,001 to 10,000 -0.86 

(0.64) 

-3.33*** 

(1.05) 

10,001 to 25,000 -1.68** 

(0.84) 

-5.13*** 

(1.36) 

More than 25,000 -2.94*** 

(0.84) 

-7.47*** 

(1.30) 

Proportion of Elementary School Students 7.18 

(1.99) 

14.74*** 

(2.98) 

Urbanicity

Urban 1.79**

(0.83) 

-1.44 

(1.10) 

Suburban —

Rural -0.63

(0.67) 

1.55 

(1.13) 

52

 
Table III.2 (continued) 

53

Variable 

(1) 

Free-Certification 

Rate Model 

(2) 

Overall ParticipationRate Model 

Racial/Ethnic Distribution (Percentage) 

White — —

Black 30.80***

(3.81) 

11.91* 

(6.28) 

Black squared -5.05 

(4.60) 

-11.71 

(8.29) 

Hispanic 33.63***

(4.87) 

-1.40 

(8.28) 

Hispanic squared -6.23 

(6.45) 

15.06 

(9.64) 

Other 11.57

(8.93) 

-15.74 

(13.16) 

Other squared 18.02 

(17.58) 

28.66 

(19.11) 

Proportion of Students Who Are “Limited English 

Proficient” 

10.61* 

(5.66) 

20.51*** 

(6.87) 

Poverty Rate 

Poverty rate within district 144.57*** 

(8.05) 

71.88*** 

(13.99) 

Poverty rate within district squared -122.30*** 

(19.08) 

-60.63** 

(28.81) 

Poverty rate within county 61.05*** 

(12.01) 

96.30*** 

(21.42) 

Poverty rate within county squared -50.54 

(37.15) 

-242.12*** 

(69.26) 

Month Verification Process Completed 

October or earlier -1.45* 

(0.75) 

-3.93** 

(1.58) 

November -0.14

(0.57) 

-2.49*** 

(0.87) 

December — —

January or later -0.41 

(0.93) 

-4.52** 

(1.82) 

Type of Verification Sample Selected 

Random sample — — 

Focused sample -0.22 

(0.62) 

-0.06 

(0.96) 

Other -3.44**

(1.62) 

-0.97 

(4.77) 

 
Table III.2 (continued) 

54

Variable 

(1) 

Free-Certification 

Rate Model 

(2) 

Overall ParticipationRate Model 

Size of Verification Sample Selected (Percentage of 

Applications) 

Less than 1 0.59 

(0.79) 

2.34* 

(1.40) 

1 to 2 1.61*** 

(0.59) 

2.79*** 

(1.06) 

2 to 4 — — 

4 to 10 -0.62 

(0.58) 

-0.45 

(1.06) 

More than 10 -1.63 

(1.61) 

-0.36 

(2.39) 

Type of Application Used

Single-child — — 

Multi-child 0.32

(0.55) 

-2.23*** 

(0.85) 

Other -0.55

(0.92) 

-0.22 

(1.29) 

District Uses of Verification for Cause 0.01 

(0.44) 

-1.29* 

(0.78) 

District Uses Electronic Point-of-Sale System 0.18 

(0.65) 

1.66 

(1.05) 

Mean of Dependent Variable 29.44 60.72 

R-Squared 0.87 0.56

Sample Size 1,212 1,208 

Source: 2001 Direct Certification Study SFA Survey. 

Note: Standard errors are in parentheses. These models were estimated using ordinary least squares (OLS)

regression techniques. Standard errors have been adjusted to account for the complex sample design using 

the SUDAAN statistical package. In addition to the variables listed above, the model contained binary

variables to represent the States in which districts were located (as described in Table III.1). Missing value

flags were also included in the model for the proportion of elementary school students, the proportion of

students who are limited English proficient, the size of the verification sample selected, the type of

application used, and whether the district uses an electronic point-of-sale system. 

 *Significantly different from zero at the .10 level, two-tailed test. 

 **Significantly different from zero at the .05 level, two-tailed test. 

 ***Significantly different from zero at the .01 level, two-tailed test. 

 
experience led to an increase in the free certification rate of 2.01 percentage points (Jackson et al. 

1999).29

The estimated effects of the other district characteristics included in the model on the free

certification rate generally correspond to expectations. The characteristic with the largest effect 

is the poverty rate. Not surprisingly, districts with the highest poverty rates (and located in the 

counties with the highest poverty rates) have the highest free certification rates, and these 

estimated positive effects of poverty on the free certification rate are highly statistically 

significant.30 In addition, being located in an urban area is positively related to the free 

certification rate, as are having high proportions of students who are black and Hispanic. A 

district’s size has a negative influence on the free certification rate, with the rate significantly

lower in districts with larger numbers of students, all else equal. Finally, the estimated effects of 

most of the variables measuring the type of certification application used, whether the district

uses an electronic point-of-sale system, and various aspects of the districts’ verification system

were not statistically insignificant. 

In the basic free certification model described above, the estimated effect of direct 

certification is assumed to be constant across all direct certification districts, regardless of the 

55

 29This finding from Jackson et al. (1999) is based on their state-level, fixed-effects model. They also

estimated a district-level model and found that the estimated effect of direct certification in that model 

was negative and statistically insignificant. However, the authors concluded that the district-level model 

potentially produced a biased estimate of the true effect of direct certification and emphasized the results

of their state-level model rather than the district-level model. Although we estimated a district-level 

model similar to that of Jackson et al. (1999), we made three sets of changes to try to avoid the problems 

associated with selection bias mentioned in that report. First, we included a second measure of poverty to

try to better control for the economic circumstances in the district. Second, we included several measures 

of the district’s administration of the NSLP not included in Jackson et al. (1999). Third, we included a set

of state dummy variables to capture state-level factors (either economic or related to NSLP

administration) not captured by other variables in the model. 

30The estimated positive effect of both district and county poverty rates diminishes as the poverty 

rate increases, but remains positive throughout the relevant range of poverty rates. 

 
district’s characteristics. Because it is possible that this effect differs for direct certification

districts with different characteristics, we estimated a series of alternative specifications of the 

model. These alternative specifications allow for differential effects of direct certification 

according to such factors as the length of time that direct certification has been in place in the 

district and the manner in which it was implemented.31 Table III.3 presents the estimated effect 

of direct certification from these and other alternative specifications. 

The first alternative specification, shown in row 1b of Table III.3, changes the direct 

certification classification of nine districts in the northeast region of the United States. These 

districts reported not using direct certification on the SFA survey even though the use of direct 

certification is mandated throughout the Northeast. For this specification, we treated these 

districts as direct certification districts and reestimated the model. This change did not 

materially affect the results; the estimated effect of direct certification declined to 1.28 (from 

1.32) and remained statistically significant. 

We also examined whether the results changed when we applied a set of weights to the 

districts in the sample based on the number of enrolled students in the district. In this 

specification (row 1c of Table III.3), the estimated effect of direct certification was 1.1

percentage points, and remained statistically significant at the 0.10 level. 

We found little evidence (based on this district-level model) that the effect of direct 

certification grows stronger the longer it has been in place in a district. The estimated effects of 

56

 31In addition to the alternative specifications described in the text below, we estimated two 

specifications in which direct certification status was interacted with some other key district 

characteristic. In one case, we interacted direct certification and district enrollment, and found that the

estimated effect of direct certification was not significantly different in large districts than in small 

districts. In the second case, we interacted direct certification with the poverty rate among students in the 

district and again found that the effect of direct certification was not significantly different in districts 

with different poverty rates. 

 
TABLE III.3 

CERTIFICATION RATE MODEL, ALTERNATIVE SPECIFICATIONS 

Specification Dependent Variable

Variables 

Measuring Direct 

Certification

Coefficient 

Estimate 

Standard

Error 

1a Basic model Free-certification rate DC (binary) 1.32** 0.67 

1b. Basic model with adjusted directcertification variable Free-certification rate Adjusted DCa 1.28** 0.65 

1c. Basic model with student-level 

weights Free-certification rate DC (binary) 1.10* 0.66 

Number of years: 

1 to 2 1.51 0.95 

3 to 5 1.93** 0.77 

6 to 10 1.07 0.72 

2. Direct certification effect allowed 

to differ by number of years it has 

been in place 

Free-certification rate 

More than 10 0.33 0.86 

DC implementation 

type: 

Non-matching 2.75** 1.14

District matching, 

passive consent

1.38* 0.73

District matching, 

active consent 

1.63 1.15

State matching, 

passive consent

0.67 0.77

State matching, 

active consent 

2.93** 1.28

3. Direct certification effect allowed 

to differ by type of directcertification implementation

Free-certification rate 

Mixed 0.89 1.10

4. Direct certification effect allowed DC (binary) 0.89 0.86 

to differ by the percentage of freecertified students who are 

directly-certified

Free-certification rate 

Percentage of freecertified students 

directly certified

1.27 2.30 

5. Reduced-price certification rate 

 basic model 

Reduced-price 

certification rate DC (binary) 0.39 0.25 

6. Total certification rate basic 

model

Total certification rate DC (binary) 1.93** 0.76 

Source: 2001 Direct Certification Study SFA Survey. 

Note: These models were estimated using ordinary least squares (OLS) regression techniques. The control

variables included in the model were the same as those listed in Table III.2. 

a

The adjusted direct certification variable is identical to the original direct certification variable except that it defines 

as direct certification districts all districts in the northeast region (except for New York City), including nine 

districts that had reported not using direct certification on the SFA survey and defined as non-direct certification

districts in the original direct certification variable. 

 *Significantly different from zero at the .10 level, two-tailed test. 

 **Significantly different from zero at the .05 level, two-tailed test. 

57

***Significantly different from zero at the .01 level, two-tailed test. 

 
direct certification for districts that have used the policy for no more than 2 years, for 3 to 

5 years, and for 6 to 10 years are all between 1 and 2 percentage points, with the effect in 

districts that have used the policy for 3 to 5 years (1.93) statistically significant. 

Direct certification does vary systematically according to the manner in which districts 

implement the policy. In particular, the estimated effect of direct certification is larger among 

districts requiring students to actively consent to being directly certified than among districts 

using passive consent. This pattern holds regardless of whether districts use matching as part of 

the direct certification process. The estimated effect of direct certification on the free

certification rate is nearly 3 percentage points (and statistically significant) in nonmatching 

districts and in State-matching, active-consent districts (Table III.3). In district-matching, activeconsent districts, the estimated effect (1.63 percentage points), while not statistically significant, 

is larger than the estimated effect of direct certification in districts using passive consent. 

It is not clear why the estimated effect of direct certification is greater in districts using 

active consent. One might expect that the requirement of active consent would limit, rather than

promote, free meal certification, since it requires an additional step for students to become

certified over and above the steps required by passive-consent districts. It may be the case that 

the use of active consent is correlated with some other unobserved aspect of direct certification 

implementation in these districts. For example, active-consent districts may have more effective 

procedures than passive-consent districts for accurately matching the list of enrolled students 

with the list of food stamp/TANF recipients (which is done in all direct certification districts 

using matching). Also, active consent districts may do more to notify students of their potential 

eligibility for direct certification. However, since the study did not collect qualitative 

information on districts’ implementation of direct certification, we cannot test any of these

explanations, which remain speculative. 

58

 
We also estimated a specification in which the effect of direct certification was allowed to

vary according to the percentage of the district’s free certified students who were directly

certified. This measure proxies for the scale of direct certification in a district—in districts with 

a large number of students on FS/TANF, one might expect a large fraction of free certified 

students to be directly certified. The specification shown in row 4 of Table III.3 examines 

whether this leads to a larger effect on the district’s certification rate. We found that although 

the estimated effect of the percentage of free certified students who were directly certified on the

certification rate was positive, this effect was not statistically significant. 

The final two alternative specifications shown in Table III.3 differ from the basic model in

the dependent variable; rather than estimating the effect of direct certification on the free

certification rate, these specifications examine the effect of direct certification on the reducedprice and total certification rates. Because students are directly certified for free meals, one 

would not expect the policy of direct certification to have a strong influence on certification for 

reduced-price meals. It is possible, however, that direct certification indirectly influences the 

likelihood that students will become certified for reduced-price meals. If direct certification 

increases the free certification rate and leads to a larger number of students in a school receiving 

school meals, then students eligible for reduced-price benefits may be encouraged to apply for 

benefits and become certified because of reduced stigma or an increase in the flow of 

information about the certification process. Alternatively, students who are directly certified in 

one school year may come back the following year eligible for reduced-price meals rather than 

free meals, but be encouraged to apply for benefits again based on their experience as a directly

certified student the previous year. Jackson et al. (1999) refer to this type of explanation for 

direct certification leading to an increase in the reduced-price certification rate as the spillover

explanation. 

59

 
The estimated effect of direct certification on the reduced-price certification rate is positive

(0.4 percentage points) but small and not statistically significant. The estimated effect of direct 

certification on the total certification rate is 1.93 percentage points—as expected, this is roughly 

the sum of its effects on the free and reduced-price certification rates.32

b. Estimated Effects on Participation

As described in Section A, the main way in which direct certification could influence a 

district’s NSLP participation rate is an indirect one. In particular, direct certification leads to 

more students becoming certified for free meals, and various studies have found that free meal

certification positively affects NSLP participation (Akin et al. 1993; Gleason 1995; and Maurer 

1984). Thus, direct certification is hypothesized as leading to greater NSLP participation overall. 

In addition, direct certification could influence participation through the same sort of spillover 

mechanisms as those described above in the context of effects of certification. In particular, a 

district’s use of direct certification and the corresponding increase in certified students could lead 

to less stigma, greater information about the meal programs, or increases in the number of 

children who had been certified or been participants in the previous year. If these effects are

substantial, then participation in the program could increase as a result of direct certification, 

even among students whose certification status did not change as a result of the policy. 

Column 2 of Table III.2 shows the coefficient estimates from the overall participation 

model. The estimated coefficient on the binary direct certification variable is positive (0.48) but 

not statistically significant. This result from the district-level model suggests that direct

60

 32The reason that the estimated effect of direct certification on the free and reduced-price

certification rates does not sum exactly to the estimated effect on the total certification rate is that the 

models were based on slightly different numbers of observations. This was because a handful of districts

reported a free certification rate but their reduced-price certification rate was missing. 

 
certification does not lead to a detectable increase in the NSLP participation rate, despite leading

to a small increase in free meal certification. 

The overall participation rate can be broken down into participation of students getting free

meals, reduced-price meals, and paid meals. Thus, in addition to estimating an overall 

participation model, we estimated models examining the effect of direct certification on free, 

reduced-price, and paid participation rates among all enrolled students. These results are 

presented in Table III.4, along with the results of additional models that show the estimated 

effects of direct certification on free, reduced-price, and paid participation rates among students 

conditional on their certification status. 

The estimated effect of direct certification on each of these participation rate measures is not 

statistically significant. In other words, according to the district-level model, direct certification

does not lead to significant changes in participation—children actually eating school lunches—

among students certified for free meals, students certified for reduced-price meals, or students

paying full price for meals. 

61

 
TABLE III.4 

PARTICIPATION RATE MODEL, ALTERNATIVE SPECIFICATIONS 

Specification 

Dependent Variable 

(Mean Value)

Variables in

Model

Representing 

Direct 

Certification

Coefficient 

Estimate 

Standard

Error 

1. Overall Participation Rate Model Overall Participation Rate 

(60.8) 

DC (binary) 0.48 1.05 

Participation Rates Among All 

Enrolled Students 

2. Free-Participation Rate Model Free-Participation Rate 

Among Enrolled Students 

(23.0) 

DC (binary) 0.35 0.58 

3. Reduced-Price Participation-Rate 

 Model 

Reduced-Price 

Participation Rate Among 

Enrolled Students (6.8) 

DC (binary) 0.23 0.19 

4. Paid Participation Rate Model Paid Participation Rate 

Among Enrolled Students 

(31.1) 

DC (binary) -0.02 0.74 

Conditional Participation Rates 

5. Free-Participation Rate Model Free Participation Rate 

Among Free Certified 

Students (77.3)

DC (binary) -0.76 0.92 

6. Reduced-Price Participation-Rate 

 Model 

Reduced-Price 

Participation Rate Among 

RP Certified

Students(71.7)

DC (binary) -0.39 1.07 

7. Paid-Participation Rate Model Paid Participation Rate 

Among Non-Certified 

Students (52.4)

DC (binary) 0.90 1.17 

Source: 2001 Direct Certification Study SFA Survey. 

Note: These models were estimated using ordinary least squares (OLS) regression techniques. The control

variables included in the model were the same as those listed in Table III2. 

 *Significantly different from zero at the .10 level, two-tailed test. 

 **Significantly different from zero at the .05 level, two-tailed test. 

***Significantly different from zero at the .01 level, two-tailed test. 

62

 
2. State-Level Model 

Since direct certification was first implemented at different times in different States, we can 

turn to an alternative source of information to determine how the implementation of this policy 

has influenced participation in the NSLP and certification for free and reduced-price meals. As

described in Section A, the State-level approach for estimating this effect involves examining 

changes over time in State certification and participation rates as direct certification is 

implemented. In particular, these changes are compared to the analogous changes in certification

and participation rates in States that did not implement direct certification over the same period 

of time. If certification and participation increased by greater amounts in the States that 

implemented the policy, holding other factors constant, this would be evidence of a positive 

effect of direct certification on rates of certification and participation. 

a. Estimated Effects on Certification 

One advantage of collecting longitudinal State-level data on certification and participation 

rates, the use of direct certification, and a variety of State characteristics is that this allowed us to 

estimate a model in which we can control for fixed State effects, which represent unobserved, 

time-invariant State-level factors potentially influencing the outcome of interest. To illustrate 

the importance of controlling for these fixed State effects, we estimated the model both with and 

without these fixed effects. 

The fixed effects model results suggest that a State’s use of direct certification leads to an

increase in the certification rate in the State. The estimated effect of an additional year of direct 

certification use is an increase of 0.20 in the State’s free certification rate, and this estimate is 

statistically significant at the 10 percent level (Table III.5). The size of this estimate suggests 

63

 
TABLE III.5 

COEFFICIENT ESTIMATES FROM STATE-LEVEL MODELS OF THE IMPACT OF DIRECT CERTIFICATION ON 

THE FREE CERTIFICATION RATE, WITH AND WITHOUT FIXED STATE EFFECTS 

Variable 

(1) 

Fixed Effects Model 

(2) 

Model Without Fixed 

State Effects 

Intercept 23.72** 

(3.59) 

-5.33* 

(3.11) 

Number of Years State Has Used Direct Certification 0.20* 

(0.12) 

-0.05 

(0.13) 

Percentage Receiving Aid to Families with Dependent 

Children or Temporary Assistance to Needy Families 

0.65*** 

(0.19) 

0.48*** 

(0.18) 

Percentage Receiving Food Stamps 0.24** 

(0.12) 

1.36*** 

(0.13) 

Median Income -0.10** 

(0.05) 

0.30*** 

(0.05) 

State Poverty Rate -0.02 

(0.06) 

1.31*** 

(0.10) 

Unemployment Rate 0.06 

(0.12) 

0.42** 

(0.20) 

Mean Wage, Manufacturing Industry -0.26 

(0.21) 

-1.29*** 

(0.15) 

Mean of Dependent Variable 29.5 29.5 

R-Squared 0.97 0.79 

Sample Size 612 612 

Source: The free certification rate and percentage receiving food stamps were drawn from the U.S. Department of 

Agriculture, Food and Nutrition Service Data Bank, 1990 through 2001. The number of years of direct 

certification use was obtained from the 1996 Direct Certification Study State Survey. The percentage

receiving AFDC/TANF was obtained from the U.S. Department of Health and Human Services, 

Administration for Children and Families. Median income and the poverty rate were obtained from the 

U.S. Bureau of the Census’ Current Population Survey and from the 2000 Census (for the county poverty

rate). The unemployment rate and mean wage in the manufacturing industry were obtained from the U.S.

Department of Labor, Bureau of Labor Statistics. 

Note: Standard errors are in parentheses. In addition to the variables listed above, both models included a set of 

binary variables indicating the year of data collection (1990 through 2001) and the fixed effects model

included a set of binary variables indicating the state of data collection. 

 *Significantly different from zero at the .10 level, two-tailed test. 

 **Significantly different from zero at the .05 level, two-tailed test. 

***Significantly different from zero at the .01 level, two-tailed test. 

64
that the certification rate would increase by 1 percentage point (from 29 to 30 percent, for 

example) for every five years the policy has been in place. Note that this estimated effect is 

substantially different from an estimate based on a model that excluded the fixed State effects

(which was negative and not statistically significant). 

Most of the other State characteristics included in the model have effects that are in the

expected direction. In particular, the size of a State’s AFDC/TANF caseload and food stamp 

caseload has a significant, positive effect on the State’s free certification rate. Conversely, 

median income has a significant, negative effect on the certification rate. The estimated effects 

of the State poverty rate, unemployment rate, and mean wage in the manufacturing industry are 

not statistically significant. 

The model presented in Table III.5 assumes a linear effect of the number of years of direct 

certification use in a State on the free certification rate. In other words, it assumes that each

additional year that a State has been using direct certification leads to the same change in the

certification rate. Since the estimated effect of this variable was 0.20, this linear effect implies 

that the first year of direct certification use leads to an increase of 0.20 percentage points in the 

certification rate, the second year leads to an additional increase of 0.20 percentage points (so

that the cumulative effect is 0.40), and so on. Jackson et al. (1999) found empirical evidence in 

support of such a linear effect in the 1996 Study of Direct Certification. In a specification in 

which the effect of each additional year of direct certification experience in a State (up through 5

years) was estimated separately, they found that these estimates suggested a linear trend, with the 

effect growing stronger over time. They argued that the effect of direct certification grew

stronger over time, for two main reasons: (1) it took time for States to learn to efficiently operate 

the policy, and (2) the use of direct certification probably spread to a greater proportion of 

districts in a State, the longer the policy was in place. 

65
By 2001, however, direct certification had been used in many States for more than 10 years. 

With a policy around that long, it seems unlikely that the implementation of direct certification 

or the proportion of districts using the policy in a State would change greatly from one year to 

the next. Thus, while the effect of the number of years of direct certification policy use may 

have grown stronger over the first few years of its implementation, we hypothesize that this

effect reaches a limit at some point and ceases growing stronger over time. At this point, in other 

words, the effect of the number of years of direct certification use in States may be nonlinear. 

To test for this possibility, we estimated an alternative specification of the fixed-effects

model presented in Table III.5, column 1 (also shown in Table III.6 as model 1). In this 

specification (model 2 in Table III.6), direct certification use was included in the model as four 

binary variables that indicate whether the policy has been in place in the State for 1 to 2 years,

3 to 5 years, 6 to 10 years, or more than 10 years. The results of the estimation of this model 

suggest that the effect of direct certification does level off after having been in place for a 

number of years. In the first two years of implementation, the model suggests that direct 

certification has no effect on the State’s free certification rate. The estimated effect is 

0.9 percentage points in years 3 to 5, growing to 1.4 percentage points in years 6 to 10. For

States that have used the policy for more than 10 years, however, the effect is only slightly 

larger, at 1.7 percentage points. Thus, it appears that at some point five years after direct 

certification has been introduced in a State, its effect on the free certification rate reaches a high 

point and levels off. 33

66

 33We estimated yet another specification in which separate binary variables were included for each 

additional year of direct certification use (through 10 years). The results of the estimation of this 

specification suggested that the effect of direct certification on the certification rate continued to grow at a 

relatively steady pace for 7 years, after which it leveled off, increasing in some years and decreasing in 

others. 
TABLE III.6 

STATE-LEVEL CERTIFICATION RATE MODEL, ALTERNATIVE SPECIFICATIONS 

Model Description Key Variable(s)

Coefficent 

(Standard 

Error) 

1 Basic fixed-effects model, direct certification enters 

model linearly as number of years of direct 

certification experience 

Number of years of 

direct-certification 

use 

0.20* 

(0.12) 

1 to 2 years of 

direct- certification 

use 

0.00 

(0.37) 

3 to 5 years of 

direct certification 

use 

0.86* 

(0.50) 

6 to 10 years of 

direct certification 

use 

1.42** 

(0.69) 

2 Direct certification enters-fixed effects model 

nonlinearly as a set of binary variables indicating 

number of years of direct certification experience

More than 10 years 

of direct 

certification use 

1.66* 

(0.93) 

Number of years of 

direct certification 

use, high 

penetration States 

0.20* 

(0.12) 

3 Direct certification enters fixed-effects model linearly 

as number of years of direct certification experience, 

separately for States with high versus low 1996 

penetration of direct certification 

Number of years of 

direct certification 

use, low 

penetration States 

0.18 

(0.13) 

Source: See Table III.5. 

Note: Standard errors are in parentheses. Model 1 is taken from Table III.5, column (1). The remaining 

models are identical to model 1 except for the manner in which direct certification enters the 

model. 

 *Significantly different from zero at the .10 level, two-tailed test. 

 **Significantly different from zero at the .05 level, two-tailed test. 

***Significantly different from zero at the .01 level, two-tailed test. 

67
Another specification shown in Table III.6 tested whether the effect of direct certification 

was any different in States with low penetration rates (defined as States in which fewer than 

60 percent of districts used the policy as of 1996), compared with high penetration States.34 We

found little difference in the estimated effects of direct certification use in high penetration

versus low penetration States—in each case, an additional year of direct certification use was 

estimated to lead to an increase in the free certification rate of roughly 0.2 percentage points.35

Together, the estimation of these alternative specifications led us to choose a “preferred 

specification” in which the number of years of direct certification use entered the model 

nonlinearly as a set of four binary variables; but the estimated effect of direct certification was

assumed to be the same in high penetration and low penetration States. We then used this 

specification to estimate the impact of direct certification on States’ rates of reduced-price 

certification (along with a number of participation rate outcomes). The results of the estimation 

of these models are shown in Table III.7. 

The reduced-price certification model (Table III.7, row 2) suggests that direct certification

has no effect on a State’s reduced-price certification rate, regardless of the number of years the 

policy has been in place in the State. For States that have been using direct certification for more 

68

 34Unfortunately, we have measures of direct certification penetration only at two points in time, 1996 

and 2001; both of these points in time occurred after the major period of expansion in the use of direct 

certification. Thus, we could not include variable in the model indicating a state’s penetration rate in each 

year covered by the sample (1990 to 2001). We could only examine whether the estimated effect of 

direct certification differed in those states that had become high penetration states versus those that had

become low penetration states by 1996 (or by 2001).

35We estimated an analogous model using the 2001 penetration rate to define high- versus lowpenetration states. We also estimated separate effects of direct certification in high versus low penetration 

states using the nonlinear specification of years of experience with the policy. In each case, we found no 

evidence of a stronger effect of direct certification in high penetration states. 
TABLE III.7 

ESTIMATED EFFECTS OF DIRECT CERTIFICATION ON STATE CERTIFICATION 

AND PARTICIPATION RATES 

Coefficients on Variables Representing Number 

of Years of Direct-Certification Use 

Model/Dependent Variable (R2) 

Mean of 

Dependent 

Variable 1 to 2 3 to 5 6 to 10

More

than 10

Certification Rates 

1. Free certification rate (0.97) 29.5 0.00 

(0.12) 

0.86* 

(0.50) 

1.42**

(0.69) 

1.66* 

(0.93) 

2. Reduced-price certification rate (0.89) 6.8 -0.13 

(0.11) 

-0.04 

(0.15) 

0.17 

(0.21) 

0.34 

(0.28) 

Participation Rates Among All Enrolled 

Students 

3. Free participation rate among all students 

 (0.97) 

22.8 0.41 

(0.27) 

0.94*** 

(0.36) 

1.06**

(0.48) 

0.55 

(0.65) 

4. Reduced-price participation rate among 

 all students (0.95) 

 4.6 -0.01 

(0.06) 

0.10 

(0.08) 

0.17 

(0.11) 

0.11 

(0.15) 

5. Paid participation rate among all students 

 (0.96) 

28.2 -0.17 

(0.34) 

0.08 

(0.45) 

0.28 

(0.60) 

0.30 

(0.81) 

Conditional Participation Rates 

6. Free participation rate among free-certified

 students (0.77) 

78.9 1.06 

(0.75) 

0.31 

(1.03) 

-0.60 

(1.41) 

-2.10 

(1.89) 

7. Reduced-price participation rate among 

 reduced-price certified students (0.87) 

68.5 0.87 

(0.70) 

0.86 

(0.96) 

0.51 

(1.32) 

-0.48 

(1.77) 

8. Paid participation rate among non-certified

 students (0.96) 

43.7 0.72 

(0.59) 

1.98**

(0.80) 

3.21*** 

(1.10) 

3.32**

(1.48) 

9. Total participation rate among all students 

 (0.96) 

55.6 0.23 

(0.44) 

1.12* 

(0.58) 

1.51* 

(0.78) 

0.96 

(1.05) 

Source: See Table III.5. 

Note: Standard errors are in parentheses. Model 1 is taken from Table III.6, row (2). The remaining models

are identical to model 1 except for the dependent variable. 

 *Significantly different from zero at the .10 level, two-tailed test. 

 **Significantly different from zero at the .05 level, two-tailed test. 

***Significantly different from zero at the .01 level, two-tailed test. 

69
than five years, this estimated effect is positive, ranging from 0.17 to 0.34 percentage points, but 

these estimates are statistically insignificant. 

b. Estimated Effects on Participation

We estimated the effects of direct certification on two different versions of the free 

participation rate (along with the reduced-price and paid participation rates). First, we examined 

how the policy influences the free, reduced-price, and paid participation rates among all enrolled 

students in a State—in other words, the average number of all students in the State who obtain a 

free, reduced-price, or paid meal on a given day. These results are shown in rows 3, 4, and 5 of 

Table III.7. Second, we examined the effect of the policy on the conditional free, reduced-price, 

and paid participation rates among only students certified for free meals, certified for reducedprice meals, or not certified for free or reduced-price meals. These results are shown in rows 6, 

7, and 8. Row 9 shows the estimated effect of direct certification on the overall participation

rate. 

The model estimates provide some evidence that direct certification leads to a small increase 

in the free participation rate among all students. For States in which the policy has been in place 

for at least three years, the estimated effect ranges from 0.6 to 1.1 percentage points. On the 

other hand, the estimated effects of direct certification on the reduced-price and paid

participation rates among all enrolled students are small and not statistically significant.

Overall, the State-level model suggests that direct certification leads to a modest increase in

NSLP participation. Use of the policy for three or more years is estimated to lead to a 1.0 to 

1.5 percentage point increase in the overall participation rate. Most of this increase appears to 

arise from the fact that the policy leads to more students becoming certified for and receiving

free meals, rather than from an increase in the number of reduced-price or paid meals served. 

70
The estimated effects of direct certification on the conditional participation rates help us 

further interpret the results. While the estimated effect on the free meal participation rate among 

all enrolled students was positive and significant, the estimated effect on the conditional free

meal participation rate among certified students is not statistically significant. In other words,

even though the policy leads to an increase in the number of free meals served in a district, it 

does not lead to an increase in the likelihood that a given student who is certified for free meals 

will participate in the NSLP. Rather, the increase in the number of free meals served arises from 

an increase in the number of students who are certified for free meals. 

By contrast, direct certification does not influence the reduced-price participation rate, either 

among all students or among students certified for reduced-price meals. The estimated effect of

each level of direct certification experience on each reduced-price participation rate is not 

statistically significant. Given that direct certification does not lead to any significant change in 

the proportion of students certified for reduced-price meals, this lack of an effect on the reducedprice participation rate is not surprising. 

And, while direct certification does not significantly influence the paid participation rate 

among all students, it does positively and significantly influence the paid participation rate 

among noncertified students. In particular, the use of direct certification for 3 to 5 years, 6 to 

10 years, and more than 10 years leads to a 2 to 3 percentage point increase in the participation

rate among students not certified for free or reduced-price meals. This effect may arise from a

small increase in the number of paid meals served (perhaps due to the spillover effect described

above) coupled with a decrease in the number of noncertified students. 

3. Summary 

71

Overall, the results of the district-level and State-level models tell a reasonably consistent

story about the effects of direct certification on rates of certification for free and reduced-price 
meals and participation in the NSLP. In addition, these results are consistent with those reported 

in Jackson et al. (1999). These three sets of estimates of the effect of direct certification on 

several key outcomes related to certification and participation are reported in Table III.8. 

Based on the results of these models, we conclude that direct certification has a positive and 

statistically significant effect on the free certification rate (and little or no effect on the reducedprice certification rate). This effect on the free certification rate is relatively small, with the

estimates ranging from 1.3 to 2.8 percentage points. Given that the average district has a free 

certification rate of about 30 percent, this percentage point effect amounts to a 4 to 9 percent 

increase in the percentage of a districts students who are certified for free meals. An alternative 

way of looking at the size of this effect is that direct certification will cause 0.4 to 0.9 million

students nationally to move from being not certified for free meals to being certified. 

The evidence indicates that direct certification has a small positive effect on the overall 

participation rate, with mixed evidence as to whether this effect is statistically significant. The 

estimated effects range from 0.5 to 1.5 percentage points, and only the State-level model 

estimate is statistically significant. If direct certification has an effect on the overall participation 

rate, results from both the State-level model and Jackson et al. (1999) suggest that this effect 

results mostly from an increase in participation among students newly certified for free meals as 

a result of direct certification. In short, the evidence suggests that direct certification leads to an 

increase in free meal certification, which, in turn, leads to a somewhat smaller increase in NSLP 

participation.

72
TABLE III.8 

ESTIMATED EFFECT OF DIRECT CERTIFICATION ON CERTIFICATION AND 

PARTICIPATION RATES, 

COMPARISON OF DISTRICT-LEVEL MODEL AND STATE-LEVEL MODEL RESULTS 

Impact of Direct Certification on: 

(1) 

District-Level

Modela

(2) 

State-Level 

Modelb

(3) 

Jackson et al.

(1999)c

Free Certification Rate 1.32** 1.42** 2.80* 

Reduced-Price Certification Rate 0.39 0.17 0.85*** 

Free Participation Rate Among All Enrolled 

Students 0.35 1.06** 1.35** 

Reduced-Price Participation Rate Among All 

Enrolled Students 0.23 0.17 0.20* 

Paid Participation Rate Among All Enrolled 

Students -0.02 0.28 -0.55 

Overall Participation Rate 0.48 1.51* 1.05 

Source: 2001 Direct Certification Study SFA Survey; various State-level data sources (see Table III.5); 

Jackson et al. (1999), Table V.6. 

a

The district-level results are drawn from the specification in which direct certification enters the model as a

single binary variable. These results are reported in Tables III.3 and III.4. 

b

The State-level results are drawn from the specification in which direct certification enters the model as a 

set of binary variables indicating the number of years in which direct certification has been in use in the

State. The estimate reported in the table is the effect of 6 to 10 years of direct certification use. 

c

The results from Jackson et al. (1999) are drawn from the specification in which direct certification enters 

the model linearly as a single variable indicating the number of years in which direct certification has been

in use in the State. The estimate reported in the table is the effect of 5 years of direct certification use

(where the estimated effect of a single year of direct certification use was multiplied by 5 to determine the

estimated effect of 5 years). The significance level reported refers to the significance of the estimated effect

of a single year of direct-certification use. 

 *Significantly different from zero at the .10 level, two-tailed test. 

 **Significantly different from zero at the .05 level, two-tailed test. 

***Significantly different from zero at the .01 level, two-tailed test. 

73

 
IV. ESTIMATED INELIGIBILITY AMONG CERTIFIED STUDENTS 

As described in Chapter I, there has been much concern recently about the integrity of the 

NSLP free and reduced-price eligibility determination system due to the perception that a

substantial proportion of certified students are ineligible for free or reduced-price meals. 

However, the evidence as to the dimensions of this problem is incomplete. Much of this 

evidence has focused on individual school districts or States, and thus does not reveal much 

about the extent of the problem nationally. The other approach taken to examine this issue has 

relied on nationally representative data that were collected for purposes other than examining the

issue of certification errors and is limited in a number of respects.36 Furthermore, the previous 

studies focusing on the issue of overcertification, or certification errors, have not examined the 

role of direct certification in program integrity. 

The Direct Certification Study provides an opportunity to examine the extent to which the 

NSLP free and reduced-price eligibility determination system provides benefits to intended 

recipients rather than to ineligible students. The study’s SFA survey collected information from

districts on the results of their verification process, which sheds light on the extent to which 

students approved for free or reduced-price meals by application have incomes that qualify them

for these benefits. Verification results, however, yield no information on the extent to which 

directly certified students are ineligible for the benefits they are receiving at a given point during

74

 36For this analysis, Tordella (2001) relied on data from the Current Population Survey (CPS). The

income information in CPS covers a different time period than that used to determine students’ eligibility 

for free and reduced-price meals. In particular, the CPS analysis based its determination of free/reducedprice eligibility on household income during the full 1999 calendar year. The number of children thus

determined to be eligible was then compared to the number approved for free meals as of October 1998.

For these approved students, however, their actual eligibility was determined based on household income

as of July, August, or September 1998. 

 
the school year. Thus, to address this issue, we collected supplemental data from State food 

stamp and/or TANF offices. 

This chapter describes the study’s estimates of the extent to which free and reduced-price

certified students are ineligible for the benefits they are receiving. Section A presents

background on ineligibility measurement and the estimates we employed. Section B describes 

the characteristics and results of districts’ income verification process, including a discussion of 

the benefit reduction/termination rate and other estimates of ineligibility among this group. 

Section C examines directly certified students, presenting an estimate of the proportion of this

group ineligible for the benefits they are receiving. And, Section D presents overall estimates of 

ineligibility among students certified for free or reduced-price meals through applications and 

students certified for free meals through direct certification. 

A. Ineligibility Measurement 

We take two alternative perspectives in examining eligibility versus ineligibility for free or 

reduced-price meal benefits, both of which exclude Provision 2 and 3 schools. One perspective 

focuses on the administrative rules for eligibility. Under this perspective, students’ eligibility 

status is determined by whether they comply with program rules, regardless of their underlying 

household circumstances. Thus, students whose incomes are 130 percent or less of poverty may 

still be administratively ineligible for free meal benefits if they have not complied with program 

rules—for example, by not providing documentation of their eligibility in response to a 

verification request. 

An alternative perspective on eligibility focuses on whether household circumstances are 

consistent with eligibility guidelines, regardless of compliance with program rules. In this 

perspective, households are considered income eligible for free meals if current income is 130

75

 
percent or less of poverty or if they receive FS/TANF/FDPIR.37 This perspective ignores 

whether households have fully complied with program rules, such as verification requests or 

reporting changes in FS/TANF/FDPIR status. 

A further complication with respect to assessing ineligibility is that it is necessary to rely on

different sources of information and methods for estimating ineligibility among different groups 

of certified students. Information on the eligibility of students certified by application is most 

readily available from the results of districts’ verification process. The verification system does

not cover directly certified students, so information on their eligibility comes from State 

administrative data. And to assess ineligibility among all certified students, we must find some

way of combining the information we have gathered on each of the groups of students described 

above. Thus, we ended up generating three sets of estimates of ineligibility among certified 

students: (1) ineligibility among students certified by application, (2) ineligibility among directly 

certified students, and (3) ineligibility among all students certified for free or reduced-price 

meals. 

1. Ineligibility Among Students Certified by Application 

To estimate the proportion of students certified by application who are ineligible for the

benefits they are receiving, we relied primarily on the results of the verification process as 

reported by districts on the SFA survey. Based on this information, we generated three estimates 

of the prevalence of errors among students certified by application: (1) the proportion of verified 

applications in which benefits were reduced or terminated, also known as the benefit 

reduction/termination rate; (2) the proportion of verified students who responded to the 

76

 37This concept of eligibility is referred to as income eligibility for simplicity, even though it can be 

determined by FS/TANF/FDPIR status rather than by income. 

 
verification request and who were found to be ineligible (and had their benefits reduced or 

terminated as a result) based on the documentation they provided; and (3) the proportion of 

students whose benefits were reduced or terminated and who did not subsequently reapply and 

be approved for benefits. 

The benefit reduction/termination rate captures the extent to which students certified by 

application are ineligible for benefits on the basis of not complying with program rules—they 

either did not provide documentation in response to the verification request or they provided

documentation showing that they did not qualify for benefits. In other words, the rate is an

estimate of the proportion of students certified by application who are administratively ineligible 

for benefits. Although some of these students may have household incomes that qualify them for 

benefits, it is appropriate to consider them to be administratively ineligible for benefits because 

they have not done what is necessary according to program rules to maintain eligibility.

The other two estimates address ineligibility of students certified by application from the 

perspective of their actual household circumstances; that is, these estimates address income

ineligibility. The proportion of verified students who respond to the verification request and

have their benefits terminated is an estimate of the proportion who we are fairly certain (based on 

the results of verification) have incomes above income eligibility thresholds and who are not

receiving FS/TANF/FDPIR. This is a very conservative estimate, or a lower bound estimate, of 

the proportion of students certified by application who are income ineligible for benefits because 

it assumes that those who did not respond to the verification request are income eligible for 

benefits. In fact, nonresponders may or may not be income eligible for benefits. We have little 

77

 
information on what proportion of nonresponders are income eligible. One piece of

information we have, however, is an estimate of the proportion of nonresponders who reapply 

and are approved for benefits. This information was provided by food service directors on the 

SFA survey. We used the information to estimate the proportion of verification cases in which 

the households either provided documentation that showed them to be income ineligible or did 

not provide documentation and subsequently failed to reapply and be certified for benefits. We

consider this proportion to be an upper bound estimate of the proportion of students certified by 

application who are income ineligible, since we believe that nonresponders who reapply and are 

approved for benefits are truly income eligible but we are not certain about the eligibility status

of nonresponders who do not reapply for benefits. 

38

Each of the estimates of ineligibility among students certified by application relied on the 

results of verification. One limitation of this methodology is that the verification system was not 

designed to measure eligibility among students certified by application. Instead, it was designed 

to detect ineligibility among currently certified students and to deter ineligible students from 

becoming certified. Because of this objective, rules permit districts to select a random sample or

a focused sample for verification. While the basic goal of random sampling is to select a simple

random sample, the actual methods used in practice may deviate significantly from that goal. 

Despite these deviations from scientific practice it is quite likely that the results of random 

selection across the many districts nationwide result in samples that are reasonably representative 

78

 38An FNS study conducted by Abt Associates during the 1986-87 school year provides some insight

about ineligibility among families that do not respond to the verification request (USDA 1990). In that 

study, 10 percent of those selected for verification did not respond. Household audits of these 

nonresponders found that 67 percent remained income eligible for the benefits they were approved for;

18.7 percent were income ineligible for NSLP benefits; and 14.3 percent were eligible only for reducedprice meals even though they had been certified for free meals. The districts from which these estimates

were computed all used random sampling to conduct verification. This study was conducted before direct 

certification was available to districts. 

 
of the underlying population of applications. Subject to these limitations, verification results 

from random samples can be generalized to all students certified by application. With focused 

samples, however, the underlying population to which the verification sample generalizes is not 

clear. Our approach to dealing with this aspect of the verification system is that after initially 

presenting verification results among all districts, we limit the remainder of the analysis to 

districts that selected random verification samples. This allows results to be generalized to 

students certified by application in those districts. But those districts may be different from 

districts that selected focused verification samples; thus, the results are not representative of 

verification results among students certified by application in all public school districts. 

 39

In interpreting the results of our analysis of verification data it is important to consider the

possible sources of inaccuracy identified through the current verification process. These include: 

79

•

•

•

•

 

Intentional misreporting of circumstances at the time of application 

Inadvertent misreporting of circumstances at the time of application 

Unreported changes in household circumstances between application and the point at which 

verification is conducted 

SFA errors in processing information provided by household 

All of these sources contribute in unknown degrees to the differences found between applications 

certified in late summer or fall of the school year and family income status or qualifying

assistance program participation at verification in December of each school year. 

Another feature of the verification system is that it does not cover directly certified students. 

The exclusion of directly certified results complicates the interpretation of verification results. 

39 According to the 1986-87 Study of Income Verification, over half of all districts used some sort of

quasi-random procedure to select applications for verification. However, that study noted that “for the

purpose of selecting applications for verification, it is likely that such methods provide an adequate 

substitute for a truly random selection procedure.” 

 
Although verification results provide information on ineligibility among students certified by 

application, the measures of ineligibility among this group will have different interpretations,

depending upon what proportion of certified students in the district became certified by 

application. In a district that does not use direct certification (and is not a Provision 2 or 3 

school), all students who are certified completed an application, and so verification results can be

generalized to all certified students. On the other hand, in a district in which a large number of 

students are directly certified, the verification pool does not contain all certified students, and 

students in the verification pool may have a different underlying likelihood of being ineligible 

than directly certified students. In this situation, the verification results will not be a very good 

guide to the overall level of ineligibility among all certified students. Thus, we need to be 

particularly careful in interpreting differences in the verification results of districts that do and do 

not use direct certification (or in which different proportions of students are directly certified), 

since such differences may arise even if the overall levels of ineligibility among all certified 

students are similar in the two types of districts. 

Despite these limitations, the verification system provides a great deal of useful evidence on 

the levels of ineligibility among certified students. Verification is required in all NSLP districts, 

and most districts select random verification samples. Furthermore, most certified students are 

certified by application rather than being directly certified; therefore, most are covered by the

verification system. Finally, while verification results alone should not be used to estimate levels 

of ineligibility among all certified students, they can be combined with estimates of ineligibility 

among directly certified students to generate estimates of overall ineligibility. 

80

 
2. Ineligibility Among Directly Certified Students 

To estimate ineligibility among directly certified students, we developed an approach 

analogous to the verification process, in that we examined the status of directly certified students 

as of December of the school year.40 Conceptually, the idea was to estimate the proportion of

students directly certified at the beginning of the school year who were not eligible for benefits 

in December. We again used two perspectives of ineligibility. Directly certified students were 

defined as administratively ineligible if they were no longer receiving FS/TANF/FDPIR as of 

December and yet remained directly certified because they did not report their change in status to 

the SFA. These students, or their districts, had not complied with program rules in the sense that

either they had not reported their change in FS/TANF/FDPIR status to the district or, if they had, 

the district had not acted on this information. According to program rules, directly certified 

students who exit FS/TANF/FDPIR are to report this information to the district. If the student 

remains income eligible for free meals, he or she can complete an application at that time and 

remain eligible. If the student is no longer income eligible, program rules require that benefits be 

terminated. 

We measured the proportion of directly certified students administratively ineligible for 

benefits by estimating an FS/TANF/FDPIR turnover rate among directly certified students. 

Conceptually, this turnover rate was defined as the proportion of directly certified students at the 

beginning of the 2001-2002 school year who were no longer receiving FS/TANF/FDPIR as of 

December 2001. The turnover rate was estimated using State administrative data. In particular,

for the zip code areas of schools located in each of the direct certification districts in the SFA 

survey, we used the data to first identify each school-age child receiving FS/TANF during 

81

 40For more information on ineligibility among directly certified students, see Appendix D. 

 
summer 2001, when students potentially eligible for direct certification for the 2001-2002 school 

year were identified by districts and/or States. We then examined the December 2001 FS/TANF 

status of each of these children to measure the turnover rate—the proportion of the directly

certified group who were no longer receiving assistance as of December.41

Some of the administratively ineligible, directly certified students may have retained their

income eligibility for free meal benefits.42 Thus, we also estimated a second measure of 

ineligibility, the proportion of directly certified students no longer on FS/TANF/FDPIR and in 

households with incomes above 130 percent of poverty. To calculate this estimate of income

ineligibility among directly certified students, we adjusted the FS/TANF turnover rate to account 

for households that may have left FS/TANF but remained eligible for free meals on the basis of 

their income. We first used information from the 1996 panel of the Survey of Income and 

Program Participation (SIPP) to identify school-age children who were FS/TANF leavers 

between summer 1999 and December 1999. Among this group, we then calculated the 

adjustment factor—the proportion of leavers whose December income was above 130 percent of 

Federal poverty guidelines. This adjustment factor was multiplied by the FS/TANF turnover rate 

in each sample district to generate the estimate of the proportion of directly certified students

who are income ineligible for benefits. For example, if the proportion of leavers with incomes 

above 130 percent of poverty turned out to be 40 percent and the FS/TANF turnover rate in a 

given district was 25 percent, then the estimate of the proportion income ineligible in that district 

would be 10 percent. 

82

 41One weakness of this approach was that not all students who received FS/TANF in summer 2001

were necessarily directly certified. 

42For example, studies of both TANF leavers (Rangarajan and Wood 1999; Kuhns et al. 1999) and

food stamp leavers (Rangarajan and Gleason 2000) have found that a substantial fraction of leavers 

remain poor even after exiting these programs. 

 
3. Ineligibility Among All Certified Students 

Finally, we took the critical step of combining the estimates of ineligibility among students 

certified by application and students directly certified. We did this both for the estimates of

administrative ineligibility and income ineligibility. These combined estimates of ineligibility 

among all certified students are used in the next chapter in estimating the effects of direct 

certification on program integrity. 

The estimate of the proportion of all certified students ineligible for the benefits they are 

receiving was calculated as a weighted sum of the rate of ineligibility among students certified 

by application and the rate of ineligibility among students directly certified. The weights in this 

calculation were set to the percentage of certified students who were certified by application and 

the percentage directly certified. For example, if 30 percent of certified students were directly 

certified in a district, the rate of ineligibility among all certified students would be set to 0.70 

times the rate among those certified by application times 0.30 times the rate among directly 

certified students. For districts not using direct certification, 0 percent of students were directly 

certified, so the estimate of the proportion of all certified students ineligible for benefits was

determined entirely by the proportion ineligible among students certified by application. 

As noted above, we generated separate estimates of the proportion of all certified students 

administratively ineligible and the proportion income ineligible. In particular, the proportion 

administratively ineligible was based on the benefit reduction/termination rate among students

certified by application and the FS/TANF turnover rate among directly certified students. 

Correspondingly, the upper bound estimate of the proportion income ineligible was based on the 

proportion of verified students whose benefits were reduced or terminated and who did not

reapply for benefits among students certified by application and on the proportion of directly 

83

 
certified students who exited FS/TANF and were in households with incomes above 130 percent 

of poverty. 

B. THE VERIFICATION PROCESS AND RESULTS 

1. The Income Verification Process 

Three dimensions of the income verification process described above involve its timing, the 

type of verification sample the district selects, and the size of this verification sample. With 

respect to the timing of income verification, districts are required to complete the process by 

December 15. They may select either a random or a focused verification sample, or they may 

verify all approved applications. The sample must be above a minimum size, which depends 

upon whether they have selected a random or focused sample. Districts selecting a random

sample must verify at least 3 percent of all approved applications (or 3,000 applications, if that is 

smaller than 3 percent). Districts selecting a focused sample must verify at least 1 percent of all 

approved applications (or 1,000 applications, if that is smaller), plus 0.5 percent of applications 

approved on the basis of a FS/TANF/FDPIR case number (or 500 such applications, if that is 

smaller). The 1 percent portion of this focused sample must be selected from among those 

approved on the basis of household size and income and that report a monthly household income

within $100 (or within $1,200 if they report annual income) of the income eligibility guideline

for their household. 

Most districts conduct the verification process during the late fall and complete it by 

December. According to the SFA survey, a majority of districts—60 percent—completed the 

process in December, while most of the remaining districts completed it earlier (Table III.1). 

Only 5 percent of all districts completed the verification process after December, presumably 

including some that had obtained waivers extending this deadline. 

84

 
Nearly 9 of 10 districts chose to verify a random sample of applications, rather than a 

focused sample or some other type of sample (Table IV.1).43 Direct certification districts were 

more likely than non-direct certification districts to verify a focused sample and less likely to

verify a random sample. 

Most districts verify few applications as part of the income verification process during a 

given year. During the 2001-2002 school year, 60 percent of districts reported that they verified 

no more than 10 applications, while nearly 80 percent verified no more than 25 applications 

(Table IV.1); the median number of applications verified across all districts was 7. At the other 

end of the spectrum, a handful of districts verified a large numbers of applications—5 percent 

verified more than 100 during the 2001-2002 school year. 

The small number of applications verified by most districts is not all that surprising, given 

that most districts are relatively small and are required to verify at most 3 percent of approved 

applications.

44 It appears, however, that a few districts did not verify the required number of 

applications during the 2001-2002 school year. In particular, 21 percent of districts verified less 

than 3 percent of their approved applications (Table IV.1). There are two legitimate reasons for

verifying less than 3 percent. First, districts using focused sampling have lower requirements for 

the number of applications to be verified. Second, very large districts may verify only 3,000 

applications, even if this number represents fewer than 3 percent of all approved applications. 

However, only 12 percent of districts use focused sampling and less than 1 percent of districts 

85

 43Most of the 2 percent of districts that used some other type of verification sample reported that they

verified all approved applications. 

44See Appendix Table C.1 for information on the distribution of enrollment among public school

districts offering the NSLP nationally.

 
Table IV.1 

CHARACTERISTICS OF VERIFICATION IN PUBLIC SCHOOL DISTRICTS NATIONALLY 

 (Standard Errors in Parentheses)

DC 

Districts 

Non-DC 

Districts All 

Districts 

Month Verification Process Completed (Percentages) 

 October or earlier 14 

(2.0) 

22 

(3.1) 

18 

(1.8) 

 November 19 

(1.9) 

16 

(2.2) 

18 

(1.4) 

 December 63 

(2.5) 

57 

(3.3) 

60 

(2.0) 

 January or later 4 

(0.9) 

6 

(1.3) 

5 

(0.8) 

Method of Selecting Verification Sample (Percentages) ** 

 Random 85 

(1.5) 

90 

(1.8) 

87 

(1.2) 

 Focused 14 

(1.4) 

8 

(1.7) 

12 

(1.1) 

 Other 1 

(0.5) 

2 

(0.7) 

2 

(0.5) 

Number of Applications Verified (Percentages) ** 

 1 to 10 64 

(2.1) 

55 

(3.2) 

60 

(1.8) 

 11 to 25 16 

(1.5) 

24 

(2.3) 

19 

(1.3) 

 26 to 100 15 

(1.3) 

17 

(2.2) 

16 

(1.2) 

 More than 100 4 

(0.5) 

5 

(1.0) 

5 

(0.5) 

 Median 5.9 

(0.7) 

8.3 

(1.3) 

6.6 

(0.5) 

Percentage of Applications Verified (Percentages) *** 

 1 to 2 percent 25 

(2.0) 

16 

(2.0) 

21 

(1.4) 

 3 percent 30 

(2.1) 

36 

(3.0) 

32 

(1.7) 

 4 to 5 percent 29 

(2.6) 

27 

(3.2) 

28 

(2.0) 

 6 to 10 percent 14 

(2.1) 

15 

(2.6) 

14 

(1.7) 

 11 to 100 percent 2 

(0.8) 

6 

(2.0) 

4 

(1.0) 

Mean 5.0*

(0.5) 

8.1 

(1.6) 

6.8 

(0.8) 

Sample Size 764 404 1,172 

SOURCE: 2001 Direct Certification Study SFA Survey.

 *DC/Non-DC difference is significantly different from zero at the .10 level, two-tailed test 

 **DC/Non-DC difference is significantly different from zero at the .05 level, two-tailed test. 

***DC/Non-DC difference is significantly different from zero at the .01 level, two-tailed test. 

86

 
are so large that they are required to verify fewer than 3,000 applications. Thus, a minimum of 8 

percent of districts fail to verify the required number of applications. 

Direct certification districts tend to verify fewer applications than non-direct certification 

ones, in both absolute and percentage terms. For example, 64 percent of direct certification 

districts verified no more than 10 applications in the 2001-2002 school year, compared with 54 

percent of non-direct-certification districts (Table IV.1). Similarly, 24 percent of direct 

certification districts verified fewer than 3 percent of approved applications, compared with 15 

percent of non-direct-certification districts. One reason for the difference in the number of 

applications verified is that, since directly certified students do not complete an application,

direct certification districts have fewer applications for the same number of certified students. In 

addition, direct certification districts are more likely to use focused sampling, which also 

requires fewer applications to be verified. 

2. Results of Income Verification 

In the average public school district nationally during the 2001-2002 school year, 

verification resulted in no change in benefits for just over two-thirds of households whose

applications were verified (69 percent) and resulted in an increase in benefits (from reducedprice to free) for 2 percent of households (Table IV.2). The remaining 29 percent of households 

whose applications were verified had their benefits reduced or terminated, including 23 percent 

whose benefits were terminated and 6 percent whose benefits were reduced.45 

87

 45Among those whose benefits were terminated, about 6 in 10 moved from free certification to paid

status, and the remaining 4 in 10 moved from reduced-price certification to paid status. 

 
TABLE IV.2

RESULTS OF DISTRICTS’ VERIFICATION PROCESS 

(Standard Errors in Parentheses)

DC 

Districts 

Non-DC 

Districts 

All 

Districts 

Verification Results (Means) 

No change in benefits 67 

(1.6) 

70 

(2.0) 

69 

(1.3) 

Benefits increased 2 

(0.4) 

3 

(0.7) 

2 

(0.4) 

Benefits reduced 5 

(0.5) 

7 

(1.2) 

6 

(0.5) 

Benefits terminated 25** 

(1.4) 

20 

(1.5) 

23 

(1.1) 

Among Districts Using Random Sampling (Means) 

No change in benefits 70 

(1.7) 

71 

(2.2) 

71 

(1.3) 

Benefits increased 3 

(0.4) 

3 

(0.8) 

3 

(0.4) 

Benefits reduced 4** 

(0.4) 

7 

(1.4) 

5 

(0.6) 

Benefits terminated 23* 

(1.6) 

19 

(1.6) 

22 

(1.2) 

Among Districts Using Focused Sampling (Means) 

No change in benefits 49 

(4.0) 

59 

(8.6) 

52 

(3.9) 

Benefits increased 2 

(0.5) 

1 

(0.6) 

2 

(0.4) 

Benefits reduced 13* 

(1.9) 

8 

(1.9) 

11 

(1.5) 

Benefits terminated 37 

(3.2) 

32 

(6.6) 

35 

(3.0) 

Benefit Reduction/Termination Rate (Percentages)

0 33

(2.7) 

34 

(3.4) 

34 

(2.1) 

1 to 20 10 

(1.7) 

14 

(2.0) 

12 

(1.3) 

21 to 40 23 

(1.9) 

22 

(2.4) 

22 

(1.5) 

41 to 60 18 

(1.7) 

18 

(2.2) 

18 

(1.3) 

61 to 100 16 

(1.7) 

12 

(2.0) 

15 

(1.3) 

88
Table IV.2 (Continued)

DC 

Districts 

Non-DC 

Districts 

All 

Districts 

Mean 30.2

(1.6) 

27.1 

(1.9) 

28.8 

(1.2) 

Reason for Benefit Reduction/Termination (Means) 

Percentage because of nonresponse 59* 

(2.5) 

49 

(3.0) 

56 

(1.9) 

Percentage with income too high 38** 

(2.5) 

45 

(3.1) 

40 

(2.0) 

Percentage with other reason 3 

(0.9) 

6 

(1.7) 

4 

(0.8) 

Among Districts Using Random Sampling or Verifying All 

Approved Applications (Means)

 Mean benefit reduction/termination rate 27.3 

(1.7) 

25.9 

(2.0) 

26.6 

(1.3) 

Percentage with benefits reduced or terminated because of 

nonresponse 

 60** 

(2.9) 

48 

(3.2) 

55 

(2.2) 

Percentage with benefits reduced or terminated because 

income too high 

 37** 

(3.0) 

46 

(3.3) 

40 

(2.2) 

Percentage with benefits reduced or terminated for other 

reason 

3 

(1.0) 

6 

(1.8) 

4 

(1.0) 

Sample Size (for Verification Results) 715 376 1,093

Sample Size (for Reason for Benefit 

Reduction/Termination) 

568 286 854

Source: 2001 Direct Certification Study SFA Survey. 

 *DC/Non-DC difference is significantly different from zero at the .10 level, two-tailed test 

 **DC/Non-DC difference is significantly different from zero at the .05 level, two-tailed test. 

***DC/Non-DC difference is significantly different from zero at the .01 level, two-tailed test. 

89
We also examined the results of the verification process separately for districts that used 

random sampling versus those that used focused sampling. Since focused sampling largely 

involves applications of households whose reported incomes are close to the eligibility 

thresholds, a larger proportion may have their benefits changed, since minor variations in income 

could result in a change in eligibility status. 

The results of income verification differed significantly for districts using random sampling 

versus focused sampling. Districts using focused sampling terminated the benefits of an average 

of 35 percent of households whose applications were selected for verification, compared with 22 

percent among districts using random sampling (Table IV.2). Similarly, benefits were reduced 

for 11 percent of households in focused sampling districts, and just 5 percent of households in 

random sampling districts. It is important to keep in mind, however, that 87 percent of districts 

used random sampling, while only 12 percent used focused sampling. Thus, random sampling 

districts dominate the verification results reported here for all school districts. 

The percentage of applications in a district’s verification sample whose benefits are reduced 

or terminated is referred to as the benefit reduction/termination rate.46 The mean benefit 

reduction/termination rate across all districts was 29 percent during the 2001-2002 school year 

(Table IV.2). In districts that used random sampling, this rate was 27 percent.47 The benefit 

 46Sometimes this percentage is called the “verification error rate.” The verification error rate could 

also be defined to include the applications with benefits increased as a result of verification, though this

group is small. 

47Throughout the remainder of the analysis, we use the benefit reduction/termination rate as a 

measure of the percentage of students certified by application who are administratively ineligible, even 

though the benefit reduction/termination rate is based on an analysis of applications rather than students. 

In districts that use multi-child applications, this could be misleading if families with multiple children 

are systematically more (or less) likely to have their benefits reduced or terminated. However, this

potential problem is unlikely to strongly affect our results for two reasons. First, a relatively small 

proportion of districts uses multi-child applications (21 percent). Second, the benefit 

reduction/termination rate in districts that use single child applications (25.4 percent) is close to the rate 

among all districts (26.6 percent).

90
reduction/termination rate varied widely across districts. In about one-third of the districts, the 

benefit reduction/termination rate was zero, since no households had their benefits reduced or 

terminated as a result of the verification process. On the other hand, a third of all districts had 

benefit reduction/termination rates of more than 40 percent, and 15 percent of districts had rates

exceeding 60 percent. Much of this variation across districts in the benefit reduction/termination 

rate reflects sampling variability rather than variation across districts in the underlying levels of

ineligibility, since the rate in each districts is based on often extremely small verification 

samples. 

The final aspect of the income verification results we examined involves the reasons that 

benefits were reduced or terminated by districts. Benefits may be reduced or terminated through 

the verification process for one of two main reasons. First, households that provide 

documentation showing that their income is higher than the income threshold for their current 

level of certification have their benefits either reduced or terminated. For example, a household 

receiving free meal benefits whose applications is verified may produce income documentation 

showing that their income is above 130 percent of Federal poverty guidelines. The benefits of

this household would be reduced if this documented income were between 131 and 185 percent 

of poverty, and would be terminated if the household’s documented income were greater than 

185 percent of poverty. Second, households that do not respond to the request for income

documentation or that do not provide sufficient documentation within a reasonable period of

time of the verification request would have their benefits terminated. 

In the average district during the 2001-2002 school year, the reason for benefit 

reduction/termination was nonresponse to the request for documentation in more than half (56 

percent) of all cases in which benefits were reduced or terminated (Table IV.2). In most of the 

remaining cases, benefits were reduced or terminated because the documentation provided 

91
indicated that the household’s income level (along with household size) made the household 

ineligible for the benefits they were receiving. 

Among cases in which benefits were reduced or terminated due to nonresponse, households 

may have failed to respond for a variety of reasons. Some households may not have responded 

to the request for documentation because they realized that they were no longer eligible for the

benefits they were receiving. Other nonresponding households, however, may have forgotten 

about the documentation request, not gotten around to complying with it, lost the information 

needed to comply with the request, or decided that it was too intrusive or too much work. 

Households not responding to the verification request for these reasons may or may not have had 

income or FS/TANF status that qualified them for the benefits they were receiving at the time

these benefits were terminated: the true income eligibility status of these households is unknown. 

As part of the SFA survey, food service directors were asked to estimate the proportion of the

households whose benefits were terminated due to nonresponse who subsequently reapplied and 

were approved for benefits. On average, districts reported that 35 percent of these nonresponders

reapplied and were approved for benefits. This estimate gives us a rough sense of the extent to 

which those who did not respond to the verification request actually were eligible for benefits at 

the time their benefits were terminated.48

3. Alternative Estimates of Ineligibility Among Students Certified by Application 

We examined rates of ineligibility from two perspectives—that of the typical district and

that of certified students, regardless of which district they attended. The estimated ineligibility

 48Actually, the proportion of nonresponders who were eligible is probably somewhat higher than

this, since some nonresponders may have been eligible for benefits even if they did not subsequently

reapply. As noted previously, the 1986-1987 Study of Income Verification found that two-thirds of those 

whose benefits were terminated due to non-response were eligible for either their current level of benefits 

or a higher level of benefits. 

92
rates turned out to be very different for the two perspectives, since the results of verification

differed greatly in large districts (affecting large numbers of students) than in small districts 

(affecting fewer students). 

Figure IV.1 summarizes our estimates of the rates at which students certified by application 

are ineligible for benefits, from both perspectives. With these and all subsequent estimates of 

ineligibility rates based on verification results, we excluded districts that selected focused 

verification samples.49 At both the district level and the certified student level, three estimates

are provided. The first estimate shows the benefit reduction/termination rate, which captures

administrative ineligibility among students certified by application and subject to verification. 

As described above, this estimate of the benefit reduction/termination rate during the 2001-2002 

school year was 27 percent in the average district that used random sampling. 

The other two estimates shown in Figure IV.1 are intended to capture income ineligibility. 

The second estimate, labeled “Lower Bound,” shows the percentage of verified applications in 

which the household provided documentation showing that they were not eligible for the benefits 

they were receiving. These households responded to the verification request and had their 

benefits reduced or terminated anyway on the basis of the information they provided. In this 

definition, households that did not respond to the verification request were not considered 

ineligible, despite the fact that their benefits were terminated. As such, it should be considered 

as a lower bound on the rate of income ineligibility among students certified by application. In 

 49We excluded focused sampling districts because we wanted the verification results for a given 

district to be representative of all students certified by application in that district. Randomly selected

verification samples are representative of this population, whereas focused samples are not. We also

included districts that verified all approved applications in the sample used to estimate rates of

ineligibility based on verification results. One limitation of excluding focused sampling districts is that

the estimates are no longer representative of all public school districts that offer NSLP lunches, just those 

that decided not to select a focused verification sample. 

93
94

38

20.8

13.5

11.3

49.2

26.6

0 20 40 60

Among

Students

Among

Districts

1. BRT

2. Lower Bound

3. Adjusted BRT

(1.8)

(0.9)

(1.6) 

(1.2)

(1.0)

(1.3)

BRT = Benefit reduction/termination rate 

Lower Bound = Percentage who responded to verification request and had 

benefits reduced or terminated 

Adjusted BRT = Percentage who had benefits reduced/terminated and did not 

reapply for benefits 

NOTE: Sample excludes only districts that selected focused verification samples.

The sample size is 863. 

Figure IV.1 

Alternative Estimates of Ineligibility Among 

Students Certified by Application

(Standard Errors in Parentheses)
the average district during 2001-2002, this lower bound was 11 percent. In other words, 11 

percent of verified households responded to the verification request but still were found to be in 

error and had their benefits reduced or terminated. 

The third estimate of ineligibility, labeled “Adjusted BRT,” uses the information provided 

by districts on the proportion of nonresponders who reapplied and were approved for benefits to 

adjust the benefit reduction/termination rate. This estimate is calculated as the percentage of

verified applications in which either (1) the household responded to the verification request and 

were found to be ineligible for the benefits they were receiving; or (2) the household did not 

respond to the verification request, had their benefits terminated, and did not reapply for benefits. 

In the average district, this estimate of ineligibility was 21 percent during the 2001-2002 school 

year. This estimate of income ineligibility is an upper bound, since households that did not 

reapply for benefits may or may not have been income eligible. 

Estimates of rates of ineligibility look quite different when measured across all certified

students, rather than for the average district. In fact, certified students whose applications were 

verified were nearly as likely to have their benefits reduced or terminated as they were to have 

their benefits increased or not changed as a result of verification. In other words, the estimated 

benefit reduction/termination rate (that is, the estimated rate of administrative ineligibility)

among certified students subject to verification was 49 percent (Figure IV.1). The reason this

estimate differs so much from the estimate of the benefit reduction/termination rate in the

average district is that the student-level estimate is driven largely by what happens in the largest 

districts, and these large districts tend to have the highest benefit reduction/termination rates. 

In these largest districts, however, many of those whose benefits were reduced or terminated 

failed to respond to the verification request. Thus, the estimate of the lower bound of the 

percentage of students certified by application who are income ineligible, which assumes that

95
these nonresponders were not ineligible, was much lower than the benefit reduction/termination 

rate. This lower-bound estimate was 14 percent during the 2001-2002 school year—that is, 14 

percent of certified students whose applications were verified responded to the verification 

request and had their benefits reduced or terminated (Figure IV.1). Finally, the adjusted benefit 

reduction/termination rate, the upper bound estimate that takes into account whether 

nonresponders reapplied and were approved for benefits, was 38 percent. This estimate is based 

on the percentage of students whose benefits were reduced or terminated as a result of 

verification and those who subsequently failed to reapply for benefits. 

Verification results appear not to be strongly correlated with whether or not districts use 

direct certification, despite the fact that verification excludes directly certified students. The

benefit reduction/termination rate was 27 percent in the average direct certification district and

26 percent in the average non-direct certification district, and this difference was not statistically 

significant (Table IV.2).50 On the other hand, non-response was significantly higher in direct 

certification districts than in non-direct certification districts (60 percent versus 48 percent). 

It is important to note, however, that differences in verification results between direct 

certification and non-direct certification districts do not indicate how direct certification 

influences overall levels of ineligibility for benefits—for two main reasons. First, verification

samples in non-direct certification districts (using random sampling) are generalizable to all 

certified students, while verification samples in direct certification districts are generalizable 

only to those students certified by application and are not directly certified. To make these 

 50Among certified students, this difference was slightly larger. Among certified students whose

applications were verified, 51 percent of those in direct certification districts and 47 percent of those in

non-direct certification districts had their benefits reduced/terminated. However, this difference was not 

statistically significant. 

96
estimates of ineligibility comparable across the two types of districts, it is important to estimate 

the rate of ineligibility among all certified students in both types of districts.51 Second, this 

comparison of ineligibility rates among direct certification and non-direct certification districts 

does not take into account other differences between the two types of districts that could 

influence estimated rates of ineligibility, such as underlying poverty levels in the districts.52

Table IV.3 shows benefit reduction/termination rates by district characteristics, again

excluding districts that selected focused verification samples. The table shows the mean benefit 

reduction/termination rate of districts by direct certification implementation type, enrollment, 

certification rate, percentage of certified students who are directly certified, urbanicity, and 

region. The characteristic most closely related to the benefit reduction/termination rate is district 

enrollment, with the error rate significantly higher in larger districts. In particular, the mean 

error rate in districts with an enrollment of more than 10,000 students was 50 percent, compared 

with only 18 percent in districts with 1,000 or fewer students. 

 51See Section D of this chapter for this analysis of ineligibility among all certified students. 

52See Chapter V for an analysis of the effects of direct certification on rates of ineligibility that takes 

into account observable district characteristics such as poverty levels. 

97
TABLE IV.3

MEAN BENEFIT REDUCTION/TERMINATION RATES,

BY DISTRICT CHARACTERISTIC 

(Standard Errors in Parentheses)

District Characteristic 

Percentage of 

Districts with 

Characteristic 

Mean District-Level 

Benefit Reduction/ 

Termination Rate 

Type of Direct Certification Used * 

 No DC 40 25.9 

(2.0) 

 Non-Matching 13 28.4 

(3.5) 

 District-Level Matching, Passive Consent 18 24.4 

(2.3) 

(2.3) 

 District-Level Matching, Active Consent 6 19.3 

(3.6) 

 State-Level Matching, Passive Consent 11 32.1 

(3.2) 

 State-Level Matching, Active Consent 4 43.6 

(10.5) 

 Mixed 8 24.8 

(4.7) 

District Enrollment *** 

 1 to 1,000 51 18.0 

(2.3) 

 1,001 to 5,000 39 32.1 

(1.2) 

 5,001 to 10,000 6 43.3 

(2.4) 

 10,001 or more 4 49.9 

(1.8) 

Total Certification Rate 

 0 to 20 percent 23 29.1

(2.8) 

 21 to 40 percent 35 28.1 

(2.2) 

 41 to 60 percent 26 24.9 

(2.2) 

 61 percent or more 16 21.8 

(3.4) 

98
District Characteristic 

Percentage of 

Districts with 

Characteristic 

Mean District-Level 

Benefit Reduction/ 

Termination Rate 

Table IV.3 (continued) 

Percentage of Free Certified Students 

Directly Certified 

 0 percent (non-DC district) 40 25.9 

(2.0) 

 1 to 20 percent 24 26.5 

(2.2) 

 21 to 40 percent 29 29.8 

(2.8) 

 41 percent or more 7 22.2 

(4.7) 

Urbanicity ** 

 Urban 5 37.7 

(5.6) 

 Suburban 42 30.1 

(2.0) 

 Rural 54 25.2 

(2.2) 

Source: 2001 Direct Certification Study SFA Survey. 

 *DC/Non-DC Difference Is Significantly Different From Zero At The .10 Level, Two-Tailed Test 

 **DC/Non-DC Difference Is Significantly Different From Zero At The .05 Level, Two-Tailed Test. 

***DC/Non-DC Difference Is Significantly Different From Zero At The .01 Level, Two-Tailed Test. 

99
C. ESTIMATING INELIGIBILITY AMONG DIRECTLY CERTIFIED STUDENTS 

The verification process described above applies only to students approved for free or 

reduced-price meals on the basis of an application. Students who are directly certified for free

meals do not complete an application and are not subject to the verification process. We 

therefore had to estimate the levels of administrative and income ineligibility using the methods 

outlined above in section A.2. 

1. Limitations

Our two-stage estimation strategy does not correspond precisely with the conceptual 

definitions of administrative and income ineligibility described above. The conceptual definition

of the rate of administrative ineligibility is: the rate at which students who were directly certified

stop receiving FS/ TANF benefits but remain directly certified by the same district in December.

The conceptual definition of the rate of income ineligibility simply adds the condition that those 

who stop receiving FS/TANF must also have household incomes above 130 percent of poverty to 

be considered ineligible. To actually estimate the rate of administrative ineligibility, we used the 

rate at which students on FS/TANF (when direct certification was conducted in the State) in the 

zip code areas covered by schools in the sample districts had stopped receiving FS/TANF by 

December. Two main limitations of this method were: (1) students who were receiving FS

and/or TANF in the month when directly certified students were identified in the State may not

actually have become directly certified by the sample districts; and (2) students who became 

directly certified may not have remained directly certified or attended the same district by

December. 

a. Students Receiving FS/TANF Who Were Not Directly Certified by Sample Districts 

Students on FS/TANF when direct certification was conducted (that is, those who appeared 

to be eligible for direct certification) may not actually have become directly certified by sample 

100
districts, for at least three reasons. First, the zip code areas used to define the sample districts 

may not have corresponded exactly to the districts’ enrollment areas. In cases where school

districts enrolled only some children from a zip code area, we had no way of determining which 

children should be matched with the districts. We included all children living in all the zip code 

areas contained by a district.53 In one State, we were able to examine how well school 

enrollment areas matched up with zip code areas. This State provided FS/TANF administrative 

data including a “town code” variable indicating the towns in which children receiving benefits 

resided. The town codes corresponded with school district areas. Using this variable, we 

generated rates of administrative ineligibility among directly certified students in two different 

ways: (1) using town codes to group children with school districts, and (2) using zip codes to

determine school districts. We compared the estimated rates of ineligibility from the two sets of 

analysis and found that the resulting error rates were similar.54

Second, not all students in the zip code areas of the sample districts actually attended the 

sample districts. They may have attended private or charter schools, been home-schooled, or 

dropped out of school. Since the data did not include school enrollment status, however, all 

children in the area covered by a particular district were included in the sample for that district.

Although we do not know how many children in the areas covered by the public school districts 

in the sample did not attend the sample district, data from the United States Department of 

Education indicates that, in the 1999-2000 school year, roughly 1 to 2 percent of low-income 

 53In a few cases, two school districts named the same zip code area as being part of their district;

here, we included the children in the zip code area in the sample to be analyzed for both school districts. 

54The mean rate of administrative ineligibility among districts in this state was 28.3 percent when the 

town codes were used and 27.4 percent when the zip codes were used. 

101
children enrolled in elementary and secondary schools in the United States were in private 

schools (Gruber et al., 2002).55

Third, and probably most important, even among students in these areas enrolled in the 

sample districts and eligible for direct certification on the basis of FS/TANF receipt, not all 

actually became directly certified. In districts where active consent was required, for example, 

households may not have returned letters to schools or taken the steps necessary for their 

children to become directly certified. In districts where matching was used, problems in the 

implementation of matching the FS/TANF and enrollment lists may have resulted in eligible

children being missed and not becoming directly certified. There is some evidence (see Chapter 

II) to suggest that due to implementation issues, many eligible children did not, in fact, become

directly certified. If the FS/TANF turnover rate among these students was different from the rate 

among students who were directly certified, then this limitation may have affected the results of

our analysis.56 

 55This estimate is based on data from the Schools and Staffing Survey, 1999-2000. Public and 

private school administrators were asked to estimate the proportion of students who were (or would be, if 

non-NSLP schools participated) eligible for free or reduced-price meals. We adjusted these estimates for

the fact that about 25 percent of private school administrators could not provide an estimate of eligible

students. Although these eligibility estimates are not completely reliable indicators of income status, 

especially among private school students, the numbers suggest that the proportion of low-income students 

attending private schools is small. 

56For this limitation to have strongly affected our estimate of ineligibility, two conditions must have 

held. First, a substantial proportion of those eligible for direct certification must have failed to become 

directly certified. Second, the FS/TANF turnover rate of eligible students who were not directly certified 

must have been substantially different from the turnover rate among those who were directly certified. 

We have no information on the turnover rates among these two groups; however, we have no reason to

believe that the rates were substantially different. 

102
b. Students No Longer Directly Certified by December in Sample Districts 

As noted above, it is possible that not all students directly certified in the summer remained 

directly certified in that district through December of the school year. For example, students 

who were directly certified by a sample district, but stopped receiving benefits by December, 

may have alerted the school district of their change in FS/TANF status and had their free meal 

benefits terminated by the district. Alternatively, these students may have moved out of the 

school district. If they moved within the same State, then the State administrative data used for 

the analysis would still have tracked their FS/TANF status, because the administrative data were 

matched at the State level prior to being grouped by school district. If they moved out of State, 

however, they simply disappeared from the State FS/TANF rolls, and we treated them as having 

exited FS/TANF. In other words, we could not distinguish between children who disappeared 

from the FS/TANF rolls because their families stopped receiving assistance versus those who 

disappeared because they moved out of State. However, based on analysis of SIPP data, we 

concluded that the proportion of children on FS/TANF in the summer who had moved out of 

State by December was very small.57

2. Estimated Rates of Ineligibility Among Directly Certified Students 

Table IV.4 presents the estimated rates of ineligibility among directly certified students in 

all public school districts that used direct certification and were located in States that provided 

administrative data.58 The first column shows the estimated rates of ineligibility in the average

 57Among those children in our SIPP analysis who were receiving FS/TANF in the month in which 

direct certification was conducted, less than 1 percent had moved out of state by December. 

58Fourteen states did not provide FS/TANF data for the analysis of direct certification error rates. 

Appendix B lists states that did and did not provide data, and compares the characteristics of direct 

certification districts in these data-providing states with direct certification districts in non-providing 

states. 

103
direct certification district. The second column shows the estimated percentages of all directly 

certified students who are administratively or income ineligible for benefits. Overall, we found 

that a relatively small proportion of directly certified students were income ineligible to receive 

free meals by December. 

The mean rate of administrative ineligibility in the average direct certification district was 

28 percent during the 2001-2002 school year. In other words, between one-quarter and one-third 

of directly certified students were estimated to have exited FS/TANF by December, and would 

not have been directly certified if that status had been determined at that time. However, a large 

proportion of those students no longer on FS/TANF in December remained eligible for free 

meals based on their household income. Once the FS/TANF turnover rate was adjusted to 

disregard FS/TANF leavers with incomes low enough to have allowed them to remain eligible 

for free meals, the estimated rate of income ineligibility among directly certified students was 9

percent. Thus, fewer than 1 of every 10 directly certified students in the average district was no

longer eligible for free meals as of December 2001. 

In districts with a large number of directly certified students, the rate of ineligibility

among directly certified students tended to be somewhat lower. Thus, the estimated percentage 

of all directly certified students who were administratively ineligible for benefits was 21 percent 

in 2001-2002. In other words, approximately one in five directly certified students had exited 

FS/TANF by December. The estimated percentage of directly certified students who were 

income ineligible for free meals as of December 2001 was 6 percent. By December, about 1 out 

of every 16 students eligible for direct certification was no longer eligible for free meals.

104
TABLE IV.4

RATES OF INELIGIBILITY AMONG DIRECTLY CERTIFIED STUDENTS 

(Standard Errors in Parentheses)

Ineligibility Rate 

Among Direct Certification 

Districts 

Among Directly Certified 

Students 

Administrative Ineligibility Rate 

FS/TANF Turnover Rate 

(Percentage) 

0 to <10 percent 8 3 

10 to <20 percent 24 50 

20 to <30 percent 28 30 

30 to <40 percent 24 14 

40 to <50 percent 9 2 

50 to <75 percent 6 <1 

75 to <100 percent 2 <1 

Mean Turnover Rate 27.9 21.0 

 (Standard Error) (1.05) (0.54) 

Income Ineligibility Rate

Percentage of Directly Certified 

Students No Longer on FS/TANF 

and with Incomes Above 130 

Percent of Poverty by December 

(Percentage) 

 0 percent 5 <1

>0 to <3% 3 3 

3 to <6% 23 47 

6 to <10% 36 37 

10 to <20% 29 13 

20 to <30% 2 <1 

30% or more 1 <1 

Mean Income Ineligibility Rate 8.6 6.5 

 (Standard Error) (0.32) (0.17) 

Sample Size 608 605 

Source: Direct Certification Study State administrative data file, 1999 data from the Survey of 

Income and Program Participation. 

105
D. ESTIMATED INELIGIBILITY AMONG ALL CERTIFIED STUDENTS 

By combining the estimates of ineligibility among students certified by application and 

directly certified students, we estimated the percentage of all certified students in non-Provision 

2 or 3 schools who are ineligible for the benefits they are receiving as of December of each

school year. This rate was calculated as a weighted average of the ineligibility rates among 

students certified by application and directly certified, where the weights were the percentage of 

students who became certified via written application (applied to the rate of ineligibility among 

students certified by application) and the percentage certified by direct certification (applied to 

the rate of ineligibility among directly certified students). For example, if 30 percent of certified 

students were directly certified in a district, the overall rate of ineligibility would be set to 0.30 

times the rate of ineligibility among directly certified students, plus 0.70 times the rate of

ineligibility among students certified by application. In districts not using direct certification, the 

overall rate of ineligibility and the rate among students certified by application would be the 

same.

59

Figure IV.2 shows the percentage of all certified students in non-Provision 2 or 3 schools 

estimated to be ineligible for benefits, among districts and across all certified students. As with 

the previous estimates of ineligibility, we present estimates of rates of both administrative and

income ineligibility. The first set of estimates is a weighted average of the benefit 

reduction/termination rate and the FS/TANF turnover rate and captures administrative 

ineligibility. The second and third sets of estimates both are intended to capture income 

 59Districts that selected focused verification samples were excluded from the estimation of these

estimated rates of ineligibility among all certified students. 

106
107

1. Administrative Ineligibility - based on benefit reduction/termination rate and FS/TANF 

turnover rate 

2. Income Inelig, Lower Bound - based on percentage who responded to verification request 

and had benefits reduced or terminated and percentage of directly certified students who left 

FS/TANF and had incomes above 130 percent of poverty 

3. Income Inelig, Upper Bound - based on percentage who responded to verification request 

and had benefits reduced or terminated; those who did not respond who had benefits 

reduced/terminated and did not reapply for benefits; and percentage of directly certified students

who left FS/TANF and had incomes above 130% of poverty 

NOTE: Sample excludes Provision 2 and 3 schools, districts that selected focused verification 

samples, and districts in States that did not provide FS/TANF data. The sample size is 724. 

33.1

19.8

12.3

11.5

43.9

27.2

0 10 20 30 40 50

Among

Students

Among

Districts 1. Administrative Ineligibility 2. Income Inelig, Lower Bound

3. Income Inelig, Upper Bound

(1.5)

(0.7)

(1.5) 

(1.1)

(1.0)

(1.2)

Figure IV.2 

Estimates of Ineligibility Among 

All Certified Students

(Standard Errors in Parentheses)
ineligibility. The second estimate presented in the figure is a lower bound estimate and considers 

students in either of the following groups to be ineligible: (1) students certified by application 

who responded to the verification request and had their benefits reduced or terminated on the 

basis of the information they presented; and (2) students who were directly certified and who

exited FS/TANF by December and had household incomes greater than 130 percent of Federal

poverty guidelines. This lower bound estimate of income ineligibility treats all other students as 

being eligible, including those whose benefits were reduced or terminated as a result of 

nonresponse to the verification request, as well as directly certified students who exited 

FS/TANF but whose household income remained below 130 percent of poverty. The third 

estimate is an upper bound estimate of income ineligibility that adds to the ineligible population, 

in addition to those already in the lower bound group, students certified by application whose 

benefits were terminated due to nonresponse and who did not subsequently reapply for benefits. 

The first estimate suggests that the estimated rate of administrative ineligibility among 

certified students was 27 percent in the average district by December of the 2001-2002 school 

year. The estimated rates of income ineligibility were lower than this estimated rate of

administrative ineligibility. The lower bound estimate of income ineligibility in the average 

district was 12 percent. This estimate does not include any students who failed to respond to the 

verification request as ineligible. When such students are treated as ineligible if they failed to 

reapply for benefits, the estimated rate of income ineligibility increases to the upper bound of 20 

percent. 

Among students, mean total certification error rates were higher than they were for districts. 

This resulted from the fact that the average student attended a school with a large enrollment, 

and estimated rates of ineligibility tended to be higher in larger districts. The estimated rate of 

administrative ineligibility was 44 percent. Again, however, many of these administratively 

108
ineligible students had incomes that qualified them for free or reduced-price meal benefits. The 

estimated lower bound of the income ineligibility rate among students was 12 percent. Under the 

assumption that nonresponders were in error except for those who reapplied and were approved 

for benefits, the estimated upper bound of income ineligibility was 33 percent. 

109
V. THE EFFECTS OF DIRECT CERTIFICATION ON INELIGIBILITY RATES

Lastly, we turn to estimating the effect of direct certification on rates of ineligibility among 

certified students in public school districts offering the NSLP. In Chapter IV, we described our 

methodology for estimating rates of ineligibility among certified students and then presented our 

estimates of those rates. We separately estimated rates of ineligibility among students certified 

by application and students directly certified. We then combined these estimates into an estimate 

of ineligibility among all certified students. Furthermore, we distinguished between two types of 

ineligibility—(1) administrative ineligibility, reflecting the extent to which certified households 

failed to comply with program rules, (2) income ineligibility, reflecting the extent to which

certified households have income above the income threshold for receiving benefits (and do not 

receive FS/TANF/FDPIR). 

Ultimately, we ended up with the following estimates of ineligibility among certified

students: 

• Ineligibility among students certified by application 

- Administrative ineligibility: Estimated by the benefit reduction/termination rate,

or the percentage of verified applications in which benefits were reduced or 

terminated 

- Income ineligibility, lower bound: Estimated by the percentage of verified 

applications in which households responded to the verification request and had 

benefits reduced or terminated (assumes that nonresponders remain eligible) 

- Income ineligibility upper bound: Estimated by the percentage of verified 

applications in which benefits were reduced or terminated and whose submitters

did not subsequently reapply and become approved for benefits 

• Ineligibility among directly certified students 

- Administrative ineligibility: Estimated by the FS/TANF turnover rate, or the 

percentage of students on FS/TANF in summer 2001 (eligible for direct 

certification) who were no longer on FS/TANF in December 2001 

110
- Income ineligibility: Estimated by the adjusted FS/TANF turnover rate, which 

considers December 2001 FS/TANF leavers ineligible only if their household

income exceeds 130 percent of the Federal poverty guideline 

• Ineligibility among all certified students 

- Administrative ineligibility: Estimated by the weighted average of the benefit 

reduction/termination rate and the FS/TANF turnover rate 

- Income ineligibility, lower bound: Estimated by the weighted average of the lower 

bound among students certified by application and the adjusted FS/TANF 

turnover rate among directly certified students 

- Income ineligibility, upper bound: Estimated by the weighted average of the 

estimated income ineligibility rate among students certified by application and the 

adjusted FS/TANF turnover rate among directly certified students 

In this chapter, the analysis focuses primarily on the effect of direct certification on the 

estimated rate of ineligibility among all certified students, particularly on the rate of income

ineligibility among this group. We find that direct certification reduces the rate of income

ineligibility among all certified students by a statistically significant amount, though its 

estimated effect on the rate of administrative ineligibility among all certified students is small 

and not statistically significant. 

We also used districts’ reported verification results to estimate the effect of direct 

certification on the rate of ineligibility among students certified by application. The primary 

mechanism through which direct certification could influence verification results is by changing

the composition of the verification sample. Because directly certified students are not subject to 

verification, the average characteristics of certified students subject to verification may differ

from the average characteristics of all certified students. If some groups are more prone to 

administrative ineligibility, the change in composition of the verification sample may affect the 

measured rate of ineligibility. We find that direct certification increases the measured rate of 

111
administrative ineligibility among students certified by application, as indicated by the benefit

reduction/termination rate. 

A. DISTRICT-LEVEL MODEL OF THE EFFECTS OF DIRECT CERTIFICATION 

ON INELIGIBILITY 

We estimated the effect of direct certification on ineligibility among certified students using 

the model represented by Equation (1) in Chapter III, page 37. We examined income and 

administrative ineligibility, as well as estimating the effect of direct certification on rates of 

ineligibility among both all certified students and students certified by application. 

Since ineligibility among certified students depends on the extent to which students who are 

ineligible for benefits become certified, the same set of district characteristics hypothesized to 

influence rates of certification among all students are hypothesized to influence certification

among ineligible students and are included as control variables in the district-level model of 

ineligibility. The sample used to estimate this model of the effect of direct certification on

ineligibility excludes Provision 2 and 3 schools and districts that selected focused verification 

samples. 

A district’s rate of ineligibility among certified students is a continuous variable that takes

on values between 0 and 1; we estimated this model using OLS regression techniques. In 

practice, however, the ineligibility rate among students certified by application took on a value of 

0 for a substantial number of districts.60 In other words, the distribution of the dependent 

variable is censored at 0. To account for this censoring of the dependent variable, we conducted 

 60Ten to 16 percent of districts have an estimated rate of ineligibility among all certified students

equal to 0 (with the exact amount depending on whether income or administrative ineligibility is being

measured). And about one-third of districts have estimated rates of ineligibility among students certified

by applications equal to 0. When districts are weighted by the number of certified students in them, the

proportions of observations in which the estimated rate of ineligibility is equal to 0 are much lower. 

112
sensitivity tests to determine whether using a tobit model rather than OLS to estimate the effect 

of direct certification would influence the estimated parameters of the model. 

B. ESTIMATED EFFECTS ON INELIGIBILITY AMONG ALL CERTIFIED 

STUDENTS 

We first examined the effects on the estimated upper bound rate of income ineligibility 

among this group. As described in Chapter IV, this estimate treats the following groups as 

income ineligible: 

• Students certified by application who responded to verification and had their benefits 

reduced or terminated 

• Students certified by application who did not respond to verification, had their

benefits terminated, and did not subsequently reapply (and become re-approved ) for

benefits 

• Directly certified students who stopped receiving FS/TANF by December of the

school year and who were in households with incomes above 130 percent of poverty 

All other certified students are defined as being income eligible according to this definition. This 

district-level model was estimated for districts in the 37 States that provided FS/TANF data that 

selected random verification samples. 

Estimates from this model indicate that direct certification has a statistically significant 

negative effect of 4.1 percentage points on the rate of income ineligibility among all certified 

students (Table V.1). The magnitude of this effect is substantial. If a given district not using 

direct certification has a rate of income ineligibility of 19.8 percent—the estimate mean among 

all districts according to estimates presented in Chapter IV—then the results of Table V.1 

suggest that the use of direct certification in this district would lead to a decrease of about 20 

percent (to 15.7 percent) in this district’s rate of income ineligibility. Given the modest size of 

the estimated impacts of direct certification on certification and participation, the size of this 

estimated effect on ineligibility is surprising. 

113
The remaining coefficient estimates in Table V.1 indicate how other district characteristics 

are related to the estimated rate of income ineligibility among all certified students. This 

ineligibility rate tends to be higher in larger districts than in smaller districts, and is also related 

to the racial/ethnic distribution of a district. The rate tends to be lower in districts with a high 

poverty rate. However, the characteristics of the verification process in a district are not strongly 

related to its rate of income ineligibility. On the other hand, the ineligibility rate is higher in 

districts in which single-child applications are used rather than multi-child applications (or other

application types). 

Overall, the explanatory variables in the model explain only 27 percent of the overall 

variation in income ineligibility across districts, as indicated by the R2

 value. The explanatory

power of this model is much lower than that of the certification and participation models, which

had R2

 values of 0.87 and 0.56, respectively. One reason for this lesser explanatory power is that

the estimated rates of ineligibility are based on part on the results of districts’ verification

process, and in smaller districts the verification sample is typically very small and thus subject to 

substantial random sampling variability. 

114
TABLE V.1 

COEFFICIENT ESTIMATES FROM DISTRICT-LEVEL MODEL OF THE IMPACT

OF DIRECT CERTIFICATION ON THE RATE OF INCOME INELIGIBILITY 

AMONG CERTIFIED STUDENTS 

Variable 

Income Ineligibility Among

All Certified-Students Model

Intercept 36.23***

(8.49) 

District Uses Direct Certification -4.05** 

(1.78) 

District Formerly Used Direct Certification 4.21 

(2.88) 

Size of District 

 <= 500 -10.61***

(2.85) 

 501 to 1,000 -3.43 

(2.57) 

 1,001 to 5,000 — 

 5,001 to 10,000 7.54*** 

(2.14) 

 10,001 to 25,000 5.50**

(2.70) 

 > 25,000 10.55***

(2.47) 

Proportion of Elementary School Students 2.28 

(5.18) 

Urbanicity

 Urban -1.27 

(2.18) 

 Suburban — 

 Rural 0.41 

(2.01) 

Racial/Ethnic Distribution 

 Percentage white — 

 Percentage black 21.65** 

(10.63) 

 Percentage black squared -24.41* 

(13.13) 

115
TABLE V.1 (continued) 

Variable 

Income Ineligibility Among

All Certified-Students Model

 Percentage Hispanic 30.95** 

(15.35) 

 Percentage Hispanic squared -32.03

(21.21) 

 Percentage other 1.35 

(19.76) 

 Percentage other squared -4.51 

(21.19) 

Proportion of Students who Are “Limited 

English Proficient” 

17.42 

(17.09) 

Poverty Rate 

 Poverty rate within district -48.31

(23.87) 

 Poverty rate within district squared 42.58 

(45.42) 

 Poverty rate within county -21.45

(49.00) 

 Poverty rate within county squared 73.10 

(143.02)

Month Verification Process Completed 

 October or earlier -1.08

(2.33) 

 November -1.63 

(1.83) 

 December — 

 January or later -4.90*

(2.84) 

Type of Verification Sample Selected 

 Random sample — 

 Focused sample — 

 Other -5.28 

(5.08) 

Size of Verification Sample Selected 

 < 1% of applications 11.07 

(3.79) 

 1 to 2% of applications 2.24 

(2.14) 

 2 to 4% of applications — 

 4 to 10% of applications -0.51 

(1.74) 

116
TABLE V.1 (continued) 

Variable 

Income Ineligibility Among

All Certified-Students Model

 > 10% of applications 4.72 

(4.37) 

Type of Application Used

 Single-child — 

 Multi-child -4.51***

(1.75) 

 Other -6.98** 

(2.78) 

District Uses Verification for Cause -0.77 

(1.46) 

District Uses Electronic Point-of-Sale System 1.45 

(1.96) 

Mean of Dependent Variable 19.8 

R-Squared 0.27 

Sample Size 713 

Source: 2001 Direct Certification Study SFA Survey. 

Note: Standard errors are in parentheses. These models were estimated using ordinary least squares (OLS)

regression techniques. Standard errors have been adjusted to account for the complex sample design using 

the SUDAAN statistical package. In addition to the variables listed above, the model contained binary

variables to represent the States in which districts were located (as described in Table III.1). Missing value

flags were also included in the model for the proportion of elementary school students, the proportion of

students who are limited English proficient, the size of the verification sample selected, the type of 

application used, and whether the district uses an electronic point-of-sale system. 

 *Significantly different from zero at the .10 level, two-tailed test. 

 **Significantly different from zero at the .05 level, two-tailed test. 

***Significantly different from zero at the .01 level, two-tailed test. 

117
Table V.2 reports on the results of the estimation of several alternative model specifications.

In specification 1b, direct certification is measured by the “adjusted direct certification” variable, 

in which all districts in the northeastern States except for New York City (including nine districts

that reported not using direct certification) are defined as using direct certification. The

estimated effect of direct certification in this specification is negative and significant, at –3.7 

percentage points. Specification 1c is identical to the basic district-level model except that is 

uses sample weights based on the number of certified students in each district. In other words, 

the estimation results from this specification depend to a greater extent on what happens in the 

largest school districts. The estimated effect of direct certification on the rate of income 

ineligibility among certified students in this model is negative—at –1.4 percentage points―but it 

is much smaller than the estimated effect in the basic model and is no longer statistically 

significant. 

Since this student-weighted estimate places more emphasis on what happens in the largest 

districts, this finding suggests that although direct certification leads to a substantial decrease in 

the rate of ineligibility in the average district, the effect on the overall rate of ineligibility among 

all certified students is much smaller. In fact, this smaller estimated effect on ineligibility among 

students is more in line with the relatively small estimated effect of direct certification on rates of 

certification and participation. The larger negative effect on ineligibility in the average district 

(based on the unweighted model) is likely being driven by what is happening in very small 

districts. Since these small districts also usually have small verification samples, the estimated 

rates of ineligibility in these districts are also subject to greater sampling variability. Thus, we

are inclined to put more faith in the smaller magnitude of the student-weighted estimate. 

118
TABLE V.2 

MODEL OF INCOME INELIGIBILITY AMONG ALL CERTIFIED STUDENTS, 

ALTERNATIVE SPECIFICATIONS 

Specification (R-squared) 

Dependent 

Variable 

(Mean Value)

Variables in Model 

Representing 

Direct Certification

Coefficient 

Estimate 

Standard

Error 

1a. Basic model (0.27) Income

Ineligibility, All 

Students 

(19.8) 

DC (binary) -4.05** 1.78 

1b. Basic model with adjusted direct 

certification variable (0.27) 

Income

Ineligibility, All 

Students 

(19.8) 

Adjusted DCa

(binary)

-3.66** 1.75 

1c. Basic model with student-level 

weights (0.61)

Income

Ineligibility, All 

Students 

(33.1) 

DC (binary) -1.38 1.67 

Number of years: 

1 to 2 -5.48* 3.00 

3 to 5 -4.26** 2.08 

6 to 10 -3.51* 2.04 

2. Direct certification effect allowed 

to differ by number of years it has 

been in place (0.27) 

Income

Ineligibility, All 

Students 

(19.8) 

More than 10 -4.02 2.82 

DC implementation 

type: 

Non-matching 1.09 4.85 

District matching, 

passive consent

-5.50*** 1.92 

District matching, 

active consent 

-4.72 3.93 

State matching, 

passive consent

-2.48 2.29 

State matching, 

active consent 

-1.04 5.41 

3. Direct certification effect allowed 

to differ by type of direct

certification implementation 

(0.27) 

Income

Ineligibility, All 

Students 

(19.8) 

Mixed -1.19 5.44 

119
TABLE V.2 (continued) 

Specification (R-squared) 

Dependent 

Variable 

(Mean Value)

Variables in Model 

Representing 

Direct Certification

Coefficient 

Estimate 

Standard

Error 

4. Direct certification effect allowed 

to differ by the percentage of free 

certified students who are directly

certified (with single DC 

percentage variable) (0.27) 

Income

Ineligibility, All 

Students 

(19.8) 

Percentage of free 

certified students 

who are directly

certified

-11.02** 4.41 

5. Model of the lower bound of 

income ineligibility among all 

certified students (0.19)

Lower Bound of

Income

Ineligibility, All 

Students 

(11.5) 

DC (binary) -4.03*** 1.40 

6. Model of administrative 

ineligibility among all certified

students (0.32)

Administrative 

Ineligibility, All 

Students 

(27.2) 

DC (binary) 0.37 1.96 

Source: 2001 Direct Certification Study SFA Survey. 

Note: These models were estimated using ordinary least squares (OLS) regression techniques. The control

variables included in the model were the same as those listed in Table IV.9. 

a

The adjusted direct certification variable is identical to the original direct certification variable except that it defines 

as direct certification districts all districts in the northeast region (except for New York City), including nine 

districts that had reported not using direct certification on the SFA survey and defined as non-direct certification

districts in the original direct certification variable. 

 *Significantly different from zero at the .10 level, two-tailed test. 

 **Significantly different from zero at the .05 level, two-tailed test. 

***Significantly different from zero at the .01 level, two-tailed test. 

120
Specifications 2 through 4 presented in Table V.2 allow for the effects of direct certification 

on the rate of income ineligibility to differ according to the length of time that direct certification

has been in place in the district (specification 2), the manner in which it was implemented 

(specification 3), and the proportion of certified free students who are directly certified in the 

district (specification 4). The estimated effect of direct certification on the rate of income

ineligibility among all students does not appear to vary greatly according to the number of years

it has been in place in a district. The effect ranges from –5.5 percentage points for direct 

certification districts that have used the policy for 1 to 2 years to –3.5 percentage points for those 

that have used direct certification for 6 to 10 years. The effect is statistically significant for three

of the four categories of years of experience. 

Specification 3 suggests that the effect of direct certification does vary according to how it is 

implemented. The estimated effect is strongest in districts using district-level matching, in 

which the policy is estimated to lead to a decrease in the rate of income ineligibility of 4.7 to 

5.5 percentage points (with the latter estimate being statistically significant). The estimated 

effects of other types of direct certification are all closer to 0 and are not statistically significant.

In addition, there is no evidence that the estimated effects of direct certification on income 

ineligibility differ according to whether districts use active or passive consent in implementing 

the policy. 

In Specification 4, districts’ use of direct certification is represented by a continuous 

variable that indicates the percentage of certified free students who are directly certified. This

variable is set to 0 for districts not using direct certification. The coefficient on this indicator of 

the prevalence of direct certification in a district is negative (–11.0) and statistically significant, 

and indicates that the rate of ineligibility falls as a larger percentage of certified free students are 

directly certified. Relative to non-direct certification districts, for example, the estimate suggests 

121
that the rate of income ineligibility among all certified students would be 1.1 percentage points

lower in a district with 10 percent of certified free students directly certified, 2.8 percentage 

points lower in a district with 25 percent directly certified, and 5.5 percentage points lower in a 

district with half of all certified free students directly certified.

The remaining specifications reported in Table V.2 show estimates of the effect of direct 

certification on other measures of ineligibility among all certified students. Specification 5 

shows the estimated effect on the lower bound of the income ineligibility rate. As described in

Chapter IV, the lower bound differs from the estimated rate of income ineligibility used in the 

basic model in that it classifies all children whose families did not respond to the verification

request as income eligible for benefits. The estimated effect of direct certification on this lower

bound is similar in magnitude to the estimated effect on the income ineligibility rate used in the 

basic model. The estimate is –4.0 percentage points, and is statistically significant. 

The other measure of ineligibility among all certified students described in Chapter IV is 

administrative ineligibility. Specification 6 of Table V.2 shows the estimated effect of direct

certification on the rate of administrative ineligibility among all certified students. In contrast to 

the estimated effect of direct certification on income ineligibility, the estimated effect on 

administrative ineligibility is close to 0 and not statistically significant. In other words, direct

certification districts and non-direct certification districts have rates of administrative

ineligibility that are about the same, holding other factors equal. 

We can better understand why direct certification is estimated to have a negative effect on 

income ineligibility but no effect on administrative ineligibility by examining the rates of income 

and administrative ineligibility among directly certified students. As presented in Chapter IV,

the estimated rate of administrative ineligibility among directly certified students is 28 percent 

while the estimated rate of income ineligibility among this group is only 9 percent. Among 

122
students certified by application, the gap between the administrative ineligibility and income 

eligibility rates is much smaller (27 percent versus 21 percent). In other words, directly certified

students are similar to students certified by application in terms of their administrative 

ineligibility but are much less likely to be income ineligible. This same relationship is evident in

the estimated effect of districts’ direct certification status on their rates of administrative 

ineligibility and income ineligibility among all students. 

C. ESTIMATED EFFECTS ON INELIGIBILITY AMONG STUDENTS CERTIFIED 

BY APPLICATION 

In this section, we focus more narrowly on the manner in which direct certification may 

influence ineligibility among students who are subject to the NSLP income verification process. 

While primary interest focuses on the effects of direct certification on ineligibility rates among 

all certified students, NSLP rules specify that students certified by application are subject to 

verification, whereas directly certified students are not subject to verification. This section uses 

data on the reported outcomes of verification to examine whether direct certification affects 

those outcomes.61

Direct certification clearly has the potential to influence verification results without

necessarily influencing the eligibility of students being verified, since direct certification changes 

the pool of students subject to verification. Thus, the interpretation of verification results may be

very different depending on whether a district uses direct certification and the proportion of

certified students who are directly certified. However, no previous analysis has been conducted 

to understand how direct certification is related to verification results. The analysis presented in 

 61As in the previous section, the analysis excludes districts that selected focused verification samples,

so that the verification results being examined would be representative of all students certified by

applications in the sampled districts. 

123
this subsection is an attempt to provide such evidence, and thereby provide a firmer basis for 

interpreting the information obtained from verification efforts. 

Table V.3 shows the estimated effects of direct certification on administrative and income

ineligibility among students certified by application. The estimated effect of direct certification 

on the benefit reduction/termination rate, a measure of administrative ineligibility among 

students certified by application, is positive and statistically significant. All else equal, the 

benefit reduction/termination rate in direct certification districts is 3.9 percentage points higher 

than the rate in non-direct certification districts. The most likely explanation for this positive 

relationship is that direct certification removes from the verification pool those students who are 

least likely to be found in error and have their benefits reduced or terminated. As a result, the 

students remaining in the verification pool are more likely to be found in error and the resulting 

benefit reduction/termination rate is higher. 

Students whose benefits are reduced or terminated by the verification process either 

provided documentation that they were ineligible for the benefits they were receiving or they

failed to respond to the verification request. In Chapter IV, we presented estimates indicating 

that among students whose meal price status was verified, 11 percent provided documentation 

showing they were ineligible. This was our estimate of the lower bound of the income 

ineligibility rate among certified students. We also presented an estimate of the income

ineligibility rate among students certified by application that started from this lower bound but 

then also considered students to be ineligible if their benefits were terminated due to nonresponse

and they did not reapply for benefits. The mean estimate of this measure of ineligibility was

21 percent. 

124
TABLE V.3

 MODEL OF INCOME INELIGIBILITY STUDENTS CERTIFIED BY APPLICATION 

Specification 

Dependent Variable 

(Mean Value) 

Variables in 

Model 

Representing

Direct 

Certification

Coefficient 

Estimate 

Standard 

Error 

1. Administrative 

Ineligibility Model 

Benefit reduction/

termination rate 

(26.6) 

DC (binary) 3.85** 1.96 

2. Lower Bound, Income

Ineligibility Model 

Percentage who 

responded to 

verification request 

and had benefits 

reduced or terminated 

(11.3) 

DC (binary) -2.03 1.40 

3. Income Ineligibility 

Model 

Percentage who had 

benefits reduced or 

terminated and did 

not reapply for 

benefits 

(33.1) 

DC (binary) 1.42 1.82 

Source: 2001 Direct Certification Study SFA Survey. 

Note: These models were estimated using ordinary least squares (OLS) regression techniques. The 

control variables included in the model are the same as those listed in Table IV.9. 

 *Significantly different from zero at the .10 level, two-tailed test. 

 **Significantly different from zero at the .05 level, two-tailed test. 

***Significantly different from zero at the .01 level, two-tailed test. 

125
Table V.3 shows that neither the estimated effect of direct certification on the rate of 

income ineligibility among students certified by application nor the effect on the lower bound of 

this rate is statistically significant. In particular, the estimated effect of direct certification on the

lower bound is –2.0 percentage points and the estimated effect on the rate of income ineligibility 

is 1.4 percentage points. Thus, while direct certification is estimated to be positively and

significantly related to the rate of administrative ineligibility among students certified by 

application, it is not significantly related to income ineligibility among this group.62

We estimated two main sets of alternative specifications of these models of ineligibility

among certified students. First, we estimated these models using tobit regression techniques to 

account for the fact that a relatively large proportion of districts reported rates of ineligibility 

among students certified by application at the minimum value of 0 percent.63 We found that the 

estimated effects of direct certification based on these tobit models are close to the estimates 

based on the OLS models presented in Table IV.11. In particular, the tobit estimates of the effect 

of direct certification on ineligibility among students certified by application are 3.8 percentage 

points (and statistically significant) for administrative ineligibility, –3.5 percentage points (and 

not statistically significant) for the lower bound of income ineligibility, and 1.1 percentage points 

(and not statistically significant) for income ineligibility. 

Second, we estimated models of ineligibility among certified students using student-level 

weights. The estimated effect of direct certification in these models depends more heavily on 

what happens in the largest school districts. These estimates differ from the estimates of the 

 62This pattern of estimated effects implies that direct certification has no significant effect on the

percentage of students whose benefits are reduced or terminated because of documentation they provided 

in response to a verification request, but has a positive effect on the percentage whose benefits are 

terminated because of nonresponse. 

63Among the 856 districts included in the model estimation, about one-third had a value of 0 for at

least one of the ineligibility rates among certified students. 

126
model that did not give more weight to large districts. Again, these student-weighted estimates 

give a better indication of how direct certification is affecting the average student as opposed to 

how it is affecting students in the average district. The estimated effects on the rates of both 

administrative ineligibility and income ineligibility are 6.6 percentage points, and statistically 

significant. The estimated effect on the lower bound of income ineligibility is 0.5 percentage 

points, and is not statistically significant. 

127
VI. SUMMARY 

Direct certification is an administrative practice with a wide range of potential beneficial 

effects—reducing administrative burden, promoting access to the program among eligible 

students, and reducing the proportion of students who are “overcertified,” or ineligible for the 

level of benefits they are receiving. As a result, a large number of districts have begun using 

direct certification in the last 10 to 15 years. In this study, we found that direct certification has

achieved some of its potential benefits. 

One key finding from the study is that direct certification leads to an increase in the 

proportion of students certified for free meals. Thus, the policy has achieved its aim of 

promoting access to the program. This finding is robust to alternative models/specifications.

Two separate models based on independent data sources showed positive and statistically 

significant effects of direct certification on the free certification rate, and these estimates were of 

roughly the same magnitude. The estimates were also consistent with the estimated effect of 

direct certification on the free certification rate from the previous study of direct certification. 

The size of the estimated effect of direct certification on the free certification rate—1.3 to 

1.4 percentage points—is not large in percentage terms. Nor would we expect this effect to be 

large, since direct certification influences only children in households receiving FS/TANF and 

many of these children would be certified for free meals even in the absence of direct 

certification. However, the estimated effect is substantial in absolute terms, implying that direct 

certification induces about 400,000 students to become certified for free meals who would not 

have done so otherwise. 

Of course, increasing the number of certified students is not meaningful unless these 

students actually eat school meals. Evidence from the study suggests that these students do eat 

128
school meals, as estimates indicate that direct certification leads to an increase in the NSLP 

participation rate, although the evidence here is not quite as clearcut as it was with respect to

certification. 

The other key outcome examined in the study was program integrity, with particular 

attention given to two questions. First, what proportion of districts’ certified students are 

ineligible for the benefits they are receiving as of December of the school year? Second, does 

the increase in certification induced by the implementation of direct certification lead to a 

reduction in the rate of ineligibility among all certified students?

We found the estimation of income ineligibility to be a complex and challenging task, 

largely because many parents do not respond to requests for verification of their children's

free/reduced-price meal status. The income eligibility status of their children is then unknown. 

As a result, we generated lower and upper bound estimates of the proportion of certified students 

that are income ineligible for the level of benefits they are receiving. In the average district, we 

estimated the lower bound at 12 percent, when all nonresponders are considered income eligible. 

The upper bound estimate is 20 percent income ineligible, when nonresponders who did not later 

reapply and receive benefits are considered income ineligible. Among students overall, we

estimated income ineligibility at a lower bound of 12 percent and an upper bound of 33 percent, 

treating nonresponders as we did in the average district estimates. 

Turning to the effects of direct certification on ineligibility, we estimated regression models 

suggesting that direct certification leads to a decrease in income ineligibility among certified 

students. However, this estimated effect was substantially influenced by the particulars of the 

specification being estimated. In addition, no previous studies have provided any evidence as to 

this relationship. So while our estimates suggest that direct certification leads to an improvement

129
in program integrity, more research is needed to solidify this finding and to determine the precise 

magnitude of the effect of direct certification on rates of ineligibility among certified students. 

Direct certification has proved to be a popular policy among districts, perhaps because of its 

effects on program access and integrity described above, or perhaps because it has reduced

administrative burden, an outcome not directly examined in this study. For example, Jackson, et 

al. (1999) found that food service directors in districts using the policy reported believing that

direct certification had led to improved program access and lower administrative costs in their

district. Overall, more than 60 percent of public school districts serving, two-thirds of students, 

use direct certification. And although a few districts have dropped direct certification after 

starting to use it, most districts continue to use the policy once they initially implement it. 

Despite its popularity, a large number of districts do not use direct certification, and the 

proportion of districts using the policy has not grown since 1996. Some of the non-direct

certification districts say that they are satisfied with their current procedures for certifying 

students and do not need direct certification. Others cite resource constraints or problems in the 

implementation of direct certification, such as problems with various aspects of the process of 

matching enrolled students with children receiving FS/TANF. Even in districts that use direct 

certification, many students who could be directly certified are somehow missed by the system. 

Again, issues with the matching process may lie at the root of this problem. 

Given the positive effects of direct certification and its potential for growth, the results of

this study suggest that policymakers consider policies aimed at expanding the use of direct 

certification. As suggested above, direct certification could be spread to additional students 

within districts already using the policy and/or to additional districts not currently using it. Since 

this study has focused on estimating the effects of direct certification among districts currently 

using the policy, the evidence of positive effects most strongly supports expanding direct 

130
certification to additional students within districts already using the policy. This evidence

supports expanding direct certification to additional districts only if one believes that the effects 

of direct certification would be the same in these additional districts as it already is in the 

districts currently using the policy. 

To help districts currently using direct certification expand it to additional students,

policymakers might look for ways to address some of the problems districts may have had in 

implementing the policy. Since many districts cited problems with the matching process, 

determining “best practices” with respect to matching may help districts to identify and directly

certify more students. Similarly, encouraging greater communication between State-level 

FS/TANF agencies and both the State-level agencies that administer the NSLP and school 

districts themselves may improve districts’ ability to efficiently implement direct certification.

Finally, since some school districts cite resource constraints as their reason for not using direct 

certification, it is also possible that resource constraints limit districts’ ability to directly certify 

as many students as possible (and that providing additional resources would help address this

problem). 

While each of the ideas described above might help to promote direct certification to

additional students within districts using the policy, this is an issue for which additional research

may be particularly useful with respect to guiding policy. In particular, research is needed to 

more precisely answer the question of how many students who could be directly certified are 

either certified by application or not certified at all. And to the extent that this “gap” in direct 

certification coverage varies by district, what are the characteristics of those districts for which 

the largest proportion of students who could be directly certified are, in fact, directly certified?

If policymakers wish to expand direct certification to additional districts, one policy option 

would be to require districts to use the policy. A disadvantage of this approach is that it entails 

131
forcing districts to do something they may not wish to do. Because implementation of the policy

in these districts would be mandatory, its effects might be different than the estimated effects 

(from this study) among districts that implemented direct certification voluntarily. 

An alternative approach involves using policies that would make direct certification more

attractive for districts not currently using it. The policy options described above—easing the 

matching process, improving communication between agencies and districts, and providing

additional resources to districts implementing direct certification—may convince additional 

districts to begin using direct certification. 

132
REFERENCES 

Akin, J.S., D.K. Guilkey, P.S. Haines, and B.M. Popkin. “Impact of the School Lunch Program 

on Nutrient Intakes of School Children.” School Food Service Research Review, vol. 7, no. 

1, 1983, pp. 13-18. 

Chromy, J.R. “Sequential Sample Selection Methods.” In Proceedings of the Survey Research

Methods Section of the American Statistical Association. 1979, pp.401-406. 

Gleason, Philip M. “Participation in the National School Lunch Program and the School 

Breakfast Program.” American Journal of Clinical Nutrition, vol. 61, no. 1(s), January 

1995, pp. 213S-220S. 

Gleason, Philip M. “Student Participation in the School Nutrition Programs: An Econometric

and Simulation Model.” Report to the U.S. Department of Agriculture, Food and Nutrition 

Service. Princeton, NJ: Mathematica Policy Research, August 1996. 

Gruber, Kerry G., Susan D. Wiley, Stephen P. Broughman, Gregory A. Strizek, and Marisa 

Burian-Fitzgerald. “Schools and Staffing Survey, 1999-2000: Overview of the Data for 

Public, Private, Public Charter, and Bureau of Indian Affairs Elementary and Secondary 

Schools.” National Center for Education Statistics, U.S. Department of Education, Office of 

Educational Research and Improvement, NCES 2002-313, May 2002. 

Jackson, Kenneth, Philip Gleason, John Hall, and Rhonda Strauss. “Study of Direct Certification 

in the National School Lunch Program.” Houston, TX: Decision Information Resources, 

Inc., November 1999. 

Kish, L. Survey Sampling. New York: John Wiley and Sons, 1965. 

Kuhns, Carole, Anne Gordon, Roberto Agodini, and Renee Loeffler. “The Virginia Closed Case 

Study: Experiences of Virginia Famliies One Year After Leaving Temporary Assistance for 

Needy Families.” Report prepared for the Virginia Department of Social Services. Falls

Church, VA: Virginia Polytechnic Institute and State University, Center for Public 

Administration and Policy, November 1999. 

Maurer, Kenneth M. “The National Evaluation of School Nutrition Programs: Factors Affecting 

Student Participation.” American Journal of Clinical Nutrition, vol. 40, 1984, pp. 425-447. 

Office of the Inspector General. “Nationwide Statistical Sample of Program Participation for

May 1980 and Verification of Free and Reduced Price Application Information.” 

Washington, DC: U.S. Department of Agriculture, February 1981. 

Rangarajan, Anu and Robert G. Wood. “Work First New Jersey Evaluation: How WFNJ Clients 

Are Faring Under Welfare Reform: An Early Look.” A Report Series of the Work First New 

Jersey Evaluation. Princeton, NJ: Mathematica Policy Research, October 1999. 

133
Rangarajan, Anu and Philip M. Gleason. “Food Stamp Leavers: An Untold Story?” Policy and 

Practice, vol. 59, no. 3, June 2001, pp. 34-38.

St. Pierre, Robert G., and Michael J. Puma. “Controlling Federal Expenditures in the National 

School Lunch Program: The Relationship Between Change in Household Eligibility and 

Federal Policy.” Journal of Policy Analysis and Management, vol. 11, no. 1, 1992, pp. 42-

57. 

Tordella, S. “CPS Analysis of National School Lunch Program Participation/Income (Updated 

to 1999).” Memorandum to the U.S. Department of Agriculture, Food and Nutrition 

Service, 2001. 

U.S. Department of Agriculture, Food and Nutrition Service. “Current Population Survey 

Analysis of NSLP Participation and Income.” Alexandria, VA: USDA, Food and Nutrition 

Service, Office of Analysis, Nutrition, and Evaluation, October 1999. 

U.S. Department of Agriculture, Food and Nutrition Service. “Study of Income Verification in 

the National School Lunch Program: National Survey Final Report.” Washington, DC: 

1990. 

134
APPENDIX A 

DESCRIPTION OF DATA COLLECTION 

A.1
This appendix describes the primary data collection activities for the Direct Certification

Study. Two primary sources of data were used in the study: (1) the School Food Authority 

(SFA) survey and (2) State administrative records data. Section A outlines the procedures used 

to conduct the SFA survey and summarizes the results of efforts to interview a nationally 

representative sample of school districts. Section B describes the collection of State

administrative records data on the FS/TANF/FDPIR receipt of families living in the areas

covered by the districts in the sample. 

A. SFA SURVEY 

1. Questionnaire Development and Pretesting 

The development of the SFA questionnaire began in November 2000. The questionnaire 

drew heavily on the 1996 Study of Direct Certification so that it would be as similar to this 

previous study as possible. The questionnaire also drew on MPR’s previous experience with

surveys of school districts, such as the first School Nutrition Dietary Assessment (SNDA-1) 

Study. To address the objectives of the Direct Certification Study, the SFA questionnaire was 

organized into eight sections: 

• Section 1: This section introduced the respondent to the interviewer and provided the 

respondent with an overview of the study’s purpose. 

• Section 2: This section was designed to ensure that the interviewer talked to the 

appropriate person: the one most knowledgeable about the NSLP and certification for

free meals in the program. 

• Section 3: This section provided both contextual information about how districts 

implement aspects of the NSLP and information on a district’s actual levels of

enrollment, certification, and participation. 

• Section 4: This section collected information on whether the district uses direct

certification and, if so, how it is implemented. 

• Section 5: This section collected information on the application and verification 

process for free and reduced-price certification. 

A.2
• Section 6: The questions in this section collected information on the timing and other 

aspects of direct certification activities. 

• Section 7: This section collected additional descriptive information on direct 

certification, such as problems that districts had in implementing this process, 

districts’ views on the costs and benefits of using direct certification, and changes in 

the process since the previous study was conducted in 1996. For districts that do not 

use direct certification, this section collected information on their reasons for not 

using it. 

• Section 8: This section collected descriptive information about the districts in the 

sample and the person who was the primary respondent in each. 

After the questionnaire was developed, it was revised on the basis of comments from the 

USDA’s Economic Research Service (ERS). It was then pretested to detect any remaining 

problems, such as poorly worded questions, terms that were not defined, missing or inadequate 

response categories, difficult transitions between questionnaire topics, or erroneous or unclear 

interviewer instructions. The pretest was also used to assess the flow of the entire instrument 

and the burden on respondents. It was conducted with nine respondents in February and March 

2001. The questionnaire was revised during spring 2001, with revisions based on the results of

the pretest and additional comments from ERS. 

This revised version of the SFA questionnaire was submitted to the Educational Information 

Advisory Committee (EIAC) for consideration at its May 2001 meeting. After MPR revised the 

questionnaire based on EIAC comments, EIAC granted its approval for data collection activities

in June 2001. The questionnaire was next submitted (along with supporting materials) to the 

Office of Management and Budget (OMB). OMB approval was granted in November 2001. 

Computer-assisted telephone interviewing (CATI) programming of the survey instrument

began in September 2001, during the OMB clearance process. This kept the time between 

receipt of OMB clearance and the start of interviewing as short as possible. All changes required 

A.3
by OMB were incorporated into the final version of the instrument. Programmers, survey staff

members, and interviewers extensively tested the program for accuracy. 

2. Sample Selection and Weights 

The target population for this study consisted of all SFAs in public school districts 

participating in the NSLP in the 50 States and the District of Columbia. The sampling frame 

consisted of 14,571 school districts included on the National Center of Education Statistics 

(NCES) Common Core of Data (CCD).64 The frame included information on the number of

students in each school district, address information, and some demographic characteristics of

the districts’ students. 

a. Stratification

Stratification is used to form relatively homogeneous groups from which separate samples

are selected. These separate samples are then combined to form the overall sample. 

Stratification was used in the SFA survey to facilitate oversampling of subgroups of SFAs and 

increase the precision of some estimates.65 The sample employed a stratified design that was 

modeled on the sample design of the 1996 study and consisted of nine strata. One stratum (9) 

was reserved for the largest districts, which were selected with certainty (that is, all of these 

districts were selected). The other strata were formed based on the number of students in the 

 64The CCD is an electronic database of school districts available from the NCES. No frame of SFAs 

was available. In a large majority of cases, however, there is one SFA per district. The CCD-based 

sample frame of 14,571 includes a small proportion of districts that do not participate in the NSLP; thus,

the true sample frame is slightly smaller than this. 

65Where strata are formed that are homogenous with respect to the variance of a statistic, the

sampling error of that statistic can be reduced (see Kish 1965). In this study, for example, one key 

estimate is of the prevalence of direct certification. Since the variance of a proportion is higher near 0.50 

and lower in the tails, stratifying on the expected prevalence of direct certification and oversampling from 

the high variance strata may increase the precision of this estimate. 

A.4
school district and on the estimated percentage (in 1998) of SFAs using direct certification in the 

State in which the district is located. The nine strata are summarized in Table A.1. 

TABLE A.1 

DESCRIPTION OF SAMPLE STRATA 

Stratuma 

Estimated Prevalence of Direct Certification in 

State (Percent) Number of students 

1 0 to1 < 28,000

2 99 to 100 < 28,000 

3 More than 1, up to 8.5 < 28,000 

4 More than 91.5, less than 99 < 28,000 

5 More than 8.5, up to 22 < 28,000 

6 More than 78, up to 91.5 < 28,000 

7 More than 22, up to 78 < 5,000 

8 More than 22, up to 78 5,000-27,999 

9 Any >= 28,000

a

The stratum numbers are those used in sampling. Stratum 9 consists of districts to be selected with certainty. The other

(noncertainty) strata are in ascending order of expected sampling error for an estimate of the prevalence of direct

certification. 

b. Sample Allocation Plan 

Districts were selected at different rates from the different strata. In particular, districts were

sampled at the highest rates from strata containing the largest districts and also from strata 

contained districts in States in which the estimated prevalence of direct certification was closest 

to 50 percent. As noted above, all districts from stratum 9 were selected into the sample. The 

sampling rates were lowest in strata 1 and 2 (because these contained districts in States with an

estimated prevalence of direct certification very close to either 0 or 100 percent) and in stratum 7 

(because it contained the smallest districts). Aside from stratum 9, the sampling rate was highest

in stratum 8, because it contained no small districts and only districts in States with estimated 

prevalence of direct certification relatively close to 50 percent. In addition (as described in 

A.5
greater detail below), from within each stratum, districts were selected into the sample with 

probability proportional to size. 

To evaluate the implications of alternative sample designs and choose the best of these 

alternatives, we conducted simulations involving sample allocation options using five different 

measures of size (MOS) for purposes of the probability proportional to size sampling. Using the

square root of the number of students as the MOS gave us the best results, and was the process 

used in selecting this sample.66 The final sample design was also similar to the design of the 

1996 study in that it defined sample strata on the basis of district size and estimated prevalence 

of direct certification in the State. 

While we estimated that only 1,547 districts would be needed to achieve the desired number 

of SFA interviews, we selected an overall sample size of 1,847 to allow for additional sample if 

survey response was lower than expected. Stratum 9 contained 204 districts, and all of these

were selected into the sample. From the remaining eight (noncertainty) strata, districts were

selected with probability proportional to the measure of size described above (the square root of 

the number of enrolled students in the district). We used the formula given below to determine 

the specific number of districts to be selected in each of the eight noncertainty strata.

nh = nnoncert * (Wh) where h = 1, 2,..., 8 

nh = sample size in stratum h 

Wh = stratum h’s MOS/Sum (of the MOS of all eight noncertainty strata) 

nnoncert = the total sample size across all noncertainty strata 

= 1,847 - 204 = 1,643

 66Details of the alternative sampling allocations evaluated and the simulation results are available

from the authors upon request. 

A.6
We used the Chromy (1979) procedure to generate the sample of 1,847 school districts using 

a serpentine sort. The Chromy procedure is a systematic selection procedure that sorts the cases

in each sampling stratum in a serpentine fashion based on specified characteristics. This

procedure imposes implicit stratification beyond the explicit strata to ensure that the sample is 

balanced by specified characteristics. For the SFA survey sample, we used State and poverty

rate as the implicit strata so that within each of the explicit strata districts were sorted first by

State and then by poverty rate. 

After selecting the overall sample of 1,847 school districts, we then used the Chromy

procedure three more times to create four subsamples for sequential release. The first release

consisted of 1,547 cases (all the certainty cases included), and the remaining three releases 

included 100 cases each. The additional releases were not needed, as the first release resulted in

the desired number of 1,200 completes. 

Table A.2 provides the total number of districts in each stratum and the number sampled 

from each stratum.

TABLE A.2 

SAMPLING RATES, BY STRATUM

Stratum

Number of Districts in 

Sample Frame 

Number of Districts Selected 

for Overall Sample 

Number of Districts 

Released into Final Sample

1 656 56 46

2 4,084 393 321

3 2,329 261 213

4 1,314 171 140

5 1,700 225 184

6 676 110 90

7 3,216 314 257

8 392 113 92

9 204 204 204

Total Sample 14,571 1,847 1,547 

A.7
To restore the sample to the proportional distribution of the population that it represents, 

sample weights were constructed. Each SFA’s weight is the product of two factors: 

•

•

 

The inverse of the probability of selection of the district associated with the SFA 

The inverse of the response rate calculated for the SFA’s sampling stratum 

Combined, these factors correct for both under- and oversampling and for differences in 

response rates among strata.67 Use of the sample weights will reduce, if not eliminate, the bias

that would arise from using unweighted survey data.68 The use of sample weights will also

affect the estimated sampling error for estimates based on the sample. We used the SUDAAN 

statistical package to correctly estimate standard errors taking into account sample weights and 

the complex sample design. 

In addition to these sample weights that make the SFA sample of districts representative of

the population of public school districts (offering NSLP lunches) nationally, we constructed 

weights that allowed us to estimate the prevalence of particular district characteristics among all 

public school students nationally (or among particular groups of public school students, such as 

certified students). These weights were constructed by starting from the initial district-level 

weights described above and within each district, multiplying the weight by the relevant count of 

the number of students in the district. 

67Other weighting steps often used in surveys—poststratification and trimming—were not employed 

for this sample. Poststratification was not used because there was no reliable external data source to use 

as a standard for adjusting the weights. Trimming was not called for because, in our judgment, there were

no weights with extreme values. 

68If there are unobserved factors that contribute to differences in SFAs’ probability of survey

response, use of the stratum-specific response rates in constructing weights might not fully eliminate 

nonresponse bias. 

A.8
3. Conducting the Survey 

To secure the cooperation of SFAs selected for the final sample and to facilitate the 

interviewing process, MPR took several steps before calling the school districts to conduct the 

survey: 

• Chief State School Officer (CSSO) Letter. In December 2001, letters were sent to

each State’s CSSO—the State superintendent of schools—to explain the objectives of

the study and data collection procedures. 

• State EIAC Representative Letter. At the same time that the CSSO letters were

mailed, a copy of the letter was sent to the EIAC representative from the same State. 

• Child Nutrition Director Letter and Follow-Up Call. In December 2001, letters 

were sent to the State child nutrition directors explaining the study’s objectives and 

data collection process. These letters asked each director to name a State-level study 

liaison and said that someone from MPR would call the director to obtain the name of 

the State liaison. As an attachment to this letter from MPR, a letter from USDA 

signed by a representative of ERS was included to assure the child nutrition directors 

of the legitimacy of the study and encourage their cooperation. The child nutrition 

directors were each contacted in early January 2002, and they all agreed to support 

the study and provide a State liaison for the study. 

• Local Superintendent Letter. In mid-January 2002, letters were mailed to the local 

school district superintendents associated with each SFA in the study sample. These

letters explained the study. They also provided the name of the State liaison for the

study so the superintendents could contact the liaison to verify the study’s legitimacy. 

• SFA Advance Letter and Fact Sheet. Shortly after the superintendent letters were 

sent, letters were mailed to the directors of each SFA included in the sample. These 

letters described the study and alerted the SFA directors that they would be contacted 

during the next few weeks to participate in the study. Accompanying the SFA letters 

were fact sheets that the SFA directors (or their staff) were asked to complete before 

the interview. This fact sheet was designed to guide SFA directors in gathering 

information from their files before the interview.

Immediately after these supporting materials had been sent, and just before the interviews 

were to begin, interviewer training was held at MPR’s central survey operations center near 

Princeton, New Jersey (January 22 to 24, 2002). The training included an overview of the 

project, a question-by-question review of the survey instrument, a discussion of frequently asked

A.9
questions and their answers, and practice interviewing and role playing. In addition, the 

interviewers provided input on the questionnaire, which was then revised as appropriate. 

Telephone interviewing for the SFA survey began on January 25, 2002, and continued until

March 22, 2002. Calls were placed to SFA directors from 8:00 A.M. to 4:00 P.M., local time, 

with appointments and callbacks scheduled at the convenience of the food service directors 

(usually early in the morning or after the lunch sessions). Specially trained monitors, who 

attended the interviewer training session, monitored all interviewers. In addition, the survey 

director and two assistants continually monitored interviews. When a respondent from one of the 

15 largest school districts was interviewed, a member of the project supervisory staff monitored 

the interview for quality assurance.

One of the purposes of the extensive monitoring of interviews was to detect and correct 

problems with the survey instrument or with the way in which questions were asked. One

problem discovered during the early stages of interviewing was that the percentage of districts 

reporting that they have schools that operate under USDA Provision 2 or 3 was much higher than 

expected. This finding led us to carefully examine the status of the districts that reported having 

Provision 2 or 3 schools, by checking their responses to survey questions and calling back the 

districts to verify their answers. This re-examination led us to conclude that respondents were 

confused by the wording of the question related to Provision 2 or 3 funding, which led some

districts to incorrectly report that they had Provision 2 or 3 schools. For districts that corrected 

their responses to the Provision 2 or 3 questions on the survey when we called them back, we 

replaced their initial responses with these corrected responses on the data file. We also asked 

these districts questions that had been skipped previously because the district had reported

receiving Provision 2 or 3 funding. Finally, we clarified the wording of the Provision 2 or 3 

questions on the SFA survey to be administered to subsequently interviewed districts. 

A.10
Several new questions were added to the SFA questionnaire midway through the JanuaryMarch interviewing period. At the request of ERS, questions about electronic point of sale, or 

point of service (POS), systems were added to the end of the survey. Sample members were first 

asked if any schools used a POS system and, if so, how many schools did so and in what school 

year they started using it. Respondents who had completed the interview before these questions 

were added (about 500) were not called back to answer the new questions. 

We used two tools to monitor each interviewer’s productivity: (1) CATI reports and

(2) interviewer productivity reports. CATI reports provided the results of each telephone 

attempt, on both a daily and cumulative basis, and showed, by interviewer, the numbers of calls 

made, completes, refusals, and minutes-per-complete. Interviewer productivity reports provided 

each interviewer’s daily and cumulative hours worked and the number of interviews completed. 

Interviewers with low productivity or high refusal rates were closely monitored and retrained. 

Interviewers whose performance was unsatisfactory, even after remediation efforts, were

removed from the project. 

As interviewing neared completion, MPR analyzed the response rate by sample strata. 

Special reports were created that showed the disposition of all 1,547 sample districts, by stratum.

We used this information to better target the survey resources to achieve similar response rates in 

each stratum. We focused particular attention on ensuring an adequate response rate in stratum 

9, so that the largest districts nationally would not be underrepresented in the analysis sample. 

4. Survey Response Rates 

Overall, interviews were attempted with 1,547 school districts. Of this full sample, 35 

districts (or about 2.3 percent) reported that they did not participate in the NSLP, which made

them ineligible for the study. Excluding these districts, the full sample size was 1,512. The 

A.11
response rate on the SFA survey was 81 percent, as full interviews were completed with 1,223 of 

the 1,512 districts in the sample. 

There were several reasons for nonresponse. Just under eight percent of districts (or 114 of

1,512) refused to participate in the study. The reasons that these districts gave for their refusal 

included: 

• They were too busy/had no time to get the information/were already overwhelmed 

with audits and paperwork. 

• They were too new on the job to be able to answer the questions. 

• They believed the USDA could obtain the requested information from other sources. 

• They were not interested/did not want to participate because it was not mandatory. 

Other districts did not participate because they could not be reached and did not return calls. 

Some districts were reached initially, could not complete the interview at that time, then could 

not be reached later. These two categories of cases accounted for 11 percent of the overall 

sample. 

Response rates were fairly consistent across the nine sample strata. Response rates in the 

strata ranged from 66 percent to 86 percent, including a response rate of 86 percent in stratum 9 

(which contained the largest districts).69 Differences in response rates across the strata did not 

appear to be correlated with the size of districts in the strata. Response rates were also fairly 

consistent across States. Among the 32 States from which at least l5 districts were sampled, the 

minimum response rate was 62 percent, in New Jersey. The maximum response rate was 100 

percent, in Oregon. Twenty-seven of the 32 States had response rates between 70 and 95 

percent. 

 69In our analysis, we used the sample weights described above to adjust for these response rate 

differences by strata. 

A.12
For each completed interview, an average of seven calls were made to districts to attempt

interviews. The interview itself took an average of 29 minutes. In addition, respondents who 

completed interviews reported the amount of time they spent completing the fact sheet before the 

interview; the mean amount of time spent completing the fact sheet was two hours, although this 

mean value includes few districts that spent an extremely long time on the fact sheet. The 

median amount of time spent on the fact sheet was just one hour. 

B. STATE ADMINISTRATIVE RECORDS DATA 

The second major data source used in the Direct Certification Study is administrative

records data from State food stamp or welfare agencies. For each of the districts in our sample

that reported using direct certification, we attempted to collect State administrative data on 

receipt of FS/TANF/FDPIR benefits among children living in the areas covered by these 

districts. In particular, we asked for lists of school-aged children receiving benefits at two times

around the 2001-2002 school year: (1) summer 2001, and (2) December 2001. For the summer

time period, we requested that State welfare agencies submit data from the same month they had 

originally generated data for direct certification. For example, if a State welfare agency had 

generated the list of children receiving benefits in late July (using July data), we requested data

from July. Although we initially requested data on FDPIR receipt, the relevant data on FDPIR 

benefits was typically kept separately from FS and TANF benefits, so we focused our data 

request on receipt of FS/TANF alone.70

 70Most states provided us with information on recipients of FS and TANF. A few states gave us

information on FS receipt only. In California, we used data from the MEDS system. This system was

designed to track participation in California’s public health insurance program, but it also has information

on food stamp receipt (and nearly all—although not all—of California’s food stamp recipients are on this 

system). We used data from the MEDS system instead of attempting to collect food stamp administrative 

records because the latter are kept only at the county level, and MEDS data are available in a single statelevel file. 

A.13
The information on the children/families on FS/TANF as of summer 2001 and as of

December 2001 was collected from States to assess caseload turnover. In particular, the children 

eligible for direct certification based on FS/TANF receipt in the summer but not in December

might be considered administratively ineligible for free meal certification as of this later date.

We also conducted an analysis using data from the Survey of Income and Program Participation 

to determine the proportion of this group that may have remained income-eligible for free meal

certification via written application, even if their families no longer received food stamps or

TANF. 

We took three steps to collect State administrative data: (1) we sent introductory letters to 

State food stamp directors, (2) we telephoned these directors to secure cooperation, and (3) we 

followed up with either the directors or data processing staff in their office to work out the

details of the data provision. We sent the letters to food stamp directors in December 2001, 

targeting them because we thought they were most likely to be able to authorize the provision of 

the data we were requesting. The letters introduced the study, explained our data request, and 

alerted the directors that we would be calling them to seek their approval of the request. To help 

secure the cooperation of the food stamp directors, the central office of the USDA’s Food and 

Nutrition Service (FNS) sent letters to its regional offices explaining the study and asking them

to send letters or call the State food stamp directors in their region to encourage them to 

participate. 

Starting in January 2002, study team members began contacting the State food stamp 

directors by telephone. They introduced the study, described the study’s goals and the data being 

requested from States, and asked the directors if they had questions or concerns about the 

research. They then asked the directors to give permission for their States to participate in the

A.14
study and provide administrative data. These initial calls to State food stamp directors continued 

until May 2002. 

If, during the telephone call, the food stamp director granted permission for the State to 

participate in the study, a detailed data request letter was sent to the director of the food stamp 

program or to a contact person in the State agency designated by the food stamp director. Often, 

the State directors named a data processing staff member as a contact person for the data request,

and a copy of the detailed data request letter was sent to this contact person. The letter explained 

in detail the variables, formats, and time frame of the data we were requesting. In particular, the 

letter asked for lists of school-age children in families receiving FS/TANF/FDPIR at two 

times—summer 2001 and December 2001—along with each child’s age, zip code, and some

type of identification number. States were encouraged to provide the data in whatever form was 

most convenient for them, as long as it met the needs of the study. 

In March 2002, we began working with States that agreed to participate to answer their 

questions about the data being requested and to determine the best way for the State to transfer 

the data files. In May, follow-up email and telephone contacts were attempted with States that 

had agreed to provide data for the study but whose data had not yet been received. 

While calling State food stamp directors to request participation in this study, study team

members encountered some concerns about the confidentiality of the data and permission to 

release the data. Because several directors requested written verification of the study from the 

USDA, the project director requested that the USDA send a verification letter to States, and this

was done in February 2002. In addition, in response to specific requests from States, FNS 

headquarters recontacted regional FNS and asked them to speak with food stamp directors in 

certain States to confirm the legitimacy and importance of the study. MPR entered into formal

data-sharing agreements with three States to obtain data for this study. 

A.15
During the telephone calls to food stamp program directors, 43 States agreed to provide us

with data (Table A.3). Of these States, 37 actually provided the data. Efforts to obtain data from 

one State were stopped when the costs of obtaining the data were determined; given the project 

budget, these costs were judged to be too high.71

Seven States refused to provide administrative records data. One refused because of 

concerns about the privacy of individuals receiving food stamp benefits in the State. Another 

State had recently suffered a disaster and had to put an emergency program in place, which left

that State with inadequate resources to comply with the data request. The remaining States also

cited a lack of time and resources to produce the data requested for the study. The results of our 

data request in the 50 States and the District of Columbia are summarized below:

TABLE A.3 

STATE RESPONSES TO ADMINISTRATIVE DATA REQUEST 

Number of States That Have: 

Agreed to Participate and Provided Data 37 

Agreed to Participate but Did Not Provide Data 6 

Agreed to Participate but Only at a High Cost 1 

Refused to Participate 7 

The overall State response rate was 72 percent. Among districts using direct certification in 

our sample, 77 percent were in States that have provided data and as a result were used in the 

analyses based on this data source. Finally, among students enrolled in direct certification 

districts nationally, an estimated 83 percent were in States that provided administrative data. 

 71The food stamp director in this state initially agreed to participate if MPR would pay a contractor to 

extract the data and transfer the data file. The estimate of these costs turned out to be higher than had 

been anticipated (and budgeted) for the collection of administrative data from this state. Since few

districts in this state were in the study sample, we decided to abandon efforts to collect the administrative

records data from the state.

A.16
APPENDIX B 

ANALYSIS OF NON-RESPONSE ON THE SFA SURVEY AND STATE 

ADMINISTRATIVE DATA COLLECTION EFFORTS

B.1 
To determine whether there were differences in the characteristics of the sample school 

districts and States that did and did not provide data for the study, we conducted an analysis of 

survey and administrative data non-response. First, we examined differences in the 

characteristics of school districts in our sample that did and did not complete the SFA Survey. 

Then, we examined differences in the characteristics of sample districts for which we did and did 

not receive State FS/TANF administrative data. For this analysis, we focused on those sample

districts that use direct certification. 

A. SFA SURVEY NON-RESPONSE ANALYSIS 

As described in Appendix A, the response rate on the SFA survey was 81 percent, with 

1,223 of 1,512 eligible districts responding. Table B.1 shows selected characteristics of the

districts that responded and that did not respond to the survey. Responding and non-responding 

districts were similar in terms of their racial/ethnic distributions and the poverty rates in their 

counties. In addition, a similar proportion of each group were located in urban areas. 

Responding districts tended to be a bit larger than nonresponding districts, with a mean 

enrollment of about 14,000 compared with just under 9,000 for nonresponding districts. The 

only other significant difference with respect to the characteristics we measured was that 

responding districts were more likely to be located in rural areas (36 percent versus 30 percent) 

and less likely to be in suburban areas (49 percent versus 56 percent). Since sample strata were 

defined in part according to districts’ enrollment and the sample weights adjusted for differences 

in response rates by stratum, the sample weights should adjust for the fact that responding 

districts were slightly larger than nonresponding districts. 

B.2 
TABLE B.1

CHARACTERISTICS OF DISTRICTS IN SFA SAMPLE, BY SURVEY RESPONSE STATUS

Districts

Responded to

Survey 

Did Not Respond 

to Survey 

Enrollment (mean) 14,024.3*** 

(1,322.0) 

8,633.5 

(1,003.6) 

Racial/Ethnic Distribution (means) 

 Percentage White 73.7

(1.08) 

73.7 

(2.37) 

 Percentage Black 12.9

(0.79) 

11.5 

(1.62) 

 Percentage Hispanic 8.5

(0.66) 

8.9 

(1.45) 

 Percentage Asian American 1.5

(0.14) 

1.8 

(0.28) 

County Poverty Rate (mean) 12.3 

(0.16) 

11.8 

(0.31) 

Urbanicity (percentages) 

 Urban 15.0

(1.02) 

14.2 

(1.94) 

 Suburban 48.7**

(1.80) 

56.2 

(2.76) 

 Rural 36.2**

(1.37) 

29.6 

(2.54) 

Sample Size 1,223 289 

 *Significantly different from zero at the .10 level, two-tailed test 

 **Significantly different from zero at the .05 level, two-tailed test 

***Significantly different from zero at the .01 level, two-tailed test 

B.3 
B. NON-RESPONSE AMONG STATES IN PROVIDING ADMINISTRATIVE DATA

Not all State agencies were able to provide data for this study for the reasons discussed in 

Appendix A. Table B.2 lists the States that did and did not submit data. Sample districts located 

in States that did not provide data had to be excluded from our analysis of ineligibility among 

directly certified students. In addition, a small number of sample districts in Maine and New 

York were not included because we did not have the correct zip code information for these

districts. 

In Table B.3, we explore differences in the characteristics of districts that were included in 

and excluded from our analysis. The table includes only those districts that use direct 

certification and that responded to the SFA survey. No differences presented in the table were 

statistically significant. Districts included in the analysis are very similar to those not included in 

the analysis in terms of their enrollment levels, mean certification rate, the year they started

using direct certification, and the proportion of students certified for free meals through direct 

certification.

B.4 
TABLE B.2

STATES INCLUDED IN AND EXCLUDED FROM THE ANALYSIS OF 

DIRECT CERTIFICATION INELIGIBILITY 

States Included in Analysis States Excluded from Analysis 

Alabama 

Arizona 

Arkansas 

California 

Colorado 

Connecticut 

Delaware 

Florida 

Hawaii 

Idaho 

Illinois 

Indiana 

Iowa 

Kansas 

Kentucky 

Louisiana 

Maine 

Maryland 

Minnesota 

Missouri 

Nebraska 

Nevada 

New Jersey 

New York 

North Carolina 

North Dakota 

Ohio 

Oregon 

Pennsylvania 

South Carolina 

Tennessee 

Texas 

Utah 

Vermont 

Washington, D.C. 

Wisconsin 

Wyoming 

Alaska 

Georgia 

Massachusetts 

Michigan 

Mississippi 

Montana 

New Hampshire 

New Mexico 

Oklahoma 

Rhode Island 

South Dakota 

Virginia 

Washington 

West Virginia 

B.5 
TABLE B.3

CHARACTERISTICS OF DIRECT CERTIFICATION DISTRICTS INCLUDED AND NOT 

INCLUDED IN THE ANALYSIS OF DIRECT CERTIFICATION INELIGIBILITYa 

Direct Certification Districts 

Included Not Included

Enrollment (percentages)

Less than 500 29 32 

500 to 1,000 20 11 

 1,001 to 2,500 25 21

 2,501 to 5,000 11 21

 5,001 to 10,000 7 9

10,001 to 25,000 5 4 

25,001 to 50,000 1 1 

More than 50,000 1 <1 

Median Enrollment 1039 1263 

Mean Enrollment 

(Standard Error) 

3886 

(193.45) 

3370 

(329.97) 

Free and Reduced-Price Meal Certification Rate 

(percentages)

0 to 10 percent 5 11 

11 to 20 percent 10 8 

21 to 30 percent 20 16 

31 to 40 percent 22 17 

41 to 50 percent 17 10 

51 to 75 percent 22 34 

76 to 100 percent 4 4 

Mean Certification Rate 

(Standard Error) 

39.0 

(0.94) 

40.6 

(2.08) 

District Started Using Direct Certification 

Within Past 2 Years 6 9 

3 to 5 years ago 30 30 

More than 5 years ago 65 61 

Percentage of Students Certified for Free Meals Who 

Were Directly Certified 

0 to 10 percent 13 19 

11 to 20 percent 27 26 

21 to 30 percent 27 26 

31 to 40 percent 22 16 

41 to 50 percent 5 5 

51 to 75 percent 5 8 

76 to 100 percent 1 0 

Mean Percentage Directly Certified 

(Standard Error) 

25.3 

(0.88) 

23.7 

(2.14) 

Sample Size 608 184 

a

Districts not included in the analysis of direct certification ineligibility are either located in States which

did not submit data on FS/TANF eligibility, or contain zip code areas for which we do not have data. 

For a list of States that are included in the analysis, see Table B.2. 

B.6 
APPENDIX C 

SUPPLEMENTAL TABLES FOR ANALYSIS OF THE EFFECTS OF DIRECT 

CERTIFICATION ON RATES OF CERTIFICATION, PARTICIPATION, AND 

INELIGIBILITY AMONG CERTIFIED STUDENTS 

C.1 
TABLE C.1 

CHARACTERISTICS OF PUBLIC SCHOOL DISTRICTS AND STUDENTS 

(Standard Errors in Parentheses) 

Among Districts Among Students

DC 

Districts 

Non-DC 

Districts All Districts 

In DC 

Districts 

In Non-DC 

Districts In All Districts 

Enrollment (Percentages) 

<= 500 30 

(2.8) 

29 

(3.6) 

30 

(2.2) 

2

(0.2) 

2 

(0.4) 

2 

(0.2) 

501-1000 18

(1.9) 

19 

(2.4) 

18 

(1.4) 

4

(0.4) 

5 

(0.7) 

4 

(0.4) 

1001-2500 24

(1.8) 

25 

(2.4) 

24 

(1.0) 

10

(0.8) 

15 

(1.5) 

12 

(0.7) 

2501-5000 14

(1.2) 

16 

(1.7) 

14 

(0.6) 

13

(1.0) 

19 

(2.0) 

15 

(0.9) 

5001-10000 8

(0.7) 

8 

(1.1) 

8 

(0.4) 

14

(1.2) 

19 

(2.4) 

16 

(1.1) 

10001-25000 5

(0.5) 

3 

(0.6) 

4 

(0.4) 

19

(1.7) 

18 

(2.6) 

19 

(1.4) 

25001-50000 1

(0.1) 

1 

(0.1) 

1 

(0.1) 

12

(0.7) 

9 

(1.3) 

11 

(0.6) 

>= 50000 1

(0.1) 

<1 

(0.1) 

1 

(<0.1)

26

(1.3) 

13 

(3.0) 

22 

(1.2) 

Median 1122

(84.0) 

1102

(117.0)

1096

(65.8) 

3381***

(1402.1)

6798

(683)

10543 

(587.6)

Certification Rates (Means) 

Free meals 29.3 

(0.7) 

29.7 

(1.6) 

29.5 

(0.8) 

33.6

(0.7) 

32.5 

(1.5) 

33.2 

(0.7) 

Reduced-price meals 10.0*** 

(0.3) 

8.2 

(0.3) 

9.3 

(0.2) 

8.0

(0.1) 

7.7 

(0.2) 

7.9 

(0.1) 

Total 39.4

(0.9) 

37.7 

(1.7) 

38.7 

(0.8) 

41.6

(0.7) 

40.1 

(1.6) 

41.1 

(0.7) 

Participation Rates (Means) 

Among all students 62.4**

(1.0) 

58.2 

(1.3) 

60.7 

(0.8) 

54.7**

(0.5) 

52.4 

(1.0) 

54.0 

(0.5) 

Among students certified

for free meals 

78.0 

(0.8) 

76.2 

(0.8) 

77.1 

(0.6) 

75.8

(0.4) 

75.1 

(0.7) 

75.6 

(0.3) 

Among students certified

for reduced-price meals 

72.4 

(0.9) 

70.5 

(1.1) 

71.8 

(0.7) 

66.5**

(0.5) 

64.2 

(1.0) 

65.8 

(0.5) 

Among students not

certified

54.8*** 

(1.2) 

48.7 

(1.5) 

52.4 

(0.9) 

40.5

(0.6) 

39.0 

(1.1) 

40.1 

(0.5) 

Sample Size 792 425 1,222 792 425 1,222 

 *Difference between DC and non-DC districts is significantly different from zero at the .10 level, two-tailed test

 ** Difference between DC and non-DC districts is significantly different from zero at the .05 level, two-tailed test. 

*** Difference between DC and non-DC districts is significantly different from zero at the .01 level, two-tailed test. 

C.2 
TABLE C.2 

DESCRIPTIVE CHARACTERISTICS OF VARIABLES INCLUDED IN DISTRICT-LEVEL ANALYSIS 

All Districts 

Weighted Using District-Level

Weights 

Variable Unweighted

District-Level 

Weights

Direct 

Certification 

Districts 

Non-Direct 

Certification 

Districts 

Whether direct certification is in place 0.650 0.610 1.000 0.000 

Whether direct certification has been in 

place for: 

1 to 2 years 0.067 0.070 --- --- 

3 to 5 years 0.179 0.168 --- --- 

6 to 10 years 0.278 0.223 --- --- 

More than 10 years 0.093 0.095 --- --- 

Whether direct certification was 

formerly used in district but not in 

school year 2001-2002 

0.080 0.064 --- --- 

Type of direct certification 

implementation

Non-matching 0.106 0.121 --- ---

District-matching, passive consent 0.259 0.191 --- --- 

District-matching, active consent 0.040 0.058 --- --- 

State-matching, passive consent 0.138 0.113 --- --- 

State-matching, active consent 0.036 0.047 --- --- 

Mixed 0.066 0.073 --- ---

Percentage of certified free students in 

districts who were directly certified 

0.165 0.152 --- --- 

Whether district has schools that operate 

under Provision 2 or 3 and the base year 

was prior to 2001-2002 

0.066 0.057 0.041 0.082 

Size of district 

<= 500 0.094 0.294 0.299 0.287 

501 to 1,000 0.108 0.181 0.180 0.185 

1,001 to 5,000 0.405 0.391 0.377 0.409 

5,001 to 10,000 0.135 0.076 0.076 0.077 

10,001 to 25,000 0.108 0.042 0.048 0.034 

> 25,000 0.149 0.017 0.021 0.009 

Proportion of elementary 

school students 0.489 0.512 0.525 0.504

Urbanicity

Urban 0.151 0.069 0.063 0.077

Suburban 0.489 0.417 0.334 0.544

Rural 0.360 0.514 0.603 0.379

C.3 
TABLE C.2 (continued) 

All Districts 

Weighted Using District-Level

Weights 

Variable Unweighted

District-Level 

Weights 

Direct 

Certification 

Districts 

Non-Direct 

Certification 

Districts 

Regiona 

Northeast 0.105 0.119 0.169 0.042

Mid-Atlantic 0.112 0.087 0.076 0.104

Southeast 0.148 0.085 0.079 0.098

Midwest 0.209 0.214 0.109 0.380

Southwest 0.140 0.159 0.153 0.169

Mountains/Plains 0.128 0.190 0.276 0.057

West 0.158 0.144 0.139 0.153

Racial / ethnic distribution 

Percentage white 0.732 0.794 0.822 0.758 

Percentage black 0.115 0.070 0.067 0.071 

Percentage Hispanic 0.107 0.084 0.070 0.104

Percentage other 0.043 0.048 0.038 0.064 

Proportion of students who are “limited

English proficient” 

0.042 0.028 0.024 0.065

Poverty rate 

Within district 0.167 0.170 0.173 0.166 

Within county 0.123 0.127 0.127 0.127 

Month verification process Completed 

October or earlier 0.099 0.171 0.140 0.220 

November 0.181 0.180 0.190 0.160

December 0.672 0.603 0.630 0.570

January or later 0.048 0.046 0.040 0.060 

Type of verification sample selected 

Random sample 0.776 0.875 0.850 0.900 

Focused sample 0.214 0.116 0.140 0.080 

Other 0.010 0.008 0.006 0.015

Size of verification sample selected 

< 1% of applications 0.071 0.027 0.023 0.030 

1 to 2 % of applications 0.213 0.147 0.123 0.166 

2 to 4 % of applications 0.495 0.504 0.534 0.469 

4 to 10% of applications 0.184 0.272 0.275 0.260 

> 10% of applications 0.038 0.072 0.045 0.075 

Type of application used 

Single-child 0.664 0.746 0.784 0.682 

Multi-child 0.290 0.212 0.175 0.269

Other 0.046 0.042 0.041 0.049

District uses verification for cause 0.308 0.192 0.214 0.138 

District uses electronic point-of-sale 

system 0.646 0.488 0.571 0.376

a

Model actually includes binary variables indicating the State in which the district is located. Descriptive statistics are

presented for region instead of States for simplicity. 

C.4 
TABLE C.3 

STATE CHARACTERISTICS: 1990 TO 2001 

Characteristic Description Mean 

Standard 

Deviation 

Free Certification Rate Percentage of all students certified for free meals, as of

October 1990-2001 29.5 9.9 

Reduced-Price (RP) 

Certification Rate 

Percentage of all students certified for RP meals as of 

October, 1990-2001 6.8 1.6 

Free Participation Rate Among

All Students 

Average percentage of all students obtaining a free school 

lunch on a given day during October, 1990-2001 22.8 8.9 

Free Participation Rate Among

Certified Students 

Average percentage of free certified students obtaining a 

free school lunch on a given day during October, 1990-

2001 78.9 7.4 

RP Participation Rate Among 

All Students 

Average percentage of all students obtaining an RP school 

lunch on a given day during October, 1990-2001 4.6 1.4 

RP Participation Rate Among 

Certified Students 

Average percentage of RP certified students obtaining an 

RP school lunch on a given day during October, 1990-

2001 68.5 9.4 

Paid Participation Rate Among

All Students 

Average percentage of all students obtaining a paid school 

lunch on a given day during October, 1990-2001 28.2 8.9 

Paid Participation Rate Among

Non-Certified Students 

Average percentage of non-certified students obtaining a 

paid school lunch on a given day during October, 1990-

2001 43.7 13.5 

Overall Participation Rate 

Among All Students 

Average percentage of all students obtaining a school 

lunch on a given day during October, 1990-2001 55.6 10.8 

Direct Certification Use Binary variable indicating whether any district in the State 

was using direct certification during the year, 1990-2001 0.73 0.44 

Number of Years of Direct 

Certification Use 

Number of years that any district in State has been using 

direct certification as of a given year, 1990-2001 4.3 3.7 

Direct Certification Penetration

Rate, 1996 

Percentage of districts in the State that were using direct 

certification as of October 1996 75.3 35.7 

Direct Certification Penetration

Rate, 2001 

Estimated percentage of districts in the State that were 

using direct certification as of October 2001 74.6 27.6 

Percentage on AFDC/TANF Percentage of State’s residents who were receiving Aid to 

Families with Dependent Children (AFDC) or Temporary

Assistance for Needy Families (TANF), 1990-2001 3.7 1.9 

Percentage on Food Stamps Percentage of State’s residents who were receiving food 

stamps, 1990-2001 8.3 3.3 

Median Income Median household income in the State ($000) 39.7 6.5 

Poverty Rate Percentage of State’s residents living in poverty, 1990-

2001 12.8 4.0

C.5 
TABLE C.3 (continued) 

Characteristic Description Mean 

Standard 

Deviation 

Unemployment Rate Percentage of State’s labor force unemployed, 1990-2001 5.3 1.5 

Manufacturing Wage Mean hourly wage in State’s manufacturing industry ($), 

1990-2001 12.2 1.9

Sample Size 612 

Source: The certification and participation rates and the percentage receiving food stamps were drawn from the U.S.

Department of Agriculture, Food and Nutrition Service Data Bank, 1990 through 2001. The number of years 

of direct certification use was obtained from the 1996 Direct Certification Study State Survey. The percentage

receiving AFDC/TANF was obtained from the U.S. Department of Health and Human Services,

Administration for Children and Families. Median income and the poverty rate were obtained from the U.S. 

Bureau of the Census’ Current Population Survey. The unemployment rate and mean wage in the 

manufacturing industry were obtained from the U.S. Department of Labor, Bureau of Labor Statistics.

C.6 
APPENDIX D 

ANALYSIS OF DATA FROM THE SURVEY OF INCOME 

AND PROGRAM PARTICIPATION (SIPP) 

D.1
For this study, the rate of administrative ineligibility among directly certified students is 

defined as the rate at which students who were eligible to become directly certified (based on the 

receipt of FS/TANF benefits) had stopped receiving benefits by December. This is 

administrative ineligibility because the child no longer qualifies for free meals on the basis of 

FS/TANF participation. However, households that had stopped receiving FS/TANF may have 

continued to have incomes that qualify for free meals. We analyzed data from the Survey of 

Income and Program Participation (SIPP) to determine the rate at which FS/TANF leavers had 

household incomes above the income eligibility threshold for free meals (130 percent of

poverty), thus also making them income ineligible. We then used the results of this analysis to 

adjust the administrative ineligibility rate among directly certified students for our sample 

districts, forming an estimate of income ineligibility among directly certified students. The 

income ineligibility rate is an estimate of the percentage of directly certified students whose 

families no longer receive benefits and whose income is above 130 percent of poverty. These 

students are no longer eligible for free meal benefits on the basis of their income. 

We used data from the 1996 panel of SIPP, which followed a nationally representative 

sample of more than 100,000 individuals for 48 months. SIPP is a household survey in which 

any given respondent household was interviewed every four months between early to mid-1996 

and late 1999 to early 2000.72 For example, a household might have been interviewed in: (1) 

May 1996, (2) September 1996, (3) January 1997, (4) May 1997, (5) September 1997, (6)

January 1998, (7) May 1998, (8) September 1998, (9) January 1999, (10) May 1999, (11) 

September 1999, (12) January 2000. In each interview, the household is asked about four 

 72We used sample weights to ensure that the percentages shown in the chapter were nationally 

representative. 

D.2
different months—the current month and the three previous months. In their first interview, the 

example household would provide information covering February, March, April, and May 1996. 

These would be referred to as panel months 1 through 4. Overall, the 12 interviews provide 48 

months of data for each individual in a sampled household. 

One complication of the SIPP design for data analysis is that not all sample households were 

interviewed in the same months. In particular, the full sample was divided into four “rotation 

groups” that were interviewed in different months. Thus, at one extreme, the interview period 

covers December 1995 (month 1) through November 1999 (month 48). At the other extreme, the

interview period covers March 1996 (month 1) through February 2000 (month 48). 

To conduct the current analysis, we used SIPP data corresponding to the months April 

through December 1999 and included in our sample children from SIPP households who were 

approximately ages 5 to 16 during any of those reference months.73 

In the first step of our analysis, we examined how many children were in households 

receiving FS/TANF during the month when direct certification was conducted in their State. Of

those children, we determined the percentage that had stopped receiving benefits as of

December.74 Table D.1 shows these leavers rates by region and nationally. The final row of the

 73 The SIPP analysis included data from all states. If we had limited the analysis to those states that 

provided administrative data for the ineligibility analysis, our sample size would have been much lower.

In addition, we found no significant differences between the characteristics of states included in and 

excluded from the analysis of administrative data, as shown in Appendix B. 

74For households in our sample that were not interviewed in December 1999 (approximately one

fourth of the sample), we used data from November. We determined the percentage of children on

FS/TANF during the month prior to when direct certification was conducted in their state who had 

stopped receiving benefits by November. This ensured that the same amount of time passed between the

point at which students would become directly certified and the point at which we checked to see if they

were still receiving benefits. Since this raised the possibility of some students being included the sample 

who were not eligible for direct certification in the month when it was actually conducted, we also

conducted the analysis using only those households interviewed in December. The results of this analysis

were similar. 

D.3
TABLE D.1 

RATES OF LEAVING THE FS/TANF PROGRAMS, BY REGION 

Region 

Sample Size: Number on 

Food Stamps or TANF in 

Month of Direct

Certification December Leaver Ratea 

Northeast and Mid-Atlantic 186 21.1 

Mountains/Plains 78 34.8

Midwest 154 21.0

Southeast 753 16.2

West and Southwest 379 16.2 

National 1,550 18.1

December Leaver Rate Based on State FS/TANF 

Administrative Data 21.0 

Source: Survey of Income and Program Participation, 2001 Direct Certification Study State Administrative Data. 

a

For households in our sample that were not interviewed in December 1999 (approximately one-fourth of the

sample), we used data from November. We determined the percentage of children on FS/TANF during the month prior to

when direct certification was conducted in their State who had stopped receiving benefits by November. 

D.4
table shows the leaver rate determined by the State administrative data we collected. According

to the SIPP data for 1999, 18.1 percent of those on benefits nationally in the month of direct 

certification had stopped receiving the benefits by December. According to the FS/TANF 

administrative data provided by the States covering 2001, an estimated 21.0 percent of those 

receiving FS/TANF in the month of direct certification had stopped receiving benefits by

December. 

Next, we used the annual Federal poverty guidelines from 1999 to establish a monthly 

poverty threshold (by dividing by 12), and we divided SIPP households’ monthly income in

December by this threshold to determine their income as a percentage of poverty. Specifically,

we examined the incomes of FS/TANF leaver households with school-aged children—those who 

were on benefits when direct certification was conducted and who stopped receiving benefits by 

December.75 We determined the percentage of these families with incomes above 130 percent of 

poverty (Table D.2). These families were no longer income eligible to receive free meals as of 

December. Nationally, 31 percent of those eligible for direct certification and who had left 

benefits by December 1999 had household incomes above 130 percent of poverty.76 

The national rate of income ineligibility among FS/TANF leavers was then used to adjust 

the administrative ineligibility rate. For each of the districts in the SFA survey sample that used

direct certification and for which we had State administrative data, we multiplied their rate of 

administrative ineligibility among directly certified students (that is, the FS/TANF turnover rate) 

by this SIPP-based adjustment factor (the national proportion of leavers with incomes above 130 

 75For sample members not interviewed in December, we looked at November household income. 

76There is substantial variation in this rate of income ineligibility by region in the SIPP data. This

variation is due in part to the small sample sizes of FS/TANF leavers by region.

D.5
percent of poverty. The resulting percentage was our estimate of the rate of income ineligibility

among directly certified students in that district.77

TABLE D.2 

PERCENTAGE OF FS/TANF LEAVERS WITH INCOMES ABOVE 130 PERCENT OF POVERTY, 

DECEMBER 1999

Region 

Sample Size: 

Number on Benefits in Month of

DC that Left FS/TANF by

December 

Percentage of Leavers with Household 

Incomes above 130% Percent of 

Povertya 

Northeast and MidAtlantic 34 21.4 

Mountain Plains 31 33.4 

MidWest 33 60.7

SouthEast 137 22.6

West and SouthWest 62 36.8 

National 297 30.8

Source: Survey of Income and Program Participation

a

For households in our sample that were not interviewed in December 1999 (approximately one-fourth of the sample), we

used data from November. We determined the percentage of students eligible for direct certification during the month in

which direct certification was conducted in their State and had exited FS/TANF by November who had incomes above

130 percent of poverty by November. 

 77We also explored using separate regional adjustment factors to determine the income ineligibility

rates across direct certification districts in different regions. However, because these regional rates were

based on small samples, they were not estimated as precisely as the national rate. We decided it would be

most appropriate to use a single, national adjustment factor. 

D.6