Federal funding for the food assistance and nutrition programs reached almost $53 billion in fiscal 2006, over half

of USDA’s budget (Oliveira, 2007). Farmers, food companies, and program participants have unequivocally benefited from the increased food spending and improved

food security among participants. However, the limited information we have on the programs’ impacts on nutrition

and diet quality is mixed. Yet, in times of tight budgets, the

pressure to demonstrate program performance increases.

Program assessments and evaluations can also help programs respond to changing needs and environments.

The Food Stamp Program is one of the largest public

assistance programs in the Federal safety net. Its large

budget, by itself, would result in keen interest in assessing its performance. Another reason for interest is the

marked evolution in nutrition concerns since the program

was first designed. The program was designed to address

problems related to insufficient quantity of food. Today,

obesity is the most common nutrition problem among

Americans, a result of consuming too many calories in

relation to energy expenditures. In addition, the food

choices Americans make—too much in the way of solid

fats and added sugar and too few fruits, vegetables,

whole grains, and other healthful foods—contribute not

only to the obesity problem but also to the risk of chronic

diseases, such as heart disease, hypertension, and cancer.

Thus, improving diet quality has become an increasingly

pressing concern. The Food Stamp Program has responded with an increased emphasis on nutrition education,

promoting healthful choices while still allowing program

participants to make their own decisions. Given these

new priorities, how can we tell if the program is making

a difference in nutrition and diet quality and, if so, how

much of a difference?

Unfortunately, evaluating effects of the Food Stamp Program on diet quality is complex, expensive, and time

consuming. Most existing research on nutrition and

health effects of food assistance programs share three key

limitations: the difficulty in separating the effect of the

program itself from other factors that may be related to

program participation (that is, selection bias); relative age

of the data (which do not capture current programs or

population behaviors); and use of outdated dietary standards and assessment methods. In addition, conducting

new evaluations is typically very costly, both in terms of

dollars and time. To alleviate some of these problems,

ERS has made it a priority to improve the necessary tools

for evaluation—in particular, improved data, measures,

and analytic methods.

Improving Data: The ERS Data Initiative and

the Flexible Consumer Behavior Survey

Timely, accurate, and comprehensive data are needed to

improve outcome evaluation efforts for food assistance

programs. The ERS Consumer Data Initiative is designed

partly to improve evaluation by enhancing existing Federal data in a cost-effective manner. Major strategies

include (1) adding important questions to existing surveys, such as consumer behavior questions in the National Health and Nutrition Examination Survey; (2) expanding use of private-sector data, such as Nielsen HomeScan

Can Food Stamps Do More to Improve Food Choices? An Economic Perspective

How Can We Tell If We Are

Making a Difference?

ERS Efforts To Improve Evaluation of

Nutrition Outcomes

Elizabeth Frazao, Joanne F. Guthrie, David Smallwood

United States

Department of

Agriculture

Economic 

Research

Service

Economic Information Bulletin Number 29-8 September 2007

USDA/NAL
food purchase data; and (3) enhancing the value of existing

survey data through linkage with administrative data from

Federal programs.

As a part of its new consumer data initiative, ERS has

developed a Flexible Consumer Behavior Survey (FCBS)

module, which, starting in 2007, will be included in the

National Health and Nutrition Examination Survey

(NHANES). The new FCBS—which has been tested with

both average and low-income audiences—collects foodrelated knowledge, attitude, and behavioral data, including

knowledge and attitudes concerning Federal dietary guidance, use of food labels, expanded measures of food assistance program participation, food expenditures, food availability, and food-away-from-home habits. The resulting data

set will have the unique ability to link knowledge, attitude,

and behavior variables to food consumption, health, and

program participation data in a nationally representative

sample. We will be able to identify food stamp participants

and eligible nonparticipants within this sample, making it

useful for examining program outcomes related to diet

quality and health. Although the expanded data provided by

the FCBS does not directly solve the problem of selection

bias, it will improve our understanding of the relationship

of important economic and policy factors to program participation and outcomes and could expand analytical options

for addressing selection bias. 

Improving Measures: 

FSNE Measure Development

Food Stamp Nutrition Education (FSNE) is USDA’s major

activity to promote healthier food choices by food stamp

participants. However, no uniform national data on outcomes associated with FSNE are currently available. ERS is

working in close collaboration with the Food and Nutrition

Service, the USDA agency that administers the program, to

develop a relatively simple, inexpensive, standardized

measure of behaviors associated with dietary quality. This

measure could be administered among adult populations

across the United States who are eligible for or who are

receiving food assistance (see box, “Requirements for a

Proposed Instrument To Measure Outcomes of Nutrition

Education Efforts”). When completed, it will provide a feasible means of collecting sufficient data to generate Statelevel, other subnational, and national estimates. It also

could be useful in assessing differences in dietary-qualityrelated behaviors of food assistance program participants at

the regional or State level.

Improving Assessment and Program

Evaluation Methodology: The new 

Dietary Reference Intakes

Early studies that measured the nutritional impact of the

Food Stamp Program simply compared average nutrient

intakes of program participants and nonparticipants, typically as a share of the appropriate Recommended Dietary

Allowance (RDA). Findings of higher nutrient intake levels

2 Economic Research Service/USDA

The proposed data collection tool should be broadly applicable in

measuring the outcomes of nutrition education efforts and contribute materially to the overall advancement of nutrition education

evaluation by increasing the measurement consistency across

evaluations, thus making them more comparable and more interpretable. Consistent with these goals, the following objectives are

particularly applicable:

• The instrument should be relatively short. This will increase the

use and acceptability in a broad range of evaluation contexts,

where the resources available for evaluation data collection are

limited. It will also increase response rates. We visualize the

instrument requiring no more than 15 minutes to be administered.

• The instrument should be technically correct. Such issues as

question flow and skip logic should be conducive to successful

interviewing. The instrument’s indicators of nutrition knowledge

also should reflect sound nutrition research.

• The instrument should be applicable and understandable to a

wide cross-section of the low-income population, as defined by

such factors as ethnicity, urbanicity, and region of the country.

Dietary knowledge and practices tend to be highly influenced by

cultural orientation. Different groups in the population may routinely use different language or different words to refer to similar concepts. Ensuring that the final instrument is general

enough to accommodate such differences is important.

• The method for administering the instrument should be flexible.

Because telephone interviews require relatively fewer

resources, they are often the data collection mode of choice in

evaluation work. However, there may be some evaluation contexts where one-on-one in-person interviewing fits better into

the overall evaluation plans. Furthermore, in the current context

of nutrition education programs, many evaluations may take

place in group settings, so the instrument should also be suitable for this approach.

• The instrument should assess behaviors consistent with current

dietary guidance. The instrument is intended to assess dietary

behaviors that are consistent with the 2005 Dietary Guidelines

and the MyPyramid Food Guidance System, covering such topics as intake of particular foods, amounts of food, and weight

management. The instrument can be used to target nutrition

education efforts and to determine changes following nutrition

education.

Requirements for a Proposed Instrument To Measure Outcomes of Nutrition Education Efforts
among participants were then interpreted to indicate that

participation in the Food Stamp Program led to “improved”

nutrient intake for participants, based on the belief that

“more is better,” an approach that may have been appropriate in an earlier era in which underconsumption was the

major nutrition issue.

Over the past decade, however, improvements in the knowledge about human nutrient requirements led to the development of a new set of dietary reference standards—the

Dietary Reference Intakes (DRIs). In addition to being

based on more recent scientific studies, the new DRIs also

make clear the problems with the presumption that “more is

better” (for more details, see box, “The DRIs”).

The first problem is that, once intake is adequate and sufficient to meet dietary needs, consuming more offers no additional benefits. This problem is particularly relevant to studies that compared intakes using the RDAs because the RDA

values included a large margin of safety in order to cover

the needs of nearly all healthy individuals. As a result,

intakes below the RDA do not necessarily indicate insufficient intake.1 The second problem is that, for some nutrients, too high an intake may present a problem.

These two problems make it clear that just because average

intake for one group is higher than for a second group does

not necessarily mean that the first group is “better off.” Instead, they point to the importance of considering the entire

distribution of nutrient intake, rather than just the average.

This discovery led to the development of a new statistically

based methodology to assess nutrient intake using the distribution of nutrient intake and the distribution of requirements. The new methodology allows analysts to estimate

the proportion of a population subgroup with inadequate as

well as excessive intakes and, thus, provides a better and

more meaningful nutrition assessment methodology.

Economic Research Service/USDA 3

The DRIs

Estimated Average Requirement (EAR) The usual intake level estimated to meet the requirements of half the healthy individuals in a life stage and gender group. At this level of intake, the other half of the healthy

individuals in the specified group would not have their needs met.

Recommended Dietary Allowance (RDA) The usual intake level that is sufficient to meet the nutrient requirements of nearly all

(97.5 percent) healthy individuals in a particular life stage and gender group. RDAs are

estimated by adding two standard deviations to the EAR. Although defined similarly as

the 1989 RDAs, the new values may be different from the 1989 values.

Adequate Intake (AI) The recommended usual intake level based on experimentally derived intake levels or

approximations of observed mean nutrient intakes by a group (or groups) of apparently

healthy people who are maintaining a defined nutritional state or criterion of adequacy.

This measure is used when scientific evidence is not sufficient to establish an EAR (and

RDA).

Tolerable Upper Intake Level (UL) The highest level of usual intake that is likely to pose no risk of adverse health effects to

almost all individuals in the specified life stage group. As intake increases above the UL,

the potential risk of adverse effects increases.

Source: Institute of Medicine, 2000.

Findings from first-generation dietary assessments consistently

show certain nutrients with dramatic dietary deficiencies or

excessive intakes among some population subgroups, although

they are seemingly unaccompanied by evidence of adverse biochemical, clinical, or anthropometric health problems. Whether

these findings represent important or potential dietary problems

that might be addressed by policy and program changes or

whether they stem from methodological weaknesses in dietary

assessment methods and/or dietary reference standards is not

clear. Because the new DRIs were established with the goal of

reducing the risk of chronic disease and not just eliminating signs

of deficiency, observing or measuring any adverse health impact

in the short term (particularly among younger age groups) may

be difficult, even though the long-term health impact may still be

important.

The Dietary Reference Intakes (DRIs) replace the

Recommended Dietary Allowances (RDAs), last published in 1989 by the National Academy of Sciences. In

addition to being based on more recent scientific studies, the DRIs also differ in three significant ways from

the former RDAs: 

1However, the lower the intake relative to the RDA, the greater the

probability of inadequate intake.

• They include, to the extent possible, a reduction in risk of chronic disease, rather than merely the absence of signs of deficiency.

• They employ a new conceptual model that takes into account nutritional problems occurring due to either insufficient or excessive intakes.

• They encompass a more complete set of values, including an upper

level—EARs, RDAs, AIs, and ULs (see below).
Although the new DRIs and the new methodology have not

yet been used to evaluate the Food Stamp Program, they are

being used increasingly for general dietary assessments,

which are helpful in identifying nutrients of public health

interest. Recent findings from first-generation studies, however, have identified some nutrients for which considerable

dietary excesses or deficiencies have been estimated,

although unaccompanied by any reports of adverse health

effects or other type of concern (table 1). These findings

have raised some concerns about the accuracy of those

DRIs and whether they should be reviewed before they are

used for program evaluation or planning. 

An ERS-sponsored review of the models and methods used

in assessments of dietary intakes relative to the DRIs for

selected nutrients concluded that errors in dietary recall

data may partially—but not fully—explain some of the

findings. For example, the large proportion of adults identified as consuming inadequate amounts of vitamin E may be

partially explained by underreporting of food intake. Additional difficulties in collecting reliable data on the amounts

and types of fats and oils consumed and highly variable and

imputed data on vitamin E values in nutrient composition

databases further suggest that vitamin E intake may be

underestimated. However, the review also identified a number of limitations in the studies and data used to derive

those DRIs, raising the possibility that some DRI values

may benefit from additional scientific review (Devaney et

al., 2007).

For the remaining nutrients, however, we anticipate that

both the new standards and the methodology for assessing

nutrient adequacy will be useful for program evaluation,

following the Institute of Medicine’s example of how to

apply the new methodology to assess program impact

(Institute of Medicine, 2000).

Conclusions

A number of changes in Food Stamp Program policy have

been proposed to improve food choices and diet quality of

participants. Yet inadequacies of data, measures, and analytic

methods have limited our understanding of the program’s

effects on food choice and diet quality. Improving evaluation of the current program could provide a better sense of

the nature and extent of the problems that need to be

addressed. Improving evaluation is also necessary to assess

the effects of any proposed changes in the program that are

adopted. The problem of selection bias has not yet been

solved. Nevertheless, expanded data and better measurement and analytical methods, such as the ability to estimate

the change in the proportion of a population subgroup with

inadequate or excessive nutrient intakes, will aid us in conducting more definitive evaluations. These evaluations will

give policymakers, program officials, and interested citizens

the information they need to make better decisions.

Information Sources

Devaney, B., M.K. Crepinsek, K. Fortson, and L. Quay. Review of

Dietary Reference Intakes for Selected Nutrients: Challenges and

Implications for Federal Food and Nutrition Policy, Contractor

and Cooperator Report No. 28, U.S. Department of Agriculture,

Economic Research Service, February 2007, available at:

http://www.ers.usda.gov/Publications/ccr28/.

Guthrie, J.F., E. Stommes, and J. Voichick. “Evaluating Food

Stamp Nutrition Education: Issues and Opportunities,” Journal of

Nutrition Education and Behavior 38(1):6-11, January-February

2006.

Institute of Medicine. Dietary Reference Intakes: Applications to

Dietary Assessment, Washington, DC: National Academy Press,

2000.

Institute of Medicine. Dietary Reference Intakes: Applications to

Dietary Planning, Washington, DC: National Academy Press,

2003. 

Oliveira, V. The Food Assistance Landscape: FY 2006 Annual

Report, Economic Information Bulletin No. eib6-4, U.S. Department of Agriculture, Economic Research Service, February 2007,

Available at: http://www.ers.usda.gov/Publications/eib6-4/.

4 Economic Research Service/USDA

Table 1

Nutrients with dramatic dietary deficiencies or

excessive intakes 

Nutrient Findings

Energy Estimated energy intakes greatly exceed

energy requirements for infants and children

Zinc, vitamin A High share of infants and children have usual

intakes above Tolerable Upper Intake Levels

(ULs)

Magnesium, All subgroups of the population have high

vitamin E prevalence of inadequacy

Fiber, Intakes are very low relative to DRI standards

potassium

Source: Devaney et al., 2007.

The U.S. Department of Agriculture (USDA) prohibits discrimination in all its programs and activities on the basis of race, color, national origin, age,

disability, and, where applicable, sex, marital status, familial status, parental status, religion, sexual orientation, genetic information, political beliefs,

reprisal, or because all or a part of an individual's income is derived from any public assistance program. (Not all prohibited bases apply to all programs.) Persons with disabilities who require alternative means for communication of program information (Braille, large print, audiotape, etc.) should

contact USDA's TARGET Center at (202) 720-2600 (voice and TDD). 

To file a complaint of discrimination write to USDA, Director, Office of Civil Rights, 1400 Independence Avenue, S.W., Washington, D.C. 20250-9410

or call (800) 795-3272 (voice) or (202) 720-6382 (TDD). USDA is an equal opportunity provider and employer.