Contract No.: 53-3198-5-044

MPR Reference No.: 8305-012

Report on the Pretest

of the Reaching the

Working Poor and 

Poor Elderly Survey

Volume I

December 1999

Sheena McConnell

Michael Ponza

Rhoda R. Cohen

Submitted to: Submitted by:

U.S. Department of Agriculture Mathematica Policy Research, Inc.

Food and Nutrition Service 600 Maryland Avenue, SW

Office of Analysis and Evaluation Suite 550

3101 Park Center Drive Washington, DC 20024-2512

Alexandria, VA 22302 (202) 484-3477

Project Officer: Project Director:

Christine Kissmer Sheena McConnell
ii

ACKNOWLEDGMENTS

The authors would like to thank the many people who have helped with this project during the past

two years. Alana Landey, the project officer at the Food and Nutrition Service (FNS) of the U.S.

Department of Agriculture for the first two and a half years of the study, provided wonderful guidance,

advice, and ideas. Steven Carlson, Christine Kissmer, and Kristen Hyatt of FNS also provided valuable

direction and advice. We would also like to thank Pat Seward of FNS for her assistance in obtaining statelevel files of food stamp participants, used to identify respondents for the pretest. 

We also thank Bruce Bentson, Edward Sanders-Bey, Mary Strubbe, Mark Tandberg, and Ben Watts

for making available to us their respective state-level files of Food Stamp Program participants. 

John Hall developed the sample design and selected the various samples, with assistance from Thu Vu

and Barbara Kolln.

Anne Ciemnecki and Larry Snell reviewed the drafts of the screening and survey instruments and

provided constructive suggestions.

Phyllis Schulman, Roland Scurato, andAlisa DeSantis were responsible for overseeing the day-to-day

operations of the data collection effort involving both telephone and in-person interviews with participants

and eligible nonparticipants.

We would like to thank the Computer-Assisted Survey Methods Program (CSM), University of

California, Berkeley for making available to us computer programs forrespondent screening. We would

like to thank Barbara Kolln, Linda Bandeh, and Michael Watts for their work designing and implementing

the computer assisted telephone interviewing (CATI) screening instruments developed for the project.

Mark Dentini designed and produced daily reports from the CATI system. We would also like to thank

the group of almost 50 interviewers who skillfully conducted the interviews.

We would like to thank Linda Gentzik for designing the data entry program and Marianne Stevenson

andRachael Reed and their staff for entering the survey data. We would also like to thank Natalie Justh,

Melissa Schettini, and Miki Satake for their programming assistance.

George Carcagno reviewed an earlier draft of the report and provided valuable suggestions, which

have greatly improved the current version. Report production was provided by Felita Buckner.

Finally, the pretest would not have been possible without the cooperation of Food Stamp Program

participants and eligible nonparticipants who graciously contributed their time to participate in our study.
iii

NON-DISCRIMINATION POLICY

The U.S. Department of Agriculture (USDA) prohibits discrimination in all its programs and activities on

the basis of race, color, national origin, gender, religion, age, disability, political beliefs, sexual orientation,

or marital or family status. (Not all prohibited bases apply to all programs). Persons with disabilities who

require alternative means for communication of program information (Braille, large print, audiotape, etc.)

should contact USDA’s TARGET Center at (202) 720-2660 (voice and TDD).

To file a complaintof discrimination, write USDA, Director, Office of Civil Rights, Room 326-W, Whitten

Building, 14 and Independence Avenue, SW, Washington, DC. 20250-9419 or call (202) 720-5964

th

(voice and TDD). USDA is an equal opportunity provider and employer.
iv

CONTENTS

Chapter Page

EXECUTIVE SUMMARY ........................................... xiii

I INTRODUCTION .................................................... 1

A. OVERVIEW OF THE PRETEST .................................... 3

B. THE SCREENING INTERVIEWS .................................. 6

C. THE MAIN QUESTIONNAIRES .................................. 10

D. SITE SELECTION .............................................. 13

E. BEHAVIORAL CODING AND INTERVIEWER COMMENTS .......... 15

II EXPERIENCES IDENTIFYING SURVEY RESPONDENTS ................ 17

A. RDD SCREENING .............................................. 18

1. Response Rates ............................................ 18

2. Resource Requirements for Identifying Respondents Using RDD ..... 28

3. Further Eligibility Tests for Respondents Who Pass the RDD 

Screening Interview ......................................... 44

B. LIST-FRAME SCREENING ...................................... 48

1. Response Rates ............................................ 50

2. Resource Requirements for Identifying Respondents Using a 

List Frame ................................................ 53

C. SUMMARY AND IMPLICATIONS ................................ 60

1. It Is Feasible To Identify FSP-Eligible Nonparticipants 

Using RDD ................................................ 60

2. A List Frame is Needed if the Survey is to Include FSP 

Participants ................................................ 60

3. A Response Rate to the Screening Interview of 70 Percent 

Could be Achieved on the Screening Interviews ................... 61

4. We Recommend Modifying the RDD Screening Interview .......... 62
CONTENTS (Con’t)

Chapter Page

v

5. We Recommend Sending Advance Letters to Persons on the RDD 

Sample Frame ............................................. 64

6. We Recommend Lengthening the Field Period .................... 64

7. We Recommend Increasing the Rate at Which Persons on the 

List-Frame are Located ...................................... 65

III ADEQUACY OF THE MAIN QUESTIONNAIRES IN COLLECTING THE

INFORMATION THEY WERE DESIGNED TO COLLECT ................. 66

A. MAIN QUESTIONNAIRE COMPLETION RATES ................... 68

1. Main Questionnaire Completion Rates by Interview Mode .......... 69

2. Main Questionnaire Completion Rates by FSP Participant Status ..... 73

3. Main Questionnaire Completion Rates by Questionnaire Length ...... 73

4. Interview Break-Offs ........................................ 74

B. MAIN QUESTIONNAIRE INTERVIEW LENGTH .................... 75

C. CHOICE OF RESPONDENT FOR THE MAIN QUESTIONNAIRES ...... 77

D. PERFORMANCE OF INDIVIDUAL QUESTIONS .................... 78

1. Section A: Household Composition ............................ 79

2. Section B: Knowledge of the Food Stamp Program ................ 79

3. Section C: Food Stamp Program Participation History ............. 80

4. Section D: History of Food Stamp Applications .................. 82

5. Section E: Reasons for FSP Nonparticipation .................... 87

6. Section F: Receipt of Food Assistance from Sources 

Other Than the FSP ......................................... 92

7. Section G: Food Security .................................... 92

8. Section H: Employment History ............................... 95

9. Section I: Health ........................................... 96

10. Section J: Social Supports .................................... 96

11. Section K: Income and Expenses .............................. 96

12. Section L: Demographic Information .......................... 103

E. SUMMARY AND IMPLICATIONS ............................... 107
CONTENTS (Con’t)

Chapter Page

vi

IV THE INFORMATION COLLECTED BY THE QUESTIONNAIRES ......... 106

A. OVERVIEW OF RESEARCH OBJECTIVES AND ANALYTIC 

APPROACHES ............................................... 107

B. INFORMATION TO DETERMINE FSP ELIGIBILITY ................ 108

C. REASONS REPORTED BY RESPONDENTS FOR NOT 

PARTICIPATING IN THE FOOD STAMP PROGRAM ............... 110

1. Reasons for Currently Not Participating in the FSP ............... 110

2. Reasons Current Nonparticipants Stopped Receiving Food Stamps, 

Started But Did Not Complete an FSP Application, and Received 

But Did Not Use Food Stamps ................................ 126

D. COMPARISONS OF CHARACTERISTICS AND EXPERIENCES 

OF FSP PARTICIPANTS AND NONPARTICIPANTS ................ 137

1. Comparisons of Characteristics of Participants and 

Nonparticipants ........................................... 137

2. Comparisons of Participant and Nonparticipants on Their Past 

Experiences with the FSP ................................... 145

3. Comparisons of Participants and Nonparticipants on Factors 

that May Influence Barriers to FSP Participation ................ 149

E. COMPARISON OF FINDINGS BY ADMINISTRATION MODE ....... 151

F. SUMMARY AND IMPLICATIONS ............................... 151

1. Recommended Modifications to the Questionnaires ............... 151

2. The Information Obtained from the Direct Questions Will 

Be More Informative than the Information Obtained from 

Comparisons between Participants and Nonparticipants ............ 154

3. The Long Versions of the Questionnaires Could Be Shortened ...... 155

4. The Minimum Set of Information that Should Be Collected 

by a Survey on the Reasons for Nonparticipation ................. 156
CONTENTS (Con’t)

Chapter Page

vii

V CONCLUSIONS AND RECOMMENDATIONS ......................... 157

A. IT IS FEASIBLE TO CONDUCT A STAND-ALONE SURVEY 

ON THE REASONS FOR NONPARTICIPATION ................... 157

B. A LIST FRAME IS NEEDED IF THE SURVEY IS TO INCLUDE FSP 

PARTICIPANTS .............................................. 158

C. A FINAL RESPONSE RATE OF ABOUT 65 PERCENT COULD

BE ACHIEVED ON A STAND-ALONE SURVEY ON THE 

REASONS FOR NONPARTICIPATION ........................... 158

D. THE RDD SCREENING INTERVIEW USED IN THE PRETEST 

STRIKES THE RIGHT BALANCE BETWEEN DETERMINING 

ELIGIBILITY AND MINIMIZING NONRESPONSE ................. 161

E. THE SCREENING INTERVIEWS AND MAIN QUESTIONNAIRES 

SHOULD BE ADMINISTERED USING COMPUTER ASSISTED 

SURVEY METHODS .......................................... 161

F. A SURVEY ABOUT THE REASONS FOR NONPARTICIPATION 

SHOULD INCLUDE A SMALL SAMPLE OF HOUSEHOLDS 

WITHOUT TELEPHONES AND ASK ABOUT TELEPHONE

 INTERRUPTIONS ............................................ 162

G. A QUESTIONNAIRE ABOUT NONPARTICIPATION SHOULD 

INCLUDE CLOSED-ENDED STRUCTURED QUESTIONS ABOUT 

NONPARTICIPATION ......................................... 164

H. A QUESTIONNAIRE ABOUT NONPARTICIPATION SHOULD 

INCLUDE QUESTIONS TO DETERMINE FSP ELIGIBILITY, 

FOOD SECURITY, AND SOURCES OF OTHER FOOD 

ASSISTANCE ................................................ 165

I. THE QUESTIONNAIRES DEVELOPED FOR THIS STUDY 

SHOULD BE REVISED TO ADDRESS FSP NONPARTICIPATION 

ISSUES ARISING FROM RECENT WELFARE REFORM ............ 166
CONTENTS (Con’t)

Chapter Page

viii

J. A 20-MINUTE MAIN QUESTIONNAIRE WOULD COLLECT 

SUFFICIENT INFORMATION ABOUT NONPARTICIPATION 

TO MAKE POLICY DECISIONS ................................. 168

K. CONSIDERATION SHOULD BE GIVEN TO EXCLUDING 

PARTICIPANTS FROM THE SURVEY ........................... 169

L. CONSIDERATION SHOULD BE GIVEN TO ADDING A MODULE 

ABOUT FSP NONPARTICIPATION TO ANOTHER HOUSEHOLD 

SURVEY ..................................................... 171

REFERENCES ....................................................... 174

APPENDIX: OTHER RECOMMENDED REVISIONS 

TO THE QUESTIONNAIRES
ix

TABLES

Table Page

 II.1 COMPLETION AND RESPONSE RATES TO THE RDD 

SCREENING INTERVIEW ........................................... 20

 II.2 DISTRIBUTION OF POINTS AT WHICH THE RESPONDENT 

BROKE OFF THE RDD SCREENING INTERVIEW ....................... 24

 II.3 COMPLETION AND RESPONSE RATES TO THE RDD

SCREENING INTERVIEW, BY SITE ................................... 29

 II.4 RATE AT WHICH RESPONDENTS TO THE RDD SCREENING

INTERVIEW WERE FOUND TO BE ELIGIBLE FOR THE MAIN

QUESTIONNAIRES, BY SITE ........................................ 30

 II.5 REASONS RESPONDENTS TO THE RDD SCREENING

INTERVIEW WERE FOUND INELIGIBLE FOR THE 

MAIN QUESTIONNAIRES ........................................... 35

 II.6 FAILURE TO PASS ASSET SCREENS IN RDD INTERVIEW,

BY TYPE OF RESPONDENT ......................................... 38

 II.7 NUMBERS CALLED, TIMES EACH NUMBER DIALED,

AND TIME SPENT ON INTERVIEWING: RDD SCREENING .............. 41

 II.8 ESTIMATES OF RESOURCES REQUIRED TO IDENTIFY

SURVEY RESPONDENTS USING RDD ................................ 42

 II.9 NUMBER OF NONPARTICIPANT RESPONDENTS WHO

PASSED THE RDD SCREENING INTERVIEW BUT SEEM

TO BE INELIGIBLE BASED ON INFORMATION GIVEN

IN THE MAIN QUESTIONNAIRES .................................... 46

 II.10 COMPLETION AND RESPONSE RATES TO THE LIST-FRAME

SCREENING INTERVIEW ........................................... 51

 II.11 COMPLETION AND RESPONSE RATES TO THE LIST-FRAME

SCREENING INTERVIEW, BY SITE ................................... 52
TABLES (continued)

Table Page

x

 II.12 RATE AT WHICH RESPONDENTS TO THE LIST-FRAME

SCREENING INTERVIEW ARE FOUND TO BE ELIGIBLE

FOR THE MAIN QUESTIONNAIRES, BY SITE .......................... 54

 II.13 REASONS RESPONDENTS TO THE LIST-FRAME 

SCREENING INTERVIEW WERE FOUND INELIGIBLE 

FOR THE MAIN QUESTIONNAIRES. .................................. 56

 II.14 NUMBERS CALLED, TIMES EACH NUMBER DIALED, 

AND TIMES SPENT ON INTERVIEWING: LIST-FRAME

SCREENING ....................................................... 58

 II.15 ESTIMATES OF RESOURCES REQUIRED TO IDENTIFY 

SURVEY RESPONDENTS USING A LIST-FRAME ....................... 59

 III.1 CONTENT OF QUESTIONNAIRES .................................... 67

 III.2 RESPONSE RATES TO MAIN QUESTIONNAIRES,

BY QUESTIONNAIRE TYPE ......................................... 70

 III.3 SELECTED SUMMARY MEASURES FOR RESPONSE

RATES TO MAIN QUESTIONNAIRES ................................. 71

 III.4 ADMINISTRATION TIME FOR MAIN QUESTIONNAIRES,

BY QUESTIONNAIRE TYPE ......................................... 76

 IV.1 REASONS REPORTED BY NONPARTICIPANTS FOR NOT

PARTICIPATING IN THE FOOD STAMP PROGRAM .................... 112

 IV.2 SELECTED CHARACTERISTICS OF NONPARTICIPANTS

WHO REPORTED THAT THE MOST IMPORTANT REASON

FOR NONPARTICIPATION WAS RELATED TO A LACK OF

NEED FOR FOOD STAMPS ......................................... 121

 IV.3 REASONS NONPARTICIPANTS THINK THEY ARE INELIGIBLE ......... 127

 IV.4 REASONS NONPARTICIPANTS THINK IT IS HARD OR

COSTLY TO GET TO THE FSP OFFICE ............................... 128
TABLES (continued)

Table Page

xi

 IV.5 REASONS NONPARTICIPANTS THINK THE FSP APPLICATION

PROCESS IS TOO LONG AND COMPLICATED ........................ 129

 IV.6 REASONS NONPARTICIPANTS THINK THE FSP OFFICE IS

UNPLEASANT OR UNSAFE ........................................ 130

 IV.7 SOURCES OF PERCEPTIONS OF ELIGIBILITY FOR ONLY

SMALL BENEFIT .................................................. 131

 IV.8 REASONS NONPARTICIPANTS THINK THE FSP APPLICATION

IS TOO PERSONAL ................................................ 132

 IV.9 NONPARTICIPANTS’ FSP EXPERIENCES OVER

THE PREVIOUS THREE YEARS ..................................... 134

 IV.10 REASONS REPORTED BY NONPARTICIPANTS FOR

DISCONTINUING FOOD STAMP RECEIPT IN THE PREVIOUS

THREE YEARS .................................................... 135

 IV.11 SELECTED CHARACTERISTICS OF PARTICIPANT AND

NONPARTICIPANT HOUSEHOLDS .................................. 138

 IV.12 SELECTED EXPERIENCES OF RESPONDENTS WHEN

APPLYING FOR FOODS STAMPS ................................... 147

 IV.13 SELECTED EXPERIENCES OF RESPONDENTS USING

FOOD STAMPS ................................................... 148

 IV.14 FACTORS THAT MAY HELP POTENTIAL PARTICIPANTS

OVERCOME BARRIERS TO PARTICIPATION ......................... 150

 IV.15 REASONS REPORTED BY NONPARTICIPANTS FOR CURRENTLY

NOT PARTICIPATING IN THE FSP, BY INTERVIEW MODE ............. 152
FIGURES

Figure Page

 I.1 OVERVIEW OF PRETEST ............................................ 4

 IV.1 ANALYSES OF REASONS FOR FOOD STAMP PROGRAM

NONPARTICIPATION BY KEY CHARACTERISTICS AND

EXPERIENCES OF NONPARTICIPANTS .............................. 117
The number of persons eligible for food stamp benefits was estimated from Survey of Income and

1

Program Participation data and the number of participants was calculated from Food Stamp Program

administrative data (Stavrianos, 1997).

EXECUTIVE SUMMARY

Less than one-half of working households and less than two-fifths of elderly households that are

thought to be eligible for food stamps actually received them in 1994. One way of increasing our 1

understanding of the reasons for these low rates of participation in the Food Stamp Program (FSP) would

be to conduct a national survey of nonparticipants who are eligible for the program. Because such a survey

would have to overcome conceptual and operational challenges, the Food and Nutrition Service (FNS)

of the U.S. Department of Agriculture contracted with Mathematica Policy Research, Inc. to design and

test a survey of the reasons for nonparticipation among low-income working and elderly households. This

report discusses our experiences conducting a pretest of this survey and our recommendations for the

design and fielding of a larger national survey about the reasons for nonparticipation in the FSP.

We faced three major challenges in designing a survey of the reasons for nonparticipation in the FSP.

The first challenge was to identify people who were eligible for the FSP but did not participate in the

program. No lists of these people exist, so we needed to start with a random-digit-dialing (RDD) sample

frame. Second, to identify persons who were eligible for food stamps, we needed to strike a balance

between asking detailed and often sensitive questions to make an accurate determination of eligibility, and

keeping the screening interview short and the response rate high. The third challenge was to develop

questionnaires that collected sufficient information to identify the reasons for nonparticipation. 

OVERVIEW OF THE PRETEST

An RDD frame was used to identify FSP nonparticipants who were likely to be eligible for food

stamps. We called nearly 17,000 telephone numbers to identify 484 nonparticipants who were likely to

be eligible for food stamps and met our other criteria for inclusion in the sample. We also identified 92 FSP

participants using RDD. Another 86 FSP participants were identified from a list of program participants

provided by state FSP agencies. A short screening interview was used to check whether the respondents

met our criteria for inclusion in the sample. The survey pretest began in January 1998 and lasted about

three months.

A main questionnaire was administered to 451 respondents who met our criteria for inclusion in the

sample. The questionnaires asked about characteristics of the households, attitudes, experiences with the

FSP and, if the respondents were nonparticipants, about the reasons they did not participate in the

program. We developed eight different versions of the questionnaire. Each type of respondent--working

nonparticipant, working participant, elderlynonparticipant, elderly participant--was administered a different

questionnaire. And for each type of respondent, we used two different lengths of interviews--a short and

a long version. 
xiv

All the initial screening interviews were conducted by telephone. To test whether the questionnaires

could also be administered in-person, we administered about 15 percent of the questionnaires in-person.

The pretest took place in six urban sites and two rural sites.

CONCLUSIONS AND RECOMMENDATIONS

Our experiences and findings during the pretest suggest the following conclusions and

recommendations regarding conducting a national survey of the reasons for nonparticipation.

1. It is Feasible to Conduct a Stand-Alone Survey on the Reasons for Nonparticipation

The pretest showed that it is feasible to conduct a stand-alone survey about the reasons for

nonparticipation. However, the survey would require considerable survey resources, mainly because of

the difficulties identifying survey respondents. We estimate that it would take just over 18,000 hours of

interviewer labor to identify a sample of about 1,000 FSP-eligible nonparticipants fromworking households

and 1,000 eligible nonparticipants from elderly households.

2. A List-Frame is Needed if the Survey is to Include FSP Participants

If RDD isbeing used to identify nonparticipants, identifying participants at the same time requires little

additional interviewer time. However, identifying participants by RDDonce the sample of nonparticipants

has been identified is many times more costly than identifying participants using the list-frame. As we found

working and elderly participant households were not as prevalent in the population as working and elderly

nonparticipant households, a mixed-frame design would be the most efficient one if participants are included

in the survey.

3. A Final Response Rate of About 65 Percent Could Be Achieved on a Stand-Alone Survey

on the Reasons for Nonparticipation

Nonresponse is a concern because, if the factors that determine whether a person responds to the

survey are related to the reasons for nonparticipation, the survey findings may be biased. The overall

response rate to the pretest survey was about 51 percent--the response rate to the screening interviews

was just under 60 percent; the completion rate to the main questionnaires was 85 percent. With the

recommended changes to the survey, the response rate to the screening interviewsin a national survey

could be as high as 70 percent and the completion rate to the main questionnaire as high as 90 or 95

percent, yielding an overall response rate to the survey of 63 to 67 percent. The recommended changes

to the survey that would have the most effect on the response rate are: 
xv

C Changing the order of the questions on the RDD screening interview, so that the interviewer

begins with questions directly related to the FSP and does not ask about income until the

fourth or fifth question.

C Adding more interviewer probes to the screening interview to assure respondents of

confidentiality.

C Sending an advance letter about the study to persons on the RDD sample frame with listed

addresses.

C Lengthening the field period to increase the number of RDD telephone numbers for which the

residential status can be determined.

C Increasing the likelihood that the respondent to the screening interview can also respond to

the main questionnaire by relaxing the criteria for determining thehousehold member who can

respond to the main questionnaire.

C Administering a main questionnaire shorter than the long version used in the pretest.

4. The RDD Screening Interview Used in the Pretest Strikes the Right Balance Between

Determining Eligibility and Minimizing Nonresponse

Most previous studies of FSP nonparticipation used crude screening rules to create samples of

nonparticipants who were likely to be eligible for food stamps. The RDD screening interview used in the

pretest used more sophisticated screening rules that required data on income, vehicles, and assets. Even

so, we estimate that 38 percent of the respondents found eligible by the RDD screening interview seem to

be FSP-ineligible based on information given later in the interview. The screening interview used in the

pretest, with some changes, hits about the right balance between the two objectives of keeping the interview

short and simple and predicting FSP eligibility well.

5. The Screening Interviewand Main Questionnaires Should be Administered Using Computer

Assisted Survey Methods

A national survey about nonparticipation should use computer assisted survey interviews(CASI) for

both the screening interviews and the main questionnaires. CASI supports sample management and

scheduling, aids the interviewer in conducting complex skip logic, and automatically determines whether

the respondent is eligible for the sample. Using CASI will shorten the administration times of the main

questionnaires.
xvi

6. A Survey About the Reasons for Nonparticipation Should Include a Small Sample of

Households Without Telephones and Ask Telephone Households About Telephone

Interruptions 

About 20 percent of low-income households do not have working telephones. Because households

without telephones may have different reasons for not participating in the FSP than households with

telephones, a national survey of the reasons for nonparticipation should include a small sample of

respondents without telephones. Interviewers would administer both a screening interview and the main

questionnaire in-person.

Because working and elderly FSP-eligible households that do not have telephones are not common,

considerable survey resources would be needed to identify such households. For this reason, the sample

of non-telephone householdswould need to be small and screening for them should take place in areas with

a high concentration of non-telephone households. 

To reduce the required sample size of non-telephone households, we recommend collecting data about

past interruptions of telephone service from households currently with telephones. If non-telephone

households and households with interruptions in telephoneservice have similar reasons for not participating

in the FSP, data on interruptions in telephone service can be used to statistically adjust for the underrepresentation of non-telephone households.

We also recommend that both FSP participants and FSP-eligible nonparticipant non-telephone

households be administered the main questionnaires. By interviewing all FSP-eligible non-telephone

households, information would be collected to determine the FSP participation rate among non-telephone

households. It may be that the participation rate is so high among non-telephone households, that

nonparticipation among non-telephone households is not a concern.

7. A Questionnaire About Nonparticipation Should Include Closed-Ended Structured Questions

About Nonparticipation

In previous surveys, the questions about nonparticipation were typically broad and open-ended and

elicited responses that were too vague to inform policy decisions. Hence, we designed a series of direct

closed-ended questions about the reasons for nonparticipation. Each question asked whether a particular

reason was applicable to the respondent. At the end of the series of questions, we asked whether there

were other reasons why the respondent did not participate and which was the most important reason that

the respondent did not participate. For some reasons, more detailed follow-up questions were asked.

Even on their own, these questions could provide much detailed information about the reasons for

nonparticipation.
xvii

8. A Questionnaire About Nonparticipation Should Include Questions to Determine FSP

Eligibility, Food Security, and Sources of Food Assistance

Questions to determine the likely FSP eligibility of the respondent are important because including

persons who are not eligible for food stamps in the sample may bias the survey findings. Because of

concerns about the response rate, the screening interview cannot ask all the detailed questionsrequired to

determine FSP eligibility. However, some of these questions can be asked later in themain interview when

the interviewer has established rapport with the respondent.

Data on food security are important because if most nonparticipants are food secure, the FSP may

be meeting its mission of providing food assistance to those who need it, and low rates of participation are

not a cause for concern.

The findings from the pretest suggest that many working and elderly households do not participate in

the FSP because they feel they did not need food stamps. It is important to determine whether the lack

of need is because nonparticipants are receiving food assistance from other sources such as other

government programs, charities, family, or friends.

9. The Questionnaires Developed For This Study Should be Revised to Address FSP

Nonparticipation Issues Arising from Recent Welfare Reform

Changes associated with recent welfare reform may have affected the likelihood that working

households and, to a lesser extent, elderly households participate in the FSP. FSP participation may have

been affected in three ways. First, Temporary Assistance for Needy Families (TANF) clients who

discontinue TANF receipt because they find work, reach the time limits for TANF receipt, or are

sanctioned for notmeeting TANF work requirements may discontinue receipt of food stamp benefits at the

same time even if they are still eligible for food stamp benefits. Second, welfare reform may have reduced

the likelihood that persons applying for TANF also apply for food stamp benefits at the same time, because

TANF applicants are not made aware of their eligibility for food stamp benefits at this time. Third, welfare

reform may have increased the importance of psychological factors, such as the stigma of FSP receipt or

use or the desire to be self-sufficient, as reasons for nonparticipation. With some minor revisions, the

questionnaires could collect information on how welfare reform has affected the reasons for

nonparticipation in the FSP.

10. A 20-Minute Main Questionnaire Would Collect Sufficient Information About

Nonparticipation To Make Policy Decisions

Because of concerns of respondent burden and response rates, we recommend using a slightly shorter

version of the long questionnaires that would take about 20 minutes to administer. It would include

questions about the reasons for nonparticipation, previous experiences respondents have had applying for

and using food stamps, food security and sources of food assistance, the demographic composition of the

household, and questions about income, expenses, and vehicles. 
xviii

11. If Survey Resources are Limited, Consideration Should Be Given To Excluding Participants

From the Survey

We recommend that FNS consider conducting a telephone survey of only nonparticipants. A survey

of nonparticipants would save considerable survey resources and could provide sufficient information about

the reasons for nonparticipation to inform policy. Comparisons of participants and nonparticipants yield

findings that are suggestive ofreasons for nonparticipation but rarely provide firm evidence that a particular

reason is important. If resources permitted, surveying participants does allow comparisons between

participants and nonparticipants of economic and demographic characteristics and previous experiences

with the FSP which are useful in identifying types of persons who are most likely to not participate. It also

allows an exploration of the ways in which participants overcame real or perceived barriers to participation.

However, comparisons between participants and nonparticipants of household demographic and economic

characteristics, sources of other food assistance, and food security can be made from existing data.

12. Consideration Should be Given To Adding A Module About FSP Nonparticipation to Another

Household Survey

An alternative to conducting a stand-alone survey would be to add a short module about the reasons

for FSP nonparticipation to another household survey. The module would begin with the screening

questions, and only persons who meet the criteria in the screening interview would then be asked questions

about nonparticipation. At a minimum, we recommend a 15- to 20-minute module that includes: (1) the

screening questions, (2) the direct questions about nonparticipation, (3) questions about whether the

respondent has previously applied for or used food stamps, (4) questions about food security and sources

of food assistance.

Adding a module to an existing household survey would significantly reduce the costs of collecting the

data since only the additional costs associated with a longer interview would be incurred. Depending on

the survey to which the module is added, the response rate to the questions may also be higher. Some

household surveys contain detailed questions about income by source, and these data could be used to

screen for eligibility for the add-on module on nonparticipation. However, the household survey that the

module is added to must be large enough to ensure sufficient samples of FSP-eligible nonparticipants in

working and elderly households.
The number of persons eligible for the Food Stamp Program (FSP) was estimated from Survey of

2

Income and Program Participation (SIPP) data and the number of FSP participants was calculated from

FSP administrative data.

This was part of a study entitled Reaching the Working Poor and Poor Elderly.

3

1

I. INTRODUCTION

Less than one-half of working households and less than two-fifths of elderly households that were

thought to be eligible for food stamps actually received them in January 1994 (Stavrianos 1997). One way

2

of increasing our understanding of the reasons for these low rates of participation in the Food Stamp

Program (FSP) would be to conduct a national survey of nonparticipants who are eligible for the program.

Because such a survey would have to overcome conceptual and operational challenges, the Food and

Nutrition Service (FNS) of the U.S. Department of Agriculture (USDA) contracted with Mathematica

Policy Research, Inc. (MPR) to design and test a survey of the reasons for nonparticipation among lowincome working and elderly households. This report discusses our experiences conducting a pretest of

3

this survey and our recommendations for the design and fielding of a larger national survey about the

reasons for nonparticipation in the FSP.

Why should we care about low rates of participation in the FSP? This is an especially pertinent

question given that the aim of recent welfare reform legislation was to reduce the number of people

dependent on welfare. The answer is because the mission of the FSP is to provide food assistance to all

persons who need it, so low participation rates may be an indication that the program is not fulfilling its

mission. If the program has features that discourage persons who need food assistance from participating,

or if persons in need don’t know about the program or how to apply, then changes in the program need

to be made for it to fulfill its mission. On the other hand, if people do not participate because they do not
2

need food stamps, then the low participation rates would not be a cause for concern and program changes

would not be needed.

We faced three major challenges in designing a survey of the reasons for nonparticipation in the FSP.

The first challenge was to identify people who were eligible for the FSP but did not participate in the

program. No lists of these people exist, so we needed to start with a random-digit-dialing (RDD) sample

frame. Second, to identify persons who were eligible for food stamps, we needed to strike a balance

between asking detailed and often sensitive questions to make an accurate determination of eligibility, and

keeping the screening interview short and the response rate high. The third challenge was to develop

questionnaires that collected sufficient information to identify the reasons for nonparticipation. In previous

surveys about nonparticipation, the questions had elicited responses that were too vague to inform policy

decisions (McConnell and Nixon 1996).

This report discusses our experiences meeting these challenges. We address three broad questions:

1. How well did the procedures work for identifying respondents for the survey?

2. How well did the questionnaires collect the information they were designed to collect?

3. Do the questionnaires collect sufficient information to address the questions of why working

and elderly households have low rates of participation in the FSP?

Each of the next three chapters of this report addresses one of these questions. We conclude the report

with a discussion of our conclusions and recommendations. The rest of this introductory chapter describes

the survey pretest. 
3

A. OVERVIEW OF THE PRETEST

The structure of the pretest was complex. It used two sample frames (an RDD frame and a list frame),

10 different instruments (2 screening interviews and 8 main questionnaires), and two interview modes

(telephone and in-person). Figure I.1 illustrates the structure of the pretest.

The RDD frame was used to identify FSP nonparticipants who were likely to be eligible for food

stamps. We called nearly 17,000 telephone numbers to identify 484 nonparticipants who were likely to

be eligible for food stamps and met our other criteria for inclusion in the sample. We also identified 92 FSP

participants using RDD. The pretest of the RDD survey began in January 1998 and lasted about three

months. Another 86 FSP participants were identified from a list of program participants provided by state

FSP agencies. A short screening interview was used to check whether the respondents met our criteria

for inclusion in the sample. The list-frame survey began in February 1998 and lasted about two months.

Respondents who met our criteria for inclusion in the sample were administered a main questionnaire

that asked about their experiences with the FSP and, if they were nonparticipants, about the reasons they

did not participate in the program. We developed eight different versions of the questionnaire. Each type

of respondent--workingnonparticipant, working participant, elderly nonparticipant, elderly participant--was

administered a different questionnaire. And for each type of respondent, we used two different lengths of

interviews--a short and a long version. 

In a national survey on nonparticipation, both the screening questionnaires and themain questionnaires

would be administered by telephone to households that have working telephones but both the screening

and the main questionnaires would be administered in-person to households without telephones. In the

pretest, we tested whether the main questionnaires worked well in-person by administering about 15

percent of the main questionnaires in-person. The other 85 percent of the
FIGURE I.1

OVERVIEW OF PRETEST

Random-Digit-Dialing

(16,648)

List-Frame

(253)

Nonparticipants Identified

(484) Participants Identified

(178)

Working

Identified

(210)

Elderly

Identified

(158)

Working

Identified

(82)

Elderly

Identified

(84)

Interviews Not

Attempteda

(128)

Short In-person

Interviews

(10)

Long Telephone

Interviews

(76)

Long In-person

Interviews

(14)

Short Telephone

Interviews

(77)

Short Telephone

Interviews

(45)

Short In-person

Interviews

(6)

Long Telephone

Interviews

(69)

Long In-person

Interviews

(6)

Long In-person

Interviews

(6)

Long Telephone

Interviews

(30)

Short In-person

Interviews

(10)

Short Telephone

Interviews

(27)

Short Telephone

Interviews

(34)

Short In-person

Interviews

(9)

Long Telephone

Interviews

(26)

Long In-person

Interviews

(6)

aThese interviews were not attempted because either the respondent spoke Spanish or the target for that type of respondent had been met.

4
Some of the statistical results that are used in this report were produced using computer programs

4

made available through the Computer-Assisted Survey Methods Program (CSM), University of California,

Berkeley. Neither the CSM staff nor the University of California bear any responsibility for the results or

conclusions presented here. 

5

questionnaires were administered by telephone. To contain the cost of the pretest, we conducted all of the

screening interviews by telephone. 

The main questionnaires were administered using hard-copy instruments, both when administered by

telephone and by person. Both the RDD and list-frame screening interviews were conducted by telephone

using computer-assisted-telephone-interviewing (CATI). With CATI, the interview questions are 4

displayed on a computer screen and the interviewers type the responses directly into the computer. The

advantages of CATI over using a hard-copy interview are:

C It allows complicated skip logic. The computer will automatically follow the questionnaire

skip logic. For example, in the RDD screening interview, the CATI system automatically

presented different questions to respondents who said they received food stamps and to those

who said they did not.

C It can perform calculations. The CATI system automatically determined FSP-eligibility

based on responses to the screening questions so that the interviewers were not required to

do manual calculations.

C It aids in managing the sample. CATI automatically assigns respondents to the

appropriate sample cells (such as working nonparticipants) and maintains records of the status

of each cell.

C It aids in the scheduling of interviews. CATI automatically selects the telephone number

to be dialed and schedules callbacks to unsuccessful contacts at different times of the day and

on different days.

C It avoids costly data entry. The data are already in electronic form and can be more easily

transformed into an analysis data file.
6

WithCATI, up-front programming is required for each instrument. The savings from using CATI will

offset these programming costs if the sample is large, as it wasfor the screening interviews in the pretest.

However, because the number of respondents for each main questionnaire in the pretest was small, it was

more efficient to administer the main questionnaires using hard-copy instruments rather than CATI.

The pretest took place in eight sites in ten counties: (1) Suffolk County, Massachusetts, (2) Galveston,

Texas, (3) Adams County, Colorado, (4)Philadelphia, Pennsylvania, (5) Ramsey County, Minnesota, (6)

Durham, North Carolina, (7) Bedford County, Pennsylvania, and (8) Murray, Lincoln, and Lyon Counties,

Minnesota. 

B. THE SCREENING INTERVIEWS

To identify respondents for the full survey, we used a short telephone screening interview. (Volume

II of this report contains copies of the screening interviews). The screening interviews contain a number

of “tests” to determine whetherthe respondent is eligible for inclusion in the sample. Once the screening

interview determined that a respondent was not eligible for inclusion in the sample, the interview was

concluded. Respondents who were found eligible for inclusion in the sample were administered a main

questionnaire. 

A respondent was determined eligible for inclusion in the sample if his or her household contained

either a working or an elderly person and either (1) the household was participating in the FSP and had

applied for benefits in the previous three years, or (2) the household was not participating in the program

but was likely to be eligible for food stamp benefits. We required that participating respondents had

applied for food stamps within the previous three years so that the respondents could
These criteria were based on the findings of a study of the errors that would be made predicting FSP

5

eligibility using survey data (McConnell 1997).

7

clearly recollect their experiences applying for food stamps. Our criteria for whether a household was

counted as working, elderly, a FSP participant, or a FSP nonparticipant were as follows:

C Working. A household was counted as working if any adult in the household worked for pay

during either the current or previous month. The two-month time period allowed us to include

people who had some recent attachment to the labor market but were not currently working.

C Elderly. A household was counted as elderly if anyone in the household was 60 years of age

or older.

C FSP Participant. A household was counted as participating if anyone in the household

received FSP benefits in either the current or previous month (and had applied within the past

three years). We included households who reported receiving food stamps in the previous

month, as some people may not view themselves as participating if they have not yet received

their benefits for the current month.

C FSP Nonparticipant. A household is counted as not participating if no-one in the household

received FSP benefits in the current or previous month.

Determining whether a person is likely to be eligible for the FSP requires a lot of detailed information--

more than we could collect in a short screening interview. Hence, we chose the following relatively simple

criteria to simulate the FSP-eligibility determination process. Respondents were considered categorically5

eligible for food stamps if they reported that everyone in their households received during either the current

or previous month Temporary Assistance for Needy Families (TANF), Supplemental Security Income

(SSI), or General Assistance (GA). If everyone in the household did not receive these benefits, the

respondent was considered likely to be eligible for food stamps only if all of the following tests were met:

1. Their household income was less than 130 percent of the poverty threshold. This

simulates the FSP-eligibility test that requires gross household income not to exceed 130

percent of poverty. Although elderly households and households that contain disabled
8

persons are not subject to this gross income eligibility test, they are subject to a

requirement that income net of certain expenses and deductions does not exceed 100

percent of poverty. As simulating the net income test requires too many detailed questions

for a short screening interview, we approximated the net income test with the requirement

that gross income must not exceed 130 percent of poverty for elderly and disabled

households.

2. The household did not own any vehicle that was manufactured in the past five

years. For most FSP applicants, the fair market value of their vehicles (exceeding $4,650)

is counted as an asset. McConnell (1997) found that the age of the household’s vehicles

was a good proxy for their value and a good predictor of whether a household was eligible

for food stamps.

3. The value of the household’s financial assets was less than $3,000 if the

household contained an elderly person, and $2,000 if the household did not

contain an elderly person. The FSP asset eligibility test requires that the value of all

counted household assets, including the counted value of vehicles, do not exceed these

levels.

4. The household had not been informed by the FSP that it is ineligible for food

stamps in the previous two months. This will screen out some respondents who may

be ineligible for other reasons, such as they do not meet the citizenship requirement or they

are able-bodied adults without dependents who have not met the work requirement.

If the respondent passed these tests, the interviewer determined who in the household should be

administered a main questionnaire. Our criterion was that a respondent to a nonparticipant questionnaire

should be the person in the household who would apply for food stamps if the household decided to

participate and that the respondent to a participant questionnaire should be the person in the household who

last applied for food stamps. This meant that the respondent to the main questionnaire sometimes differed

from the person who was administered the screening interview. 

Using the responses to the screening interview, the computer determined whether an eligible

respondent should be administered a working nonparticipant, an elderly nonparticipant, a working

participant, or an elderly participant questionnaire. The computer determined randomly whether a

respondent should be administered a short or a long questionnaire. It also determined, by the respondent’s
9

zip code, whether the questionnaire would be administered by telephone or in-person. If the main

questionnaire was to be administered by telephone, the interviewer administered the questionnaire directly

after completing the screening interview. If the questionnaire was to be administered in person, at the end

of the screening interview the interviewer would tell the respondent that an interviewer would be calling

within one week to arrange an interview in his or her home.

To identify persons who are likely to be eligible for food stamps but not receiving them, we used RDD.

InRDD, the sample frame includes telephone numbers of households with high income and households that

contain neither a working nor an elderly person. Thus respondents were first asked a set of screening

questions to establish their eligibility for the survey. If we identified from the RDD sample frame a FSP

participant who was in an elderly or working household, and had applied for food stamps within the

previous three years,we included the participant in the sample and administered a participant questionnaire.

We also used a list frame developed from program-records files to locate FSP participants. We

requested that the states in the study provide us lists of current FSP participants who were in either a

working or an elderly household. We sent the lists to the local FSP offices, where caseworkers updated

the addresses and telephone numbers of persons on the sample frame. Persons on the list-frame were also

administered a screening interview to check that they met the criteria for inclusion in our sample. If we

found persons on the list-frame who were no longer receiving food stamps, they were deemed ineligible

for the survey.

The screening interviews were conducted in English and also, when needed, in Spanish. However,

we did not translate the main questionnaires and so did not conduct any main interviews in Spanish.
In-depth cognitive tests of the screening interviews and questionnaires were conducted in two counties

6

in Texas in March 1997. The questionnaires were revised to take into account the findings from these tests

(Ponza et al. 1997).

10

C. THE MAIN QUESTIONNAIRES

Respondents who met our criteria for inclusion in the sample were administered a questionnaire

designed to collect information about the reasons for nonparticipation. We designed eight versions of the

6

questionnaires that differ according to: (1) whether they were to be administered to persons in working

or elderly households; (2) whether they were to be administered to participants or nonparticipants; and (3)

whether they were “short” or “long” in the time required for their administration. (Volume II of this report

includes copies of all the questionnaires).

We included FSP participants in our survey so that we could compare the experiences, attitudes, and

characteristics of participants and nonparticipants. To facilitate this comparison, the participant and

nonparticipant questionnaires are similar. For example, they both ask respondents about their previous

experiences with the FSP. The working and elderly questionnaires are similar because in focus groups of

low-income working and elderly persons conducted for this study in 1996, the two groups gave similar

reasons for not participating in the FSP (Ponza and McConnell 1996). The main differences are that the

questionnaires for the respondents in working households include a section that asks about employment

and the questionnaires for the respondents in elderly households include a section that asks about health.

The long version of the questionnaires mainly differs from the short version in that it contains questions about

employment, income, expenses, and food assistance from other sources that are not included in the short

version. It also contains more questions about food security.
11

The main questionnaires covered the following topics:

C Reasons for Nonparticipation (Section E). This section was included in the

nonparticipant questionnaires only. Respondents were asked in a series of closed-ended

questions whether a specific factor was a reason they did not participate. It was important

to ask closed-ended questions as previous surveys have found that responses to open-ended

questions about reasons for nonparticipation have been too general to be useful. We also

asked respondents whether the reason was an “important” reason they did not participate and

asked them to name the one most important reason. For some reasons, we included followup questions that asked about the reason in more detail. To ensure that we asked about all

factors, we also asked whether there were “other” reasons why the respondent did not

participate.

C History of FSP Applications (Section D). We collected information from both

participants and nonparticipants who had previously applied for food stamps in the past three

years about the reasons they applied for food stamps, whether they applied for other benefits

at the time they applied for food stamp benefits, how the respondent applied for food stamp

benefits (such as, in-person or via authorized representative), and specific problems that they

may have encountered applying. We also identified respondentswho started the application

process but did not complete it and probed for the reasons they did not complete the process.

We also asked participants about factors that helped them overcome barriers to applying for

or using food stamp benefits.

C FSP Participation History (Section C). We asked both participants and nonparticipants

who previously received food stamps about their experiencesreceiving food stamp benefits

in the past three years. We ask the nonparticipants who previously received food stamps why

they stopped receiving them. 

C Knowledge of the FSP (Section B). As a lack of knowledge about how to apply for food

stamp benefits or the FSP eligibility rules may lead to nonparticipation, the questionnaires

included questions about factors that may be related to the respondents’ knowledge of the

program. We asked both participants and nonparticipants whether they received food stamps

as a child and whether they knew someone who received food stamp benefits. 

C Employment History (Section H). We collected information on employment for two

reasons. First, factors related to employment (such as the stability of employment) may

influence the decision to participate in the FSP. Second, information on earnings can be used

to make a more accurate determination of FSP eligibility. Only the long versions of the

questionnaires collected information on employment. All the long versions collected

information on earnings. The working questionnaires also asked about the type of job worked

and the work history of the person in the household who worked the most hours.
12

C Receipt of Food Assistance from Other Sources (Section F). We collected information

on the receipt of other food assistance for two reasons. First, receipt of other food assistance

has been found to be correlated with receipt of food stamps (McConnell and Nixon 1996).

Second, because a lack of need for food stamp benefits is sometimes given as a reason for

nonparticipation (Ponza and McConnell 1996), it is important to know whether the reported

lack of need is because of receipt of food assistance from other sources. All the long versions

of the questionnaires contain questions about the receipt of other food assistance.

C Food Security (Section G). In focus groups conducted for this study, people who said they

did not need food stamp benefits also admitted to sometimes going without food (Ponza and

McConnell 1996). From a policy perspective, it is important to determine whether the

respondents who say they do not need food stamps are actually food secure. All

questionnaires contained questions to determine the food security of the respondents’

households. 

C Health (Section I). As some elderly persons may have difficulties applying for and/or using

food stamps, we asked questions about the general health and physical and cognitive

functioning of the respondent. We did not ask these questions in the working questionnaires.

C Social Supports (Section J). The presence of social supports may be an important factor

in determining participation in the FSP. It may indicate the extent to which family and friends

can act as a safety net to the respondent. We asked in the long questionnaires a series of

questions aboutthe length of time the respondent has lived in the neighborhood, the frequency

of social visits, and whether the respondent’s relatives live close by. 

C Income and Expenses (Section K). Information on income and expenses of the respondent

is important for two reasons: (1) economic factors, such as income, may influence the

decision to participate, and (2) information on income and expenses will be used to make a

more informed decision about whether the respondentis FSP-eligible. The questionnaires also

asked about household vehicles. This section was included in the long questionnaires only.

C Household Composition and Demographic Information (Sections A and L).

Information on the demographic composition of the household is collected because it may

determine the likelihood of participation. In addition, this information can be used as covariates

when examining factors that affect the decision to participate in the FSP. We also asked

about citizenship--a factor that may affect FSP-eligibility.

We administered the questionnaires to 451 respondents. The interviews were divided approximately

equally between respondents from working households and respondents from elderly households. About

two-thirds of the respondents were nonparticipants and one-third of the respondents were participants.
13

We conducted 69 (15 percent) of the interviews in-person. The remainder of the interviews were

conducted by telephone. To contain the cost of the pretest, the in-person interviews were all conducted

in areas with specific zip codes in two of the eight pretest sites. All the main questionnaires, in both inperson and telephone sites, were administered by an interviewer using a hard-copy questionnaire. 

D. SITE SELECTION

We chose to conduct the pretest in eight sites rather than selecting a nationwide probability sample,

because it reduced the costs of in-person interviews and obtaining program records, while still providing

sufficient information to thoroughly test the questionnaires and methods for identifying FSP-eligible

nonparticipants. The eight sites were:

1. Suffolk County, Massachusetts (Boston)

2. Galveston, Texas

3. Adams County, Colorado (Denver)

4. Philadelphia, Pennsylvania

5. Ramsey County, Minnesota (Minneapolis)

6. Durham, North Carolina

7. Bedford County, Pennsylvania

8. Murray, Lincoln, and Lyon Counties, Minnesota

Telephone interviews were conducted in all eight sites. In-person interviews were conducted in Galveston,

Texas and Durham, North Carolina.

The sites were selected with a combination of random and purposive sampling. The sampling took

place in two stages. In the first stage, we randomly selected six areas corresponding to the Census defined
Raleigh is the MSA that contains Durham. 7

These areas were selected by asking the FSP agency for areas with a high concentration of working

8

or elderly persons.

14

metropolitan statistical areas (MSAs) and primary metropolitan statistical areas (PMSAs). One area was

selected from each of six of the seven FSP regions. We stratified the sampling to include at least two

MSAs with Electronic Benefit Transfer (EBT) (Galveston and Raleigh ) and two large metropolitan areas

7

(Philadelphia and Boston), two medium-sized metropolitan areas (Denver and Minneapolis), and two smallsized metropolitan areas (Galveston and Raleigh).

In the second stage, we randomly selected one urban county within each MSA/PMSA from counties

that have 10 percent or more of their population in poverty. In one randomly selected MSA/PMSA--

Philadelphia--we selected only from counties that also had 20 percent or more of their population over 60

years of age. To select the two rural sites, we first randomly selected two of the six study states--

Pennsylvania and Minnesota. We chose the two rural sites in states that also contain an urban site to

reduce the cost of collecting program-records data. We then randomly selected a rural county in the

chosen state from a list of rural counties that have poverty rates of 10 percent or more. Because the

selected counties in Philadelphia, Boston, and Galveston were large and contained many FSP offices, we

selected an area within each county as our site. Conversely, the population of Murray County was too

8

small, so we expanded the definition of the site to include adjacent Lincoln and Lyon Counties. 

E. BEHAVIORAL CODING AND INTERVIEWER COMMENTS
15

In most ways, the pretest was administered like a regular survey with a small sample size. However,

we added two components to the survey to collect more information on the performance of the

questionnaires.

First, the last section of the questionnaires included a set of debriefing questions forthe interviewers.

These questions asked the interviewers to record any questions that respondents complained about or

found difficult to answer.

Second, we conducted behavioral coding on 40 of the completed interviews. These interviews were

divided roughly equally among interviews using questionnaires of each type. Ten of the coded interviews

were conducted in-person. The coding involved taping the interviews and then coding respondent and

interviewer behaviors during the interview. 

Respondent behaviors that were coded included: long pauses before answering; asking interviewers

to repeat the question; objecting to the question; reluctance to answer the question; asking the interviewer

to clarify the question; digressing when answering; or interrupting the reading of the question. While it is not

necessarily a problem if some respondents exhibit some of these behaviors, if many respondents exhibit

these behaviors, it usually indicates that there is a problem with the question. Similarly, if interviewers make

wording changes, probe incorrectly, or do not follow the skip patterns, this may also indicate a problem

with the questionnaire. 

The coder also noted whether the interaction between the interviewer and respondent was low,

medium, or high. A low interaction was coded if the interviewer just asked the question and the respondent

gave an answer. The respondent may have paused or made a comment before answering, but did not ask

the interviewer for clarification. A medium interaction was coded if the interviewer asked a question, the

respondent asked for clarification or for the interviewer to repeat the question, the interviewer provided
16

clarification or repeated the question, and then the respondent answered the question. A high interaction

was coded if the interviewer was required to repeat or clarify the question at least twice after the initial

asking of the question. For example, the interviewer asked a question, the respondent asked a question,

the interviewer replied, and then the respondent asked another question requiring a reply. A high

interaction may indicate a problem with the question.
17

II. EXPERIENCES IDENTIFYING SURVEY RESPONDENTS

A major challenge in conducting a survey ofthe reasons for FSP nonparticipation is to identify persons

who are eligible for food stamps but are not receiving them. While the FSP agencies maintain lists of

program participants, no lists exist of persons who are eligible for, but do not receive, food stamps. One

of the main purposes of the pretest was to investigate whether it is feasible, at a reasonable cost, to identify

eligible nonparticipants for the survey using random-digit-dialing(RDD) and a short screening interview.

This chapter discusses our experience in identifying both eligible program nonparticipants and program

participants using RDD. We also discuss our experience using lists of program participants to locate FSP

participants eligible for the survey.

Two main criteria are used to assess our success in identifying persons eligible for the survey--the

response rate to the screening interview and survey costs. A low response rate is a cause for concern

because findings from the survey will be biased if persons who do respond to the interview differ

fundamentally from those who do not respond in their reasons for nonparticipation, experiences with food

stamps, or general attitudes. Cost is always a concern, especially so for a survey about nonparticipation

for which the survey costs are likely to be high. In addition to these two criteria, we also examine how

many respondents are determined eligible for food stamps by our RDD screening interview, but later in the

main questionnaires report income, vehicles, and expenses that suggest that they are ineligible for food

stamps. 

The remainder of this chapter is organized into three sections. Section A discusses our experience

identifying respondents using RDD. Section B discusses our experience identifying participants using a list-
18

frame. We summarize our main findings and discuss their implications for administering the survey on a

national scale in Section C.

A. RDD SCREENING

To identify persons who were eligible for food stamps but not participating in the FSP, we used listassisted RDD sampling. It was “list-assisted” in that we purchased lists of telephone numbers in the chosen

sites from a commercial vendor. The vendor creates a list of all possible telephone numbers in an area,

including both those that are listed in the telephone directory and those that are not. 

To decrease the number of business telephone numbers on the lists, the vendor removes all telephone

numbers that are listed in the yellow pages but not the white pages of the telephone directory and restricts

the sample to telephone numbers for which there is at least one known residential telephone number with

the same eight first digits (including area code). However, even after these steps, the sample includes

telephone numbers that are not in service and telephone numbers of businesses and other nonresidential

organizations. So the survey interviewer’s first task when calling a telephone number is to determine

whether the number is in service and belongs to a residence. If the telephone number is found to be

working and it belongs to a residence, the screening interview is conducted to determine whether the

respondent is eligible for a main questionnaire.

1. Response Rates

The response rate to the RDD screening interview was about 60 percent. While this response rate

is about the average for recent RDD telephone interviews, this rate is still a cause for concern.

Nonresponse to a survey generates uncertainty about the validity of the survey’s findings. If nonresponse

was completely random it would not be a problem. However, we generally do not know whether or not
19

it is random. If nonresponse varies with characteristics of interest, there will be nonresponse bias--the

observed findings will differ from the findings that would have been observed if there had been no

nonresponse. 

The response rate to an RDD screening interview has two components: (1) the rate at which the

interviewer can determine whether the telephone number belongs to a residence, and (2) the rate at which

the respondent completes the screening interview. Table II.1 summarizes the components of the response

rate.

a. Determining Residential Status

Interviewers called 16,648 different telephone numbers in the RDD sample. They determined whether

the number worked and belonged to a residence for 13,870 numbers--a completion rate for determining

residential status of 83 percent. Of these 13,870 numbers, 8,623 (62 percent) numbers belonged to

residences and 5,247 (38 percent) numbers were either not working or belonged to businesses or other

nonresidential organizations. The interviewers called each number up to 40 times on different days and at

different times of the day before the number was “retired.” Even so, interviewers could not determine

whether 2,778 telephone numbers belonged to a residence, mainly because no-one answered the

telephone. 

The completion rate for determining residential status varies considerably between RDD surveys, but

it is typically over 90 percent. One explanation for the low completion rate for determining residential status

in this study was that the field period for the RDD survey was only three months. MPR’s experience has

shown that the completion rate can increase significantly if the field period is extended, allowing more

attempts at obtaining an answer to a telephone call over a longer period of time. 
20

TABLE II.1

COMPLETION AND RESPONSE RATES TO THE RDD SCREENING INTERVIEW

Total Numbers Dialed 16,648

Residential Status Determined 13,870

Residential 8,623

Nonresidential 5,247

Residential Status Unknown 2,778 a

Rate of Completing Determination of Residential Statusb83.3%

Residential Eligibility Ratec62.2%

Eligible for Screening 8,623 d

Eligibility Determined 6,155

Eligible for main questionnaire 576

Working nonparticipants 355 e

Elderly nonparticipants 191 e

Working participants 54 e

Elderly participants 48 e

Ineligible for main questionnaire 5,579

Eligibility Unknown 2,468

Hung-up during introduction 231

Refused after introduction 815

Refused during screening interview 1,282

Language or disability prevented completion of interview 140

Rate of Completing the Eligibility Screening Interviewf71.4%

Rate of Eligibility for Main Questionnairesg9.4%

Response Rateh59.5%
TABLE II.1 (Continued)

21

NOTES:

 Cannot determine whether the telephone belongs to a residence. In most cases, the telephone was never

a

answered.

The number of telephone numbers for which residential status is determined as a percentage of the number

b

of telephone numbers dialed.

The number of telephone numbers which belong to a residence as a percentage of the number of

c

telephone numbers for which the residential status is determined.

Telephone numbers that belong to a residence. d 

 Eligible respondents who are in both working and elderly households are counted as both working and

e

elderly. We identified 484 nonparticipants and 92 participants in total.

The number of respondents who completed the screening interview as a percentage of the number of

f

respondents eligible for screening.

The number of respondents found eligible for the main questionnaires as a percentage of the number of

g

respondents who completed the screening interview.

 The number of respondents who completed the screening interview as a percentage of the number of

h

residential (both known and unknown) telephone numbers called. It is the product of the rate of completing

the determination of residential status and the rate of completing the eligibility screening interview.
22

b. Determining Eligibility for the Survey

After determining that the telephone number belonged to a residence, the interviewer conducted a

short screening interview to determine whether the respondent was eligible for the survey. The

interview was designed so that most people who were ineligible for the survey completed the interview after

being asked only the first two questions. These questions asked about household size and whether

household income was less than an amount equal to 130 percent of the poverty threshold (an amount

dependent on the household’s size.) If the respondent reported income of less than 130 percent of poverty,

the interviewer then asked questions about whether the household contained either a working or an elderly

person, food stamp receipt, whether the household had recently been determined ineligible for food stamps,

whether everyone in the household received TANF, GA, or SSI, and questions about the household’s

assets. We count a screening interview as “complete” if the respondents answered sufficient questions for

us to determine whether the person was eligible for the survey. We completed 6,155 screening interviews,

71 percent of the 8,623 telephone numbers that we determined belonged to residences.

The 2,458 persons who did not complete the screening interview can be divided into four broad

groups (see Table II.1):

1. Persons who hung-up the telephone during the introduction. Although the introduction

consisted of only four short sentences, this group comprised 9 percent of all respondents who

did not complete the screening interview and about 3 percent of all respondents eligible for

screening.

2. Persons who refused immediately after the introduction in the screening interview.

A further 33 percent of all respondents who did not complete the screening interview (about

9 percent of all respondents) refused to answer any screening questions. 

3. Persons who refused during the screening interview. Fifty-two percent of persons who

did not complete the screening interview (about 15 percent of all respondents) could not

answer or refused to answer a specific question in the screening interview.
Just over 1 percent (103) of the RDD screening interviews were conducted in Spanish. 9

The denominator includes respondents who did not respond to later questions. 10

23

4. Persons who did not complete the screening interview because of language or

disabilities. Screening interviews were conducted in English and Spanish. About 6 percent

9

of respondents who did not complete a screening interview (2 percent of all respondents) did

not complete it because they spoke neither English nor Spanish or they were too ill or

incoherent to respond to an interview. 

Just over 40 percent of the respondents who did not complete the screening interview were not even asked

the first question, suggesting that the length of the screening interview was not an important contributor to

nonresponse.

To explore whether there were specific questions that discouraged persons from completing the RDD

screening interview, we present a summary of the nonresponse to each question in Table II.2. If a

respondent did not answer one of these questions, the screening interview was concluded after that

question. With one exception, the questions are listed in the order they are asked on the screening

interview. The exception is the question about whether the respondent had applied for food stamps within

the past three years. This question was asked after the question about the receipt of food stamps but only

of those respondents who received food stamps in the current or previous month. The second, third, and

fourth columns present the percentage of respondents who did not respond to each question (either

because they refused the question or because they said they did not know the answer) as a percentage of

all respondents asked the question. The last column presents the percentage of all respondents who did

10

not respond to the question as a percent of all respondents who did not complete the screening interview.

A striking finding from Table II.2 is that nearly 10 percent of people who were asked the first question

(about how many people lived in their household) refused to answer it. Nearly 57 percent 
24

TABLE II.2

DISTRIBUTION OF POINTS AT WHICH THE RESPONDENT BROKE OFF THE RDD SCREENING INTERVIEW

Percentage of Respondents Who Did Not Respond to the Question

Of All Respondents Asked the Question During the Screening Interview

Of All Respondents Who Broke Off

Question Refused Know Nonresponse Total Nonresponse a

Don’t Total

s1. How many people live in your household? 9.6 0.2 9.8 56.9

s1a/ab Is your income less than (130% of poverty)? 5.0 1.7 6.8 35.5

s2. Did you receive food stamps this month or last? 0.6 0.3 0.9 1.0

s4/5 Does anyone in your household work?b

Is anyone in your household over age 60? 0.8 0 0.8 0.8 b

s7. Does anyone in your household own a vehicle? 0.5 0.2 0.7 0.5

s8. Was the vehicle manufactured in the previous 5 years? 0.5 0.7 1.2 0.5

s13. Is your cash and money in checking and savings accounts less than

($2,000/$3,000)? 5.2 2.3 7.5 3.8

s14. Do you have any other assets? 0.7 0.7 1.4 0.5

s15. Are these other assets less than ($2,000/$3,000)? 0 4.6 4.6 0.2

s16. Is the value of all your assets less than ($2,000/$3,000)? 0 0 0 0

s19a. Did you apply within the previous three years? 0.9 2.6 3.5 0.9

Total number of respondents who began the screening interview - - 7,437 -

Total number of respondents who broke off during the screening interview - - - 1,282

NOTES:

The questions have been paraphrased for brevity. a 

The screening interview is considered incomplete if the respondent did not respond to both the question of whether the household contains a working person and the question about b 

whether the household contains an elderly person. The question numbers for participants are s23 and s24.
The CATI system automatically selected the appropriate income threshold for the reported

11

household size. The income threshold was rounded to the nearest fifty dollars so that it would not give the

impression that the respondent needed to know the exact amount of his or her household income.

25

of all persons who began the interview but did not complete it, refused to answer the first question.

Including persons who refused before or during the introduction, over 70 percent of all persons who did

not complete the screening interview refused before the interviewer asked the second question.

The second question asked whether the respondent’s household income was less than an amount

approximately equal to 130 percent of poverty for the respondent’s household size. Even though this is

11

a sensitive question and asked early in the interview, only about 5 percent of persons asked the question

refused and just less than 2 percent of persons said they did not know the answer. Nonresponse to this

question accounts for about 36 percent of all persons who began the screening interview but did not

complete it. Persons who did not answer the first two questions account for 92 percent of the persons who

were asked the first question but did not complete the screening interview. 

In an attempt to decrease nonresponse to the income question, one week into the pretest survey we

added an interviewer probe to the income question. If a respondent either answered “don’t know” or

refused to answer the income question, the interviewer said to the respondent “We do not need to know

your income, we only need to know if your income is below a certain amount. Your response will

be kept confidential” and then repeated the income question to the respondent. This probe was

successful in reducing nonresponse to the question. Of the 444 times the probe was used, the respondent

answered the income question after the probe (after initially refusing to answer the question) 77 times--a

rate of converting refusals of 17 percent.

As expected, the questions about assets also proved to be sensitive and difficult. The RDD screening

interview included four questions about respondents’ assets: (1) whether the respondent’s liquid assets
The CATI system automatically selected the appropriate threshold based on whether the household

12

contained an elderly person.

26

(cash and resources in checking and savings accounts) exceeded the asset threshold for FSP eligibility,12

(2) whether the respondent had any other assets, (3) whether the amount of these other assets exceeded

the asset threshold for FSP eligibility, and (4) whether the liquid assets and other assets together exceeded

the asset threshold for FSP eligibility. 

Of the persons asked the first asset question, over 5 percent refused to answer the question and over

2 percent did not know the answer. Of respondents asked this question, a higher proportion of persons

refused to answer it than refused to answer the income questions. The nonresponse to the second asset

question was very low. The third asset question--whether the amount of other assets exceeded the asset

threshold for FSP eligibility--caused some difficulties. While no-one in the pretest refused to answer the

question, nearly 5 percent of those asked did not know the answer to the question. This reflects the

difficulty of the question. All the persons asked the fourth question responded to the question. 

Despite relatively high nonresponse rates to the asset questions, only 4.5 percent of all people who

broke off during the screening interview did so because of these questions. This is because the questions

were asked later in the interview when many respondents had either already been found ineligible for the

survey or had refused to answer a prior question.

The question about whether the respondent last applied for food stamps within the previous three

years, asked of participants, also proved somewhat problematic. Although less than 1 percent of persons

asked this question refused to answer it, nearly 3 percent could not answer it. This is a difficult question

for respondents to answer--they need to think back over a period of several years; and while for program

administrators an “application” is a well-defined term, it may be a confusing term to food stamp recipients.
27

The date the respondent last applied for food stamps can be easily confused with the date the respondent

first applied for food stamps or the date the respondent last recertified. 

The other questionsin the screening interview had low rates of item nonresponse. Respondents did

not have any difficulties answering questions about whether they received food stamps, whether their

household included a working or an elderly person, whether they owned a vehicle, or whether the vehicle

was manufactured in the previous five years. 

c. Overall Response Rates

The overall response rate to the screening interview was just under 60 percent. Following the

recommendation of the Council of American Survey Research Organizations (CASRO 1982), the response

rate was calculated as the number of completed screening interviews as a percent of the number of

residential units, both known and unknown. The number of unknown residential units was estimated by

assuming that the proportion of residential telephone numbers among telephone numbers in which the

residential status was unknown is the same as the proportion of residential telephone numbers among

telephone number for which the residentialstatus was known. Using this estimate, the overall response rate

is equal to the product of the completion rate for determining residential status and the completion rate for

determining eligibility for the main questionnaire.

Response rates for RDD interviews are typically low. In a review of 39 RDD surveys, Massey,

O’Connor, and Krotki (1997) found that the average response rate was 62 percent. About one-half of the

surveys had response rates between 60 and 70 percent. One-third of the surveys had response rates

below 60 percent. Only about one-sixth of the surveys had response rates above 70 percent. The

response rate to the pretest screening interview is in the same range as found in previous RDD surveys,

even though it asks difficult and sensitive questions. 
28

Both the completion and response rates varied by site, as illustrated in Table II.3. The response rate

varied from just under 50 percent in Boston to just over 70 percent in the rural county in Minnesota. Both

the completion rate for determining residential status and the completion rate to the interview were lowest

in the big city sites of Boston and Philadelphia, and highest in the rural site in Minnesota. Low response

rates in large metropolitan areas, especially in the Northeast of the United States, have been found in other

studies (Cohen and Carlson 1992 and Kristal et al. 1993). If we exclude respondents from Boston and

Philadelphia from our sample, the response rate to the pretest increases nearly three percentage points to

over 62 percent.

2. Resource Requirements for Identifying Respondents Using RDD

An important factor in designing any survey is its cost. Because households that are eligible for food

stamps are not common, an RDD survey requires many phone calls to identify FSP-eligible households.

The focus on working and elderly households adds to the difficulty of finding respondents eligible for the

survey. This section begins with a discussion of the rate at which we found respondents eligible for the main

questionnaires, and then discusses the reasons we found respondentsineligible for the main questionnaires.

We conclude the section with a discussion of the amount of time spent by interviewers on RDD screening.

a. Rate at Which We Found Respondents Eligible for the Survey

The most important determinant of the cost of the RDD screening interview is the number of calls that

need to be made to identify the target number of respondents. One ofthe main determinants of this is the

eligibility rate--the number of respondents that we find are eligible for the survey as a percent of the

number of respondents who completed a screening interview. The eligibility rates are presented in Table

II.4, separately by site and by working nonparticipants, elderly
29

TABLE II.3

COMPLETION AND RESPONSE RATES TO THE 

RDD SCREENING INTERVIEW, BY SITE

Site Residential Status Questionnaires Interview

Completion Rate for Determining to RDD

Identifying Eligibility for Main Screening

a

Completion Rate for Response Rate

b c

Urban

Suffolk, MA (Boston) 73.6 67.5 49.7

Galveston, TX 84.4 71.8 60.6

Adams, CO (Denver) 85.8 71.6 61.4

Philadelphia, PA 80.5 67.8 54.6

Ramsey, MN (Minneapolis) 84.4 74.8 63.1

Durham, NC 85.2 71.2 60.6

Rural

Bedford, PA 89.4 74.0 66.2

Murray, MN 90.9 77.6 70.6

All 83.3 71.4 59.5

NOTES:

The number of telephone numbers for which residential status is determined as a percentage of the number

a

of telephone numbers dialed.

The number of respondents who completed the screening interview as a percentage of the number of

b

respondents eligible for screening.

The number of respondents who completed the screening interview as a percentage of the number of

c

residential (both known and unknown) telephone numbers called.
30

TABLE II.4

RATE AT WHICH RESPONDENTS TO THE RDD SCREENING INTERVIEW

WERE FOUND TO BE ELIGIBLE FOR THE MAIN QUESTIONNAIRES, BY SITE

Nonparticipants Participants

Site Poverty Elderly Working Elderly All Working Elderly All Total

Percent in Percent

a a

Urban

Suffolk, MA (Boston) 18 16 3.6 2.3 5.8 0.5 0.5 0.9 6.7

Galveston, TX 16 15 7.1 3.5 9.4 0.8 0.8 1.3 10.7

Adams, CO (Denver) 10 11 5.6 2.3 7.1 0.6 0.4 0.9 8.1

Philadelphia, PA 20 20 10.1 4.5 12.9 2.2 1.7 3.7 16.6

Ramsey, MN (Minneapolis) 11 16 5.5 4.2 8.6 0.8 0.8 1.3 9.9

Durham, NC 12 14 4.2 2.8 6.0 0.6 0.8 1.3 7.3

Rural

Bedford, PA 14 21 6.4 4.8 10.0 0.4 0 0.4 10.4

Murray, MN 14 26 5.5 4.2 8.6 0.8 0.8 1.2 9.9

All 13 17 5.5 3.1 7.9 0.9 0.8 1.5 9.4 b b

NOTES:

The rate for all nonparticipants/participants is not equal to the sum of the rates for the working and elderly because some households contain both a

a

working and an elderly person.

 The national rate for the U.S. b
If we found that a household contained an elderly and a working person, we only administered one

13

questionnaire to the household. The CATI system determined randomly whether the respondent was

administered a working or an elderly questionnaire.

The sum of these two percentages exceeds 9.4 percent because a respondent could be eligible for

14

both a working and an elderly questionnaire.

31

nonparticipants, working participants, and elderly participants. As some households (about 12.5 percent)

contain both working and elderly persons, the sum of the eligibility rates for working nonparticipants (or

participants) and elderly nonparticipants (or participants) exceeds the overall eligibility rate for identifying

respondents in either working or elderly households. 

13

Overall, we found 576 persons eligible for one of the main questionnaires out of 6,155 persons who

completed the screening interview--an eligibility rate of 9.4 percent. Of the persons who completed the

screening interview, we found 6.6 percent who met our criteria for the working questionnaires and 3.9

percent who met our criteria for the elderly questionnaires. 

14

Estimates using survey data from the 1994 Survey of Income and Program Participation (SIPP)

suggest that just over 5 percent of U.S. households are eligible for the FSP and contain a person who

works, and just less than 5 percent of U.S. households are eligible for the FSP and contain an elderly

person (Stavrianos 1997). Our findings on the prevalence of FSP-eligible working and elderly households

are similar.

We were surprised by the relatively high number of nonparticipants we found for each participant.

For every 100 people who completed the screening interview, we identified 7.9 FSP-eligible

nonparticipants (in either working or elderly households) and 1.5 participants (in either working or elderly

households). We found more than five nonparticipants for every participant, about six times as many

working nonparticipants for every one working participant, and about four elderly nonparticipants for every
32

one elderly participant. Stavrianos (1997) estimated that in 1994 there was just over one working

nonparticipant for every one working participant and just under two elderly nonparticipants for every one

elderly participant. 

There are four possible explanationsfor the discrepancy between our findings and the findings in

Stavrianos (1997). First, we screened out any participant who said that they last applied for food stamps

more than three years ago. As we show later, our findings suggest that without this screen we would

increase the number of participants that we identified by about 82 percent.

 Second, the earlier study used data from 1994 while our findings are based on information collected

at the beginning of 1998. Since welfare reform there has been a decrease in participation in many welfare

programs, including the FSP, that cannot be explained by the decrease in poverty rates (which are usually

good measures of FSP eligibility rates). This would suggest a decrease in the FSP participation rate. 

Third, respondents may have stated in the screening interview that they do not receive food stamps

when in fact they do. FSP participation is typically underreported in survey data. Comparisons of FSP

operations data and 1992 SIPP data suggest that FSP participation is underreported by about 22 percent

in the SIPP (Trippe and Sykes 1994). For this reason, Stavrianos used estimates of the number of FSP

participants from FSP operations data rather than the SIPP. If this explanation is true, this would cause

some concern because it suggests that some of the persons we believe are nonparticipants are in fact

participants. We do not think this is a serious concern, as no respondent to a main nonparticipant

questionnaire later revealed that they were in fact a participant when asked about the reasons for their

nonparticipation.
This explanation is also consistent with the underreporting of FSP participation found in the SIPP.

15

33

Fourth, it is possible that the response rate to an RDD screening interview is lower for participants than

for nonparticipants. As most persons who did not complete the interview had broken off by the time we

15

asked about participation, we cannot tell whether the response rate differed between participants and

nonparticipants. (Although we do know the completion rate to the main questionnaire was higher for

participants.) If the response rates do differ between participants and nonparticipants, this would be a

concern as it suggests differences in the rate of response that depend on factors related to the decision to

participate. 

The eligibility rate varied by site from less than 7 percent in Boston to about 17 percent in Philadelphia.

The differences in the eligibility rate reflect differences in poverty rates and the percent elderly in the survey

sites. Sites with high poverty rates and/or large elderly populations, such as Philadelphia, Bedford, and

Murray, have high eligibility rates. Sites with low poverty rates and smaller elderly populations, such as

Ramsey (Minneapolis), have lower eligibility rates. However, the poverty rates and the percent of the

population who are elderly do not explain all the differences--we found higher rates of eligibility in

Galveston than in Boston, despite Boston having a high poverty rate and a higher prevalence of elderly

persons than Galveston. 

It is important to remember that the sites were selected only from counties which had poverty rates

of 10 percent or higher, and Philadelphia and Murray were selected only from counties which also had 20

percent of more of the population over age 60. In nationally-representative sites, we would expect the

eligibility rates to be lower. Based on an average of the eligibility rates in the four sites with lower rates of
These sites are Galveston (Texas), Adams (Colorado), Ramsey (Minnesota), and Durham (North

16

Carolina).

34

poverty and less elderly populations, wewould expect the eligibility rate for a nationally-representative

16

sample to be around 9 percent.

b. Reasons Respondents Were Found Ineligible for the Survey

The reasons respondents were found ineligible for the survey are presented in Table II.5. The reasons

are listed in the order they are used in the screening interview to screen out respondents. All respondents

are first asked about the size of their household and whether their household income

exceeds 130 percent of poverty. Respondents who report household income below that level and that they

do not receive food stamps need to satisfy six additional screening criteria to be eligible for the main

questionnaires:

1. No-one in the household can have been informed that he or she was ineligible for food stamps

in either the current or previous month

2. The household must contain either a working or elderly person

3. The household must not own a vehicle that was manufactured in the past five years

4. The household’s cash and assets in checking and savings accounts must not exceed the FSP

eligibility threshold

5. Other household assets must not exceed the FSP eligibility threshold

6. The value of all household assets must not exceed the FSP eligibility threshold 

The last four screening criteria are not applied if everyone in the respondent’s household receives TANF,

GA, SSI, or other welfare benefits because the household is considered categorically eligible for food

stamps. Two additional screens are applied to participants: (1) they must have applied within the last three
35

TABLE II.5

REASONS RESPONDENTS TO THE RDD SCREENING INTERVIEW WERE FOUND INELIGIBLE 

FOR THE MAIN QUESTIONNAIRES

Number of Respondents Each Reason as Percent of: a

Number of Respondents Found Ineligible For

Reason Answered the Ineligible Question

Found Ineligible All Respondents All Respondents

for Reason Who Who Were Found Who Answered the

Questions 

Income exceeds 130% of poverty 4,716 6,155 84.5 76.6

Nonparticipants 

Had recently been informed that they were

ineligible for food stamps 32 1,095 0.6 2.9

Neither a working nor an elderly person in the

household 207 1,063 3.7 20.2

Owned vehicle less than 5 years old 194 856 3.5 22.7 b

Cash and assets in checking and savings accounts

exceeded limit 142 662 2.5 21.5 b

Other nonliquid assets exceeded limit 29 520 0.5 5.6 b

Total assets exceeded limit 7 491 0.1 1.4 b

Passed all screens - 484 - -
TABLE II.5 (Continued)

Number of Respondents Each Reason as Percent of: a

Number of Respondents Found Ineligible For

Reason Answered the Ineligible Question

Found Ineligible All Respondents All Respondents

for Reason Who Who Were Found Who Answered the

Questions 

36

Participants

Last applied more than three years previously 155 344 2.8 45.1

Inconsistent response to question about when they

last received food stamps 9 189 0.2 4.8 c

Neither a working nor an elderly person in the

household 88 180 1.6 48.9

Passed all screens - 92 - 92

Total Number Of Respondents Who Completed a

Screening Interview - 6,155 - 6,155

Total Number of Respondents Found Ineligible 5,579 - 5,579 -

NOTES:

 Only includes respondents who completed the screening interview. a

 This question is skipped if the respondent reports that everyone in the household receives TANF, GA, or SSI. b

Persons responded that they had received food stamps this month or last to question s2 but then responded that they had received last food stamps c 

more then one month ago when asked when they had last received food stamps in question s19. 
37

years, and (2) the household must contain either a working or an elderly person. As soon as a respondent

indicates that they are not eligible by not “passing a screen” the interviewer ends the interview by thanking

the respondent for his or her time. 

For most respondents, a household income above the FSP eligibility threshold was the reason they

were found ineligible for the main questionnaire. About 85 percent of all ineligible respondents were

ineligible because their income was too high, and 77 percent of respondents who were asked about their

income reported that their income exceeded 130 percent of poverty. 

The question about whether the household contained a working or elderly person screened out about

20 percent of the nonparticipant households and nearly 49 percent of the participant households.

The asset questions also screened out quite a large number of nonparticipant households. If we had

not asked the asset questions, the main questionnaires would have been administered to 856

nonparticipants, nearly twice the number of nonparticipants (484) who actually passed all the screens.

Many nonparticipants whowere in fact ineligible would have been administered the main questionnaires

if we had not included the asset screens. Table II.6 presents the number of nonparticipants who did not

pass the asset screens, broken down by whether the respondent is working or elderly. Although the

differences are not large, respondents from working households were more likely than respondents from

elderly households to fail the vehicle test and respondents fromelderly households were more likely than

the ones from working households to have cash or other assets that exceeded the threshold.

About 45 percent of participants said that they had not applied for food stamps in the previous three

years and so were screened out. If this question had not been used as a screen, we estimate that we would

have identified 167 participants for the main questionnaire, about 82 percent more than
38

TABLE II.6

FAILURE TO PASS ASSET SCREENS IN RDD INTERVIEW, BY TYPE OF RESPONDENT

Working Elderly a b

Number of Percent of Number of Respondents Percent of

Respondents Respondents Respondents

Reason for Reason Screens for Reason for Reason Screens for Reason

Who Were Who Who Passed Who Were Who Who Passed

Found Passed Previous Screens Found Passed Previous Screens

Ineligible Previous Found Ineligible Ineligible Previous Found Ineligible

Owned vehicle less than five

years old 157 635 24.7 65 327 19.9

Cash and assets in checking

and savings accounts exceeded

limit 96 478 20.1 58 262 22.1

Other nonliquid assets

exceeded limit 21 382 5.5 12 204 5.9

Total assets exceeded limit 6 361 1.7 1 192 0.5

NOTES:

 Respondents in a household that contains a working person. The household may also contain an elderly person. a

 Respondents in a household that contains an elderly person. The household may also contain a working person. b
The estimates were made using electronic records of the length of time interviewers spent logged on

17

to the RDD CATI program, electronic records of the length of time interviewers spent logged on to the listframe CATI program, interviewer-recorded time spent administering the main questionnaires, and records

of hours worked from the interviewers’ time-sheets. 

39

the 92 that we actually identified. We used this question as a screen because the survey asks participants

about their experiences applying for food stamps and we did not want to include participants who last

applied many years ago. Despite the relatively high nonresponse to this question (see Table II.3), the

proportion of pretest participantsscreened out because they applied within three years (55 percent) is the

same as the proportion of FSP participants that applied within the last three years in a cross-sectional

sample of FSP participants in the 1991 SIPP (Gleason, Schochet, and Moffitt 1998).

Another screen for the participants merits some discussion. We asked persons who said they received

food stamps, when they last received food stamps and the amount they received. Nine respondents said

they last received food stamps more than two months previously. As this directly contradicts the response

that these persons gave earlier in the screening interview that they had received food stamps in the current

or previous month, we treated these respondents as ineligible for the survey.

c. Interviewer Time Spent on RDD Screening

The most important determinant of the cost of a survey is the number of hours spent by the telephone

interviewers on the screening. The amount of time spent by interviewers on RDD screening during the

pretest is shown in Table II.7. Time spent screening includes all time spent by telephone interviewers

conducting the screening. This includes time spent logging on to the CATI system, calling numbers,

17

determining whether the number belongs to a residence, talking with supervisors, and time between calls.

Table II.7 also presents the average number of times each telephone number was called during the pretest.
Not all of the interviewers worked throughout the pretest. Asis typical, there was some attrition of

18

the interviewers and in later stages of the pretest we used only the more productive interviewers.

40

A total of 46 telephone interviewers spent 3,347 hours identifying the 576 persons eligible for the

18

main questionnaires--about 5.8 hours for each eligible respondent. Each telephone number was called

between 1 and 40 times. On average, each number was called 5.4 times. The interviewers spent an

average of 12 minutes per telephone number on screening. 

Most of the interviewers’ time was spent not on conducting interviews, but on dialing numbers,

determining residentialstatus, and attempting to persuade the respondents to complete the interview. The

RDD screening interview took on average less than four minutes.

The amount of interview time required to identify respondents using RDD is driven by the amount of

time it takes to identify the type of respondent that is least prevalent in the population. The additional cost

of identifying other types of respondents that are more prevalent in the population is negligible because

types of respondents with a higher prevalence are identified in the course of screening for the target

population with the lowest prevalence. For example, because elderly nonparticipants are less prevalent than

working nonparticipants, the amount of time required to identify 1,000 elderly nonparticipants and 1,000

working nonparticipants will not be much more than the amount of time it takes to find 1,000 elderly

nonparticipants and no working nonparticipants. 

Table II.8 illustrates how many telephone numbers would need to be called to identify 100

respondents of each type and how much interviewer time it would take to screen each eligible respondent.

These estimates are based on the assumptions that the response rate for each type of
41

TABLE II.7

NUMBERS CALLED, TIMES EACH NUMBER DIALED, AND TIME SPENT ON 

INTERVIEWING: RDD SCREENING

Telephone numbers called 16,648

Average times number was dialed 5.4

Total time spent on screening (hours) 3,347 a

Average time spent screening per number called (minutes) 12

Average time spent screening per eligible respondent (hours) 5.8

NOTE:

Includes all time spent by telephone interviewers on RDDscreening. This includes time spent logging into

a

the CATI system, calling numbers, determining residential status, talking with supervisors, and time between

calls.
42

TABLE II.8

ESTIMATES OF RESOURCES REQUIRED TO IDENTIFY SURVEY 

RESPONDENTS USING RDD

With Pretest Eligibility Rate Sites a

With Eligibility Rate Expected

With Nationally-Representative

b

Respondent Group Respondents (Hours) Respondents (Hours)

Numbers Average Time Numbers Average Time

Called to Spent Screening Called to Spent Screening

Identify 100 Per Eligible Identify 100 Per Eligible

Eligible Respondent Eligible Respondent

Working nonparticipants 4,690 9.4 4,896 9.8

Elderly nonparticipants 8,716 17.5 9,100 18.3

Working participants 30,830 61.9 32,187 64.7

Elderly participants 34,683 69.7 36,209 72.8

NOTES:

 In the pretest we found that 9.4 percent of respondents who completed the RDD screening interview were

a

eligible for the survey.

We expect that with nationally-representative sites, only about 9.0 percent of respondents who complete b

the RDD screening interview would be eligible for the survey. This increases the number of calls that need

to be made by 4.4 percent (9.4/9.0 = 1.044).
In the pretest, the eligibility rate was 9.4 percent. We estimate that in nationally-representative sites

19

the eligibility rate would be 9.0 percent. 

43

respondent is the same, and the amount of time spent calling each telephone number is the same as it was

in the pretest. The estimates presented in the second and third columns are based on the rate at which we

found persons eligible for the survey in the pretest. However, a survey to obtain a nationally-representative

sample could not be limited, as the pretest was, to sites that do not have low poverty rates or small elderly

populations. So we present in the fourth and fifth columns the number of calls needed and the time it would

take to make the calls based on an estimate of the eligibility rate in nationally-representative sites. We

estimate that wewould need to call 4.4 percent more telephone numbers in nationally-representative sites

than we did in the pretest to identify the same number of respondents.19

Because of the nature of an RDD survey, the amount of time required to identify one type of

respondent is higher thanthe cost of identifying either a working nonparticipant or an elderly nonparticipant

or a working participant or an elderly participant as we did in the pretest (see Table II.7). We estimate

that it would take an average of nearly 10 hours of interviewer time to identify one working nonparticipant

and over 18 hours of interviewer time to identify one elderly nonparticipant in a nationally-representative

survey. To identify 1,000 working nonparticipants and 1,000 elderly nonparticipants for a national survey

would require interviewers to spend about the same time as it would take to identify 1,000 elderly

nonparticipants--18,300 hours. 

Participants from either working or elderly households are difficult to identify using RDD. Table II.8

shows that it takes an enormous number of hours to identify a participant from either a working or an

elderly household. Hence, it would not be efficient to identify participants using RDD unless it was in the

course of identifying nonparticipants. Based on our pretest findings, we estimate that we would identify 15
We also have no information on whether the screening interviewscreened out respondents who were

20

FSP-eligible.

The long questionnaires did not ask about the amount of the household’s financial assets. These

21

questions were viewed as too difficult and sensitive.

Four of the respondents in the third group were administered a working questionnaire because their

22

households contained a working and an elderly person.

44

working participants in the course of identifying 100 working nonparticipants (a ratio of participants to

nonparticipants of 1 to 6.6) and we would identify 25 elderly participants in the course ofidentifying 100

elderly nonparticipants (a ratio of participants to nonparticipants of 1 to 4.0).

3. Further Eligibility Tests for Respondents Who Pass the RDD Screening Interview

An important purpose of the RDD screening interview was to identify respondents who were likely

to be eligible for food stamps. How well did it do? We cannot determine for sure whether respondents

who passed the tests in the screening interview are FSP-eligible. To do so would require a respondent to

submit to the full FSP application process. However, the long nonparticipant questionnaires did ask for

20

detailed information about income, expenses, vehicles, and U.S. citizenship, information that can be used

to make a more accurate determination of eligibility than was made in the screening interview.21

To check the FSP-eligibility of nonparticipants in our sample, we divided the 165 respondents who

were administered long nonparticipant questionnaires intothree groups: (1) respondents in households with

no elderly or disabled persons, (2) respondents in households with disabled persons and no elderly

persons, and (3) respondents in households with elderly persons. The FSP eligibility rules differ slightly

22

for each of these groups.

Using the data from the long questionnaires, we applied four FSP-eligibility tests to the nonparticipant

respondents.
45

C Was their total household income less than 130 percent of poverty? This test applies

only to respondents from nonelderly nondisabled households. This test was also applied in

the screening interview. However, in the main questionnaire, respondents are asked to

provide data on their income by source, with the interviewer naming each potential source.

This is likely to lead to less underreporting of income than when respondents are just asked

about their total household income, without any probing about sources.

C Wastheir net household income less than 100 percent of poverty? This net income test

is applied to all households that apply for food stamps. Net income is total household income

minus six deductions: (1) a standard deduction of $134, (2) an earnings deduction of 20

percent of earned income, (3) out-of-pocket costs of dependent care up to a maximum per

month, (4) medical expenses of elderly or disabled persons that exceed $35 per person, (5)

legally-owed child support payments made by a noncustodial parent of a child living outside

the household, and (6) shelter costs in excess of 50 percent of remaining gross income after

applying all other deductions, subject to a cap of $250 for nonelderly nondisabled households.

C Was the countable value of household vehicles greater than the asset limit? The FSP

eligibility criterion is that countable household assets must not exceed $2,000, unless the

household is elderly and then they must not exceed $3,000. The value of vehicles is often a

large component of the assets oflow-income households (McConnell 1997). The FSP counts

assets in a household’s first vehicle and vehicles used to commute to work as the fair market

value of the vehicle in excess of $4,650. They count assets in other vehicles as either the fair

market value in excess of $4,650 or the equity in the vehicle, whichever is larger. Using the

pretest data we check whether the total value of a household’s vehicles minus $4,650 exceeds

the appropriate FSP asset limit.

C Is there any U.S. citizen in the household? With some exceptions, only U.S.citizens are

eligible for food stamps. Using our pretest data, we check whether everyone in the household

is a U.S. citizen.

The numbers of nonparticipant respondents to the long questionnaires who do not pass each of these

tests are presented in Table II.9. Out of 165 respondents, 63 or about 38 percent failed one or more of

the four tests using the data in the main questionnaires. So, a significant proportion of the nonparticipants

in our sample may not be eligible for food stamps.
46

TABLE II.9

NUMBER OF NONPARTICIPANT RESPONDENTS WHO PASSED THE RDD SCREENING INTERVIEW BUT SEEM TO BE

INELIGIBLE BASED ON INFORMATION GIVEN IN THE MAIN QUESTIONNAIRES

Failed Eligibility Test Households Households Households

Nonelderly Nondisabled Nonelderly Disabled Elderly All Households

Gross income test 36 -- -- 36 a a

Net income test 24 4 17 45

Value of the household vehicles minus $4,650 5

exceeds countable asset threshold 3 -- 2

b

No U.S. citizen in household 5 1 0 6

Failed at least one of the eligibility tests 41 5 17 63 c

Sample size 73 13 79 165

NOTES:

 Elderly and disabled households are not subject to the gross income eligibility test. a

 Vehicles used to transport disabled persons are not countable assets. Hence, we did not subject disabled households to the vehicle test. b

This is not the column total because households may fail more than one test. c 
In Chapter III, we recommend modifying the questions about earnings in the main questionnaires. 23

47

We found that disabled and elderly households were less likely to be found ineligible with the additional

information than householdswith earnings. We found that out of 73 respondents from households without

elderly or disabled persons 36 (49 percent) failed the grossincome test based on income reported later

in the main questionnaire. Of these, 30 respondents failed because their earnings exceeded 130 percent

of poverty. These respondents reported gross household income less than 130 percent of poverty at the

beginning of the interview but then later in the interview reported a wage rate and hours worked inconsistent

with that low a household income. 

It may be that the earnings reported in the main questionnaires do not reflect the household earnings

for that month. The respondent was asked for the “usual” number of weekly hours worked and the “usual”

hourly wage rate for workers in the household. But it could be that the workers in the household had not

worked the “usual” number of hours that month. We also found that 19 percent of participants, who

presumably are FSP-eligible, reported earnings and other income in the main questionnaire that exceeded

130 percent of poverty. This would be consistent with this explanation for the discrepancy. 

23

Another explanation for the discrepancy is that respondents either misreported their aggregate

household income in the screening interview or missreported their earnings in the main questionnaires.

Previous studies have found that income is more likely to be underreported when the question asks for

aggregate income rather than income by source (Citro and Michael 1995). 

The net income test failed the most respondents, mainly because this test was applicable to all

respondents. We did not include questions to calculate any FSP deductions in the screening interview

because to apply the net income test would require too many detailed and sensitive questions to include

in a screening interview. Few respondents had countable vehicle assets more than the threshold, suggesting
Some states in which the FSP is county-administered may keep their data at the county level. 24

48

that the cutoff for the age of the vehicles used in the screening interview worked well. Few respondents

were in households that contained no U.S. citizens. This may, however, differ if the survey was

administered in other areas of the country, such as California or New York.

B. LIST-FRAME SCREENING

While lists of persons who are eligible for food stamps but choose not to receive them are not

available, the Food Stamp Program does maintain electronic lists of FSP participants. These lists can be

used as a sample frame for a survey. For the survey pretest, we used both the program lists and RDD to

identify working and elderly participants for the survey.

We asked the six states participating in the study to provide lists of names, addresses, and telephone

numbers of current FSP participants who had last applied forfood stamps within the previous three years

and who were in households with earnings or households that contained an elderly person. All states keep

some electronic data of current FSP participants as part of their administrative systems for issuing benefits.

Most states keep these data at the state level and can send the data on computer tape or disk. We found

24

that states could send us lists of FSP participants with an indicator for which households were elderly.

However, some states did not have data readily available on whether the household had earnings and so

sent us lists of participants who were in elderly households and lists of participants in nonelderly households.

All the states had difficulty providing us with the date the participant last applied. Hence, our sample frame

included some households that did not contain either a working or elderly person and FSP participants

whose most recent application for food stamps was more than three years ago.
49

The states provided us with data on the names of the participants, their addresses, telephone numbers

(when they were available), and the names of their caseworkers. After sampling participants from these

lists, we sent lists of the names of the participants to the local FSP offices for the caseworkers to check

whether we had the most recent telephone number and address. Once these were checked, we then

mailed each FSP participant with a working telephone a letter describing the study and informing the

participant that we would be calling them shortly to interview them.

Obtaining the data from the state agencies and the corrected telephone numbers from the local offices

took considerable time. We approached the state agencies to request the data in August 1997, requesting

data on persons who were receiving food stamps in September or October. We received the first set of

data in November. However, we had still not received all the data in January 1998 when the RDD survey

began. The local offices needed two or three weeks to provide us with updated lists of telephone numbers.

These delays meant we could not begin the list-frame survey until mid-February. Hence, even at the

beginning of the pretest of the list-frame screening, our lists were four or five months out of date. By April,

at the end of the pretest, the data were six or seven months out of date. 

 This section discusses our experience in using the list-frame to identify FSP participants for the survey

pretest. We compare it with the RDD survey in terms of both response rates and interviewer time spent

on the survey.
50

1. Response Rates

The response rate to the list-frame screening interview was similar to the response rate to the RDD

screening interview. Table II.10 summarizes the number of completes at each stage of the screening. We

dialed 253 different telephone numbers and were able to locate the participant and complete an interview

for 151 of these numbers--a response rate of 60 percent. 

The completion and response rates to the list-frame screening interview for each site are presented in Table

II.11. As for the RDD screening interview, we found the response rates were highest in the rural sites and

lowest in the urban sites.

a. Locating Respondents

The main reason for nonresponse was the difficulty in locating respondents. The respondent was not

reached at the listed telephone number in 79 of the 253 cases (31 percent). This was mostly because when

we called the number the telephone had been disconnected or reassigned to a person who was not the

respondent. We called directory assistance to find numbers, but were successful in only a few cases. 

b. Determining Eligibility for the Survey

The completion rate for the screening interview was high--nearly 87 percent--much higher than the

completion rate for the RDD screening interview (71 percent). We attribute this high completion rate to

three factors. First, participants are more likely to be interested in a survey about the FSP than

nonparticipants. Second, an advance letter was sent to each respondent before the interview. Surveys that

use advance letters are generally able to achieve higher response rates than those who do not (Brunner and

Carroll 1969; Dillman, Gallegos, and Frey 1976; and Traugott,
51

TABLE II.10

COMPLETION AND RESPONSE RATES TO THE 

LIST-FRAME SCREENING INTERVIEW

Total Numbers Dialed 253

Respondents Located 174

Respondents Not Located 79 a

Rate of Locating Respondentsb 68.8%

Respondents Located 174

Eligibility Determined 151

Eligible for main questionnaire 86

Working participants 38

Elderly participants 48

Ineligible for main questionnaire 65

Eligibility Unknown 23

Hung-up during introduction 0

Refused after introduction 5

Refused during screening interview 3

Language or disability prevented completion of interview 15

Rate of Completion of Eligibility Screening Interviewc 86.8%

Rate of Eligibility for Main Questionnairesd 57.0%

Response Ratee 59.7%

NOTES:

 Could not locate person on the list frame. a

 The number of respondents located as a percentage of the total number of telephone numbers dialed. b

 The number of respondents who completed the screening interview as a percentage of the number of

c

respondents located.

The number of respondents found eligible for the main questionnaires as a percentage of the number of d

respondents who completed the screening interview.

 The number of respondents who completed the screening interview as a percentage of the number of

e

telephone numbers dialed.
52

TABLE II.11

COMPLETION AND RESPONSE RATES TO THE LIST-FRAME SCREENING INTERVIEW,

BY SITE

Site Respondent Screening Interview

Rate of Locating Rate of Completing Response Rate

a b

c

Urban

Suffolk, MA (Boston) 69.6% 75.0% 52.2%

Galveston, TX 53.4% 100.0% 53.4%

Adams, CO (Denver) 74.2% 82.6% 61.3%

Philadelphia, PA 76.9% 80.0% 61.5%

Ramsey, MN (Minneapolis) 75.0% 77.7% 58.3%

Durham, NC 57.1% 83.3% 47.6%

Rural

Bedford, PA 88.9% 87.5% 77.8%

Murray, MN 84.6% 95.5% 80.8%

All 68.8% 86.8% 59.7%

NOTES:

The number of respondents located as a percentage of the total number of telephone numbers dialed. a

The number of respondents who completed the screening interview as a percentage of the number of

b

respondents located.

The number of respondents who completed the screening interview as a percentage of the number of

c

telephone numbers dialed.
53

Groves, and Lepkowski 1987). This is because the advance letter authenticates that the study is legitimate

and provides other information that encourages participation in the survey. Third, the screening interview

did not ask any sensitive questions--only three respondents refused any of the questions during the

screening interview. Fifteen respondents could not complete the interview because of language, sickness,

or a disability.

2. Resource Requirements for Identifying Respondents Using a List Frame

The main advantage of using the list-frame was that it was a less costly way to identify FSP

participants. In this section, we identify the resources needed to identify FSP participants using the listframe. We begin by discussing the rate at which we found persons eligible for the main questionnaires.

a. Rate at Which We Found Respondents Eligible for the Main Questionnaires

The advantage of using a list-frame over an RDD sampling frame is that all persons on the list-frame

were FSP participants, at least at the time the frame was constructed. We could also ensure that the

participant belonged to an elderly household at the time the frame was constructed. An RDD frame,

however, includes all persons with telephones, including middle- and high-income households, persons in

households with no working or elderly persons, and persons who have never participated in any

government program. We found that 57 percent of persons who completed the list-frame screening

interview were found to be eligible for the main questionnaires, compared with less than 10 percent of

persons who completed the RDD screening interview.

The rates at which we found persons eligible for the main questionnaires are presented, by site, in

Table II.12. The table distinguishes between persons that were designated on the list-frame as
54

TABLE II.12

RATE AT WHICH RESPONDENTS TO THE LIST-FRAME SCREENING INTERVIEW ARE

FOUND TO BE ELIGIBLE FOR THE MAIN QUESTIONNAIRES, BY SITE

Specified by the Food Stamp Agency as:

Site Working Elderly Total

Urban

Suffolk, MA (Boston) 14% 60% 33%

Galveston, TX 57% 91% 67%

Adams, CO (Denver) 33% 86% 53%

Philadelphia, PA 25% 50% 38%

Ramsey, MN (Minneapolis) 50% 65% 61%

Durham, NC 0% 60% 30%

Rural

Bedford, PA 80% 50% 71%

Murray, MN 40% 82% 62%

All 45% 72% 57%
We treated both the persons designated as “working” and those designated as “elderly” identically.

25

In the screening interview, the interviewer checked that the respondent was from either a working or an

elderly household. A person who was designated as “working” would have been eligible for an elderly

main questionnaire if they reported that an elderly person lived in the household.

55

“working” from persons designated as “elderly.” The eligibility rate for the elderly households is much

25

higher than that for working households--72 percent for elderly households and 45 percent for working

households. This finding was expected for two reasons. First, the state FSP agencies identified elderly

households for us (at least at the time the frame was created), but not all state agencies were able to identify

working households. Second, the elderly do not move on and off food stamps as much as persons in

working households (Gleason et al. 1998). So the elderly are likely to still be on food stamps if they

received them several months ago. This is much less likely to be true for the working participants. The

eligibility rate for working participants and, to a lesser extent, elderly participants would almost certainly

be higher if the sample frame was more up-to-date.

b. Reasons Respondents Were Found Ineligible for the Main Questionnaires

Respondents to the list-frame screening interview were found ineligible for the mainquestionnaires for

three reasons (see Table II.13):

1. They were no longer receiving food stamps. This was the most common reason

respondents were found ineligible. Of 151 persons who completed the screening interview,

32 persons (21 percent) were not receiving food stamps. 

2. They had not applied within the last three years. As in the RDD screening interview, this

was an important reason why participants on the list-frame were found to be ineligible for the

main questionnaires. Of the 119 persons who completed the screening interview and were

still on food stamps, 26 persons (22 percent) reported that they had not applied within the

past three years.

3. The household contained neither an elderly nor a working person. Seven persons

reported that they did not live in a household with either a working or an elderly person.
56

TABLE II.13

REASONS RESPONDENTS TO THE LIST-FRAME SCREENING INTERVIEW WERE FOUND 

INELIGIBLE FOR THE MAIN QUESTIONNAIRES

Number of Respondents Each Reason as Percent of: a

Number of Respondents Found Ineligible For

Reason Answered the Ineligible Questions

Found Ineligible All Respondents All Respondents

for Reason Who Who Were Found Who Answered the

Questions

Did not receive food stamps this month or last 32 151 49.2% 21.2%

Last applied for food stamps more than three years

previously 26 119 40.0% 21.8%

Neither a working nor an elderly person in the

household 7 93 10.8% 7.5%

Total Number of Respondents Who Completed

Screening Interviews - 151 - 151

Total Number of Respondents Found Ineligible 65 - 65 -

NOTE:

 Only includes respondents who completed the screening interview. a
57

c. Interviewer Time Spent on List-Frame Screening Interviews

It was much quicker to identify each respondent for the main questionnaire using the list-frame than

it was to identify a respondent using RDD (see Table II.14). Interviewers spent an average of 31 minutes

to identify each respondent for the main questionnaires using the list frame, compared with 5.8 hours to

identify a respondent for the main questionnaires using RDD. Interviewers spent an average of 10 minutes

per telephone number called, similar to the time spent per RDD telephone number called. 

Our estimates of the time needed to identify each type of eligible survey respondent using the list-frame

are presented in Table II.15. To identify one working participant requires 38 minutes of interviewer labor.

Because the rate at which we found persons on the elderly lists was higher, it required less interviewer time

to identify an elderly participant--only about 24 minutes.

A survey about nonparticipation would always need to identify nonparticipants using RDD. Once an

RDD survey is being conducted to identify nonparticipants, is it cheaper to identify participants using RDD

or the list-frame? The answer is that it is cheaper to identify participants using RDD as long as it does not

increase the total number of RDD telephone numbers that need to be called. If the design allows all the

participants to be found as a by-product of identifying the nonparticipants, the additional fixed costs of

obtaining the lists can be avoided by not using a list-frame. However, if the design requires more

participants than could be identified as a by-product of identifying nonparticipants, then using a list-frame

to identify the “additional” participants would reduce survey costs. 

As an example, our findings suggest that to find 100 working nonparticipants, 4,690 RDD telephone

numbers would need to be dialed (Table II.8). While calling these numbers, we would
58

TABLE II.14

NUMBERS CALLED, TIMES EACH NUMBER DIALED, AND TIME SPENT ON 

INTERVIEWING: LIST-FRAME SCREENING

Telephone numbers called 253

Average times number was dialed 4.4

Total time spent on screening (hours) 44 a

Average time spent screening per number called (minutes) 10

Average time spent screening per eligible respondent (minutes) 31

NOTE:

Includes all time spent by telephone interviewers on list-frame screening. This includes time spent logging

a

into the CATI system, calling numbers, determining residential status, talking with supervisors, and time

between calls.
59

TABLE II.15

ESTIMATES OF RESOURCES REQUIRED TO IDENTIFY SURVEY 

RESPONDENTS USING A LIST-FRAME

Respondent Group Respondents (Minutes)

Numbers Called to Average Time Spent Screening

Identify 100 Eligible Per Eligible Respondent

Working participants 371 38

Elderly participants 233 24
60

find about 15 working participants. The cost of identifying these 15 participants is very small--it is just the

additional time it takes to ask two more questions on the interview. But the cost of finding the sixteenth

participant using RDD is huge, because it would require the interviewers to call another 308 telephone

numbers. 

C. SUMMARY AND IMPLICATIONS

We discuss below our conclusions and recommendations about identifying FSP nonparticipants and

participants for a survey about FSP nonparticipation based on our experiences in the pretest.

1. It Is Feasible To Identify FSP-Eligible Nonparticipants Using RDD

The pretest showed that it is feasible to identify FSP-eligible nonparticipants using RDD. However,

it does require considerable survey resources. Using RDD, we called nearly 17,000 telephone numbers

to identify 484 nonparticipants who were likely to be eligible for food stamps and were from either working

or elderly households. We estimate that it would take over 18,000 hours of interviewer labor to identify

a sample of about 1,000 nonparticipants from working households and 1,000 nonparticipants from elderly

households.

2. A List Frame is Needed if the Survey is to Include FSP Participants

If RDD isbeing used to identify nonparticipants, identifying participants at the same time requires little

additional interviewer time. However, identifying participants once the target for nonparticipants has been

reached would be extremely costly, many times more than identifying the participant using the list frame.

Our pretest findings suggest that we would find about one working participant for every six working

nonparticipants and one elderly participant for every four elderly nonparticipants. As most survey designs

would require a ratio of participants to nonparticipants of at least one participant for every three
Massey et al. (1997). 26

61

nonparticipants, if it is decided that the survey should include participants, a mixed-frame design would be

the most efficient one.

3. A Response Rate to the Screening Interview of 70 Percent Could be Achieved on the

Screening Interviews

Nonresponse is a potential concern because it could bias the survey findings. The response rate to

an RDD screening interview to identify nonparticipants is unlikely to be high for three reasons. First,

response rates to RDD surveys are typically low. Use of answering machines, call-forwarding, and

telephone solicitation all contribute to low response rates to RDD surveys. Obtaining a response rate

above 70 percent for an RDD survey is rare. Second, the screening interview includes questions about

26

household income and assets--questions that are both difficult and sensitive. Third, nonparticipants may

be uninterested in topics related to a program that they have chosen not to participate in. 

We found response rates of just under 60 percent on the RDD screening interview, in the same range

as found in other RDD surveys (Massey et al. 1997). A similar response rate was found for the list-frame

survey. With some changes in survey design and operations, the response rate to both screening interviews

could be 70 percent. These changes include modifications to the screening interview, sending an advance

letter to persons on the RDD lists, lengthening the field period, obtaining current lists of FSP participants

more quickly, and using commercial services to identify persons on the list-frame. These changes are

discussed in more detail below. 
62

4. We Recommend Modifying the RDD Screening Interview

When designing a screening interview, there is a fine line between developing an interviewthat makes

a good determination of FSP eligibility and one that asks so many detailed and sensitive questions that its

response rate is unacceptably low. We believe the RDD screening interview used in the pretest, with the

modifications described below, hits about the right balance between the two objectives of keeping the

interview short and simple and doing a good job of predicting FSP-eligibility.

We suggest the following four modifications to the RDD screening interview:

a. Change the Order of the Questions on the RDD Screening Interview

In the pretest, the first two questions of the RDD screening interview were about household size and

income. To the respondent, neither question would seem relevant to the topic of the survey and the income

question is both difficult and sensitive. Our rationale for placing these questions at the beginning of the

interview was that it would keep the interview extremely short for the people who were ineligible. While

we found that most people were ineligible and the interview was indeed short for these people, the “price”

of the reduction in the length of these interviews may have been a lower response rate. 

To increase the response rates, the first two questions of a survey should be easy, unobjectionable,

and related to the survey topic (Dillman 1978 and Frey 1989). We recommend that the screening

interview begin with the question about whether the respondent receives food stamps. The next questions

could be about whether the household contained a working or elderly person--again questions directly

related to the topic. The third question could be a question directly related to the respondents experiences

with food stamps. For example, we could ask nonparticipants whether they have ever received food

stamps. We could then follow these questions with the questions about receipt of TANF, GA, and SSI,
63

whether they have been found ineligible to receive food stamps, and household vehicles. Only after these

questions would we ask about income and assets. 

b. Modify the Income Question for Respondents in Working Households

We found that nearly half of the respondents in households with earnings who reported income of less

than 130 percent of the poverty threshold in the screening interview later reported earnings and other

income greater than 130 percent of the poverty threshold. We recommend that respondents in working

households are reminded by the interviewer to include all earnings from all adults in the household when

asked the income question. 

c. Insert An Interviewer Probe After Each Asset Question

The questions about assets are important screens. For example, the question about cash and other

assets in checking or savings accounts screens out 22 percent of persons who passed the preceding

screens. However, the rate of nonresponse to the asset questions was typically high. We recommend

keeping the asset questions, but adding probes for people who do not respond that reiterate why we need

this information, that we only need to know whether their assets are less than a certain amount, and that

the information will be confidential.

d. Probe Inconsistent Responses to the Questions About Receipt of Food Stamps

We asked respondents who said that they received food stamps when they last received food stamps.

If the respondent said they last received food stamps more than two months previously, contradicting an

earlier response, we treated the respondent as ineligible for the main survey. We recommend that an

interviewer probe is inserted after this question that asks respondents about the discrepancy in their
64

responses. If the respondent is in fact a nonparticipant, then the interviewer would circle back and ask the

respondent the screening questions to determine whether they are likely to be FSP-eligible.

5. We Recommend Sending Advance Letters to Persons on the RDD Sample Frame

Prior to calling the respondents in the list-frame, we sent them a short letter, explaining the study and

notifying the respondents that we would call them shortly. The lower rate of refusals before the first

question in the list-frame screening interview may be because of the advance letter. It is possible to also

send advance letters to some persons on the RDD frame. (To conserve resources, we did not send

advance letters to the persons on the RDD frame during the pretest.) Names and addresses are attached

to telephone numbers in the RDD sample frame only for those persons whose telephone numbers are listed

in the telephone directory--about 30 or 35 percent of the telephone numbers. MPR’s experience in

previous surveys is that 20 to 30 percent of advance letters sent are returned because the address is

incorrect. Hence, we would expect that between 20 and 30 percent of persons on the RDD frame could

receive an advance letter. 

6. We Recommend Lengthening the Field Period

In the pretest we found that the completion rate for determining whether the telephone number

belonged to a residence was lower than is typically found in RDD surveys. One explanation is that RDD

survey was conducted in only three months. MPR’s experience has been that the completion rate for

determining residency increases with the length of the field period. With a longer field period, more calls

can be made and with longer periods of time between calls.
65

7. We Recommend Increasing the Rate at Which Persons on the List-Frame are Located

We could not locate over 30 percent of the persons on the list frame. We recommend three changes

to increase the rate at which we located persons on the list frame:

1. Obtain Current Lists of FSP Participants as Quickly as Possible. The importance of

obtaining the data quickly needs to be emphasized to FSP office staff. The state FSP

agencies should be notified many months in advance of the intended request. The samples

should be created quickly and the lists of addresses and telephone numbers to be checked

should be sent to the caseworkers as soon as possible. In this way, the delay between when

the lists are created and when they are used can be reduced.

2. Use Commercial Services to Obtain More Locating Information. Commercial services

exist that can provide telephone numbers, changes of addresses, and telephone numbers for

neighbors. These can provide additional contact information for some persons. However,

our experience is that information from these sources will be available for only a small

proportion of the persons on the list-frame. This is because most of the information comes

fromcredit agencies and many low-income persons do not have established credit histories.

3. Conduct In-Person Follow Up. Those persons on the list-frame who cannot be located

by telephone may be located by an interviewer going in-person to the person’s address. Even

if the person is not at home, neighborsmay provide information about how the person could

be reached.
66

III. ADEQUACY OF THE MAIN QUESTIONNAIRES IN COLLECTING THE

INFORMATION THEY WERE DESIGNED TO COLLECT

Eight different questionnaires were administered during the pretest. A separate questionnaire was

administered to each of the four types of respondents--working nonparticipants, elderly nonparticipants,

working participants, and elderly participants. And we designed a long and a short version of the

questionnaire for each type of respondent. 

All thequestionnaires included questions on household composition, past experiences applying for and

using food stamps, food security, and some demographic characteristics of the respondent. The

nonparticipant questionnaires also include a section that asks in detail why the respondent does not

participate. The long versions of the questionnaires also include questions on receipt of food assistance

other than food stamps, employment, health (elderly only),social supports, income and expenses, and

questions about the respondent’s knowledge of the FSP. They also include more questions about food

security. The content of each type of questionnaire is summarized in Table III.1.

This chapter reports on how well these instruments collected the information they were designed to

collect. We address such issues as the adequacy of question wording, response categories, instructions

and probes, interview length, choice of respondent, and, more generally, identify ways in which the

instruments could be improved. Three sources of information were used to make our assessment: (1) the

frequency of item nonresponse and responses that do not fit any of the response codes; (2) respondent

debriefing questions administered at the end of each interview; and (3) the behavioral coding of 40

questionnaires.
67

TABLE III.1

CONTENT OF QUESTIONNAIRES

Working Poor Poor Elderly

Sections Long Version Short Version , , Long Version Short Version , Short Version

Participant, Participant, Nonparticipant Nonparticipant Participant, Participant, Nonparticipant Nonparticipant, 

Long Version Short Version Long Version

A: Household Compositiona T T T T T T T T

B: Knowledge of the FSP T T T T

C: FSP Participation History T T T T T T T T

D: History of FSP Applications T T T T T T T T

E: Reasons for Nonparticipation T T T T

F: Receipt of Other Food T T T T

 Assistance

G: Food Securityb T T T T T T T T

H: Employment Historyc T T T T

 I: Health T T

 J: Social Supports T T T T

 K: Income and Expenses T T T T

 L: Demographic Informationd T T T T T T T T

NOTES:

In the long version, the questionnaire obtains information on the age and relationship of everyone in the household; the short version only asks about the number of elderly persons, children, and a

working persons in the household. 

The long version contains a battery of questions on the food security of the household; only six questions on food security are included in the short version. b

All long versions ask about the wage rate and hours worked for each person in the household; the working long questionnaires also ask about the type of job worked and the work history of the c

person in the household who works the most hours. 

Only the long version contains questions about citizenship of household members. d
68
69

The remainder of this chapter is organized into five sections. Section A discusses completion rates to

the main questionnaires. Section B describesthe time required to administer the questionnaires. Section

C discusses the choice of respondent for the main questionnaires. Section D summarizes the performance

of the questionnaires at the question-level and recommends changes to some questions. (Appendix A lists

some other minor changes that should be made to the questionnaires.) Finally, Section E summarizes our

main findings and discusses their implications for the content and implementation of the survey on a larger

scale.

A. MAIN QUESTIONNAIRE COMPLETION RATES 

Completion rates are an important indicator of the feasibility of fielding the questionnaires nationally.

Related, the prevalence in which respondents “break-off” the interview once it begins and the point in the

questionnaire at which these break-offs occur are indicators of the performance of the questionnaires.

Break-offs early in the process of administering the questionnaire could be indicative of sensitive topics;

while break-offs occurring toward the end of the questionnaire could be evidence that the questionnaires

are too long. 

Details on completion rates and respondent break-offs by interview mode and type of questionnaire

are provided in this section. In order to place these findings in context, it is important to understand

differences in study procedures between telephone and in-person interviews. All pretest respondents--

regardless of whether the main questionnaire was to be administered by telephone or in-person--were first

administered a screening interview by telephone. If the respondent lived in an area selected for telephone

interviewing, once a household was determined eligible during the screening interview, the interviewer

would immediately begin administering a hard-copy main questionnaire. On the other hand, if the

respondentlived in an area selected for in-person interviewing, at the completion of the screening interview
Although screening interviews were completed with 662 respondents who were eligible for the main

27

questionnaires, we did not attempt to administer main questionnaires to 128 of these respondents. Of

these, 104 were not attempted because we had already met our target for that group and 24 because the

respondent only spoke Spanish. These 128 respondents are not included in our calculations of the main

questionnaire completion rate. 

70

the interviewer would tell eligible respondents that a field interviewer would be contacting themwithin one

week to schedule an in-person interview. 

Overall, 451 of 534 eligible pretest respondents, or 85 percent, completed a main questionnaire (Table

III.2). Combined with the response rate to the screening interviews of 60 percent, the overall response

27

rate to the pretest survey was 51 percent (.60 x .85 = .51). If we consider only telephone interviews, the

overall response rate was 53 percent (.60 x .88= .53).

1. Main Questionnaire Completion Rates by Interview Mode 

The overall completion rate to the main questionnaires masks important differences by interview mode.

We obtained substantially higher completion rates administering the questionnaires by telephone than inperson. Overall, 384 of 438 eligible pretest respondents (88 percent) surveyed by telephone completed

main questionnaires compared with 67 of 96 eligible respondents(70 percent) interviewed in-person (Table

III.3). Completion rates for questionnaires administered by telephone ranged between 82 and 100 percent

for the four respondent groups, whereas rates for in-person interviews ranged between 50 percent and 83

percent (Table III.2). For each respondent subgroup, completion rates were higherfor telephone than inperson interviews. 

Completion rates were higher for telephone interviews than for in-person interviews for several

reasons. The most important reason was the reluctance of households selected for in-person interviews
71

TABLE III.2

RESPONSE RATES TO MAIN QUESTIONNAIRES, BY QUESTIONNAIRE TYPE 

Type of Respondent Questionnaire Mode Eligible to Field Completes Rate

Length of Questionnaire Number Number Sent Number of Response

a b

Working nonparticipant Short Telephone 84 n.a. 77 91.7

Short In-person 16 10 10 62.5

(100.0)

Long Telephone 91 n.a. 76 83.5

Long In-person 19 18 14 73.7

(77.8)

Elderly nonparticipant Short Telephone 55 n.a. 45 81.8

Short In-person 10 7 6 60.0

(85.7)

Long Telephone 81 n.a. 69 85.2

Long In-person 12 8 6 50.0

(75.0)

Working participant Short Telephone 29 n.a. 27 93.1

Short In-person 12 10 10 83.3

(100.0)

Long Telephone 33 n.a. 30 90.9

Long In-person 8 7 6 75.0

(85.7)

Elderly participant Short Telephone 34 n.a. 34 100.0

Short In-person 10 9 9 90.0

(100.0)

Long Telephone 31 n.a. 26 83.9

Long In-person 9 7 6 66.7

(85.7)

Total Both Both 534 n.a. 451 84.5

Spanish-speaking Both Both 24 n.a. n.a. n.a. c

Target met Both Both 104 n.a. n.a. n.a. d

Total Both Both 662 n.a. n.a. n.a.

NOTES:

Number of eligible cases agreeing to participate in an in-person survey. a

Number in parentheses is the percentage of cases released to the field that were completed interviews. b

Respondents who completed a screening interview, but were not eligible to complete a main questionnaire because they spoke only

c

Spanish.

Respondents who were eligible to complete a main questionnaire but who were not interviewed because we had already met our target for

d

that group.

n.a. = not applicable
72

TABLE III.3

SELECTED SUMMARY MEASURES FOR RESPONSE RATES TO MAIN QUESTIONNAIRE 

Mode of Interview or Length of Number Number of Response

Type of Respondent Questionnaire Eligible Completes Rate

Telephone Interviews Both 438 384 87.7

In-person Interviews Both 96 67 69.8

Nonparticipants Both 368 303 82.3

Participants Both 166 148 89.2

All Short 250 218 87.2

All Long 284 233 82.0
73

to allow field interviewers into their homes. Twenty of the 96 eligible pretest respondents selected for an

in-person interview (21 percent) refused to participate in the study once they learned at the end of the

screening interview that the main interview would be conducted in-person. These respondents reported

that they would be willing to participate if the main interview was conducted over the telephone. We

completed in-person interviews with 67 of the remaining 76 eligible pretest respondents, or 88 percent.

This percentage is similar to the overall completion rate achieved for telephone interviews. 

Another reason for the higher completion rate for telephone interviews was that the lag between the

time of the initial telephone contact and the call by the field interviewer to schedule an interview gave

respondents time to think about whether they in fact wanted to participate in the survey. Nine respondents

changed their mind. A few respondents said they did not want to participate when field interviewers called

to schedule appointments for in-person interviews. The others scheduled interviews, but were not home

when the interviewer arrived at the scheduled time and could later not be reached to schedule another

appointment. Although some of these cancellations may have in fact been legitimate,we suspect that some

of them were intentional--the respondents changed their minds about being interviewed so they scheduled

interviews at times they knew they would not be at home. It is possible that completion rates for in-person

interviews could have been higher if field interviewers were given more time and resources to attempt to

persuade reluctant respondents to participate. 

In-person interviews were conducted in the pretest to test the in-person administration of the

questionnaire. In a national survey, in-person interviews would only be administered to persons without

telephones who would also be screened in-person, and respondents who are screened by telephone would

also complete the main questionnaire by telephone. Hence, the in-person response rates found in this study
74

are not indicative of the response rates that would be found in a national study in which only non-telephone

households would be administered questionnaires in-person.

2. Main Questionnaire Completion Rates by FSP Participant Status

Completion rates were higher for FSP pretest participants than nonparticipants; this was especially

true for in-person interviews. Overall, 148 of 166 FSP participants (89 percent) completed main

questionnaires compared with 303 of 368 nonparticipants (82 percent, see Table III.3). 

Completion rateswere higher for participant than nonparticipant respondents for two reasons. First,

virtually all FSP participants sampled for this study were sent an advance letter describing the purpose of

the study and how their household was selected for the study, whereas nonparticipants were not provided

an advance letter. Second, as current recipients of program benefits, FSP participants probably feel a

greater sense of obligation to respond than nonparticipants, who are not directly benefitting from the

program. In addition, to the extent that they believe the input they provide will be considered and used to

improve the FSP, participants may be more predisposed than nonparticipants to respond because they

perceive that they would directly benefit from future program enhancements through their continued

program participation. 

3. Main Questionnaire Completion Rates by Questionnaire Length

In general, completion rates were also higher for those pretest respondents administered the short

version of the questionnaire than the long version. The differences were approximately 5 to 6 percentage

points (see Table III.2). Overall, 218 of 250 individuals (87 percent) administered a short questionnaire

completed the interview compared with 233 of 284 individuals (82 percent) administered the longer version

of the questionnaire (see Table III.3). A similar pattern holds when we control for interview mode (see
In fact, as discussed in the next section, it turned out that the interviews took longer than this. 28

75

Table III.2). For interviews conducted by telephone, 91 percent of households administered a short

version completed the questionnaire compared with 85 percent administered a long version. Overall, 73

percent of households administered a short version in-person completed the interview compared with 67

percent of households administered a long version in-person. 

Most of the nonresponse to the main questionnaire occurred priorto its administration; there were

relatively few break-offs to the main interview once it began (see next section). In the case of telephone

interviews, after the completion of the screening interview, interviewers would lead into the main

questionnaire by saying they have additional questions, giving the length of time the remainder of the

interview would take (10 minutes for a short-version and 20 minutes for a long version). Not surprisingly,

28

compared with respondents who were selected for the short-version of the questionnaires, a greater

proportion of respondents who were selected for the long-version of the questionnaires refused to continue

once they learned how much more time they would have to spend completing the interview. In addition,

some respondents who completed the interviews, typically elderly ones, commented during the debriefing

section of the interview that the length of the interview was “trying” and “too long,” often asking

interviewers several times during the course of the interview how much longer the interview would take.

4. Interview Break-Offs

Once the main questionnaire was being administered, only five respondents interviewed by telephone

broke-off the interview; none of the respondents interviewed in-person broke-off interviews once they

started. There was no pattern to the break-offs. A few respondents said they did not feel well; the others
76

mentioned needing to tend to small children. None of the break-offs appeared to be related to sensitive

questions. Break-offs also did not appear to be related to the length of the interview. 

B. MAIN QUESTIONNAIRE INTERVIEW LENGTH

Overall, long-versions of the questionnaire administered by telephone took on average 26 minutes to

complete; short-versionstook approximately 15 minutes (see Table III.4). Both the short- and longerversion instruments took approximately 10 minutes longer to administer in-person than by telephone. 

Longer administration times for in-person interviews most likely reflect the fact that in-person

interviews tend to be more “conversational” than telephone interviews. There are greater opportunities for

social interaction in face-to-face personal interviews than for interviews conducted over the telephone by

essentially unknown interviewers. In-person interviews also tend to be longer because respondents are

more likely to be interrupted or distracted. For example, this might happen when other family members

present during the interview interject comments or children interrupt the discussions. 

The administration time for working questionnaires did not differ much from the administration time for

the elderly questionnaires. Interview administration time for elderly respondents is usually longer than for

other respondents. But in this survey, the respondent was chosen to be the person who would have applied

for food stamps, or did apply for food stamps, a person who is less likely to be cognitively impaired than

the broader elderly population. Also, the respondent to 15 percent of the elderly questionnaires was not

elderly. 
77

TABLE III.4

ADMINISTRATION TIME FOR MAIN QUESTIONNAIRES, BY QUESTIONNAIRE TYPE

Type of Questionnaire Interview Time (Minutes) 

Type of Respondent Length Mode Median Mean

Questionnaire Questionnaire 

a

Working nonparticipant Short Telephone 16 17

Short In-person 27 27

Long Telephone 25 26

Long In-person 38 39

Elderly nonparticipant Short Telephone 15 17

Short In-person 23 23

Long Telephone 25 25

Long In-person 38 38

Working participant Short Telephone 15 14

Short In-person 23 22

Long Telephone 27 28

Long In-person 31 32

Elderly participant Short Telephone 15 16

Short In-person 30 26

Long Telephone 27 29

Long In-person 31 29

Summary Measures

Nonparticipants Short Telephone 17 15

Nonparticipants Long Telephone 26 25

Participants Short Telephone 15 15

Participants Long Telephone 29 27

All Short In-person 25 25

All Long In-person 35 35

All Both Both 20 23

NOTE:

Eight cases--all telephone interviews--had recorded interview lengths in excess of 100 minutes. When estimating the mean, we treated

a

these cases as an error in reporting by the interviewer and assigned them “missing data.” These cases are not included in the calculation

of the mean of the interview administration time.
78

FSP nonparticipant interviews were generally somewhat longer than participant interviews. This was

true for boththe short and long versions of the questionnaires. Participant and nonparticipant questionnaires

were designed to be similar except nonparticipants were asked about reasons for nonparticipation

(Section E). Despite this difference, we anticipated that administration times would be similar between

nonparticipants and participant questionnaires. On one hand, nonparticipant interviews would tend to be

longer than participant ones because they contain Section E--the detailed sequence of questions on reasons

for nonparticipation--and participant questionnaires do not. On the other hand, although both

nonparticipant and participant questionnaires contain sections on FSP application and participation

experiences, all participants would be asked most questions in these sections whereas only those

nonparticipants that were former participants or had previously applied for food stamps would be asked

these questions. It turned out that about 16 percent of nonparticipants in our pretest samples had prior

experience with the FSP, having either applied or received food stamps in the past three years. This

resulted in nonparticipant interviews being somewhat longer on average than those of participants.

C. CHOICE OF RESPONDENT FOR THE MAIN QUESTIONNAIRES 

When conducting a survey about nonparticipation among working and elderly households, who in the

household should be the respondent to the main questionnaire? In nonparticipating households, we

interviewed the person in the household who would most likely go to the FSP office and complete an

application form if the household decided to participate in the program. Our rationale was that this person

would be the most knowledgeable about the reasons for nonparticipation. Similarly, in participating

households, we interviewed the person who last applied for food stamps. 

At the end of the screening interview, we asked nonparticipant respondents whether they or someone

else in the household would most likely go to the FSP office and complete an application form. We asked
79

participant respondents whether they or someone else in their household last applied for food stamps.

About 83 percent of the respondents replied that they would either apply for food stamps or they had

previously applied for food stamps. In these cases, the respondent to the screening interview was

administered the main questionnaire. About 17 percent of the respondents replied that it was someone else

in the household who would apply for food stamps or who had previously applied for food stamps. In

these cases, we administered the main questionnaire to someone other than the respondent to the screening

interview.

Our decision to administer the main questionnaire to the person in the household who would apply for

food stamps or who did apply for food stamps meant that the respondent to the main questionnaire was

sometimes not the working person in the working household or the elderly person in an elderly household.

In about 20 percent of working households, the person who would (or did) apply for food stamps was not

working; and in about 15 percent of elderly households the person who would (or did) apply for food

stamps was not elderly. 

D. PERFORMANCE OF INDIVIDUAL QUESTIONS

Thissection discusses the performance of the main questionnaires at the question-level. The discussion

is organized around each topic section. For each section, we first briefly describe the section and its

informational objectives; summarize the section’s performance; and then discuss problem questions and

corrective action for those questions that did not appear to work well.

1. Section A: Household Composition
80

In the long versions of the questionnaires, Section A asks respondents to list each member of the

household including themselves, and then for each listed household member, to report the household

member’s age and the relationship of the household member to the respondent. The short versions only ask

respondents to report the number of household members age 60 or older and the number of members less

than 18 years of age. 

Respondents did not have any difficulty answering these questions. Less than 5 percent of the

respondents did not answer either the question about the household member’s relationship to the

respondent or the question about the age of the household member. Behavioral coding revealed only three

instances in which respondents asked for clarification or inwhich interviewers did not probe correctly. One

respondent did not feel comfortable listing the “first name” of a household member on the household grid.

Based on this evidence, there is no need to revise any of the questions in Section A of the questionnaires.

2. Section B: Knowledge of the Food Stamp Program

This section of the questionnaire collects information about factors that may be related to the

respondent’s awareness of the program, such as whether the respondent received food stamp benefits as

a child, or whether he or she knew someone (for example, a neighbor, friend, or coworker) who received

food stamps. It also asks whether the respondent had heard of the FSP prior to the pretest. Section B

only appears in the long versions of the questionnaire. 

Section B questions worked well; there is no need to revise any of the questions. Refusals or

responses of “don’t know” varied between 7 and 15 percent. While this might seem like a relatively large

proportion, “don’t know” responses predominated. These are legitimate responses to questions about

whether the respondent’s parents received food stamps when the respondent was a child and whether the

respondents neighbors or friends receive food stamps. 
The questions corresponding to the question number are almost identical across the questionnaires.

29

81

Behavioral coding indicated a few cases in which respondents asked interviewers to repeat questions.

About one-quarter of the questionnaires subject to behavioral coding involved respondents taking a long

pause before answering. Again, this is not evidence of problematic questions; rather, this evidence is

consistent with acceptable response behavior on the part of respondents who are simply taking a little extra

time to answer because they are being asked to think retrospectively.

3. Section C: Food Stamp Program Participation History 

This section asks respondents about their experiences receiving and using food stamps. Current FSP

participants as well as nonparticipants who have received food stamps in the past were asked these

questions. In addition, we asked nonparticipants who had previously received food stamp benefits why

they stopped participating in the FSP. Section C questions are included in both the long and short versions

of the questionnaires.

Most ofthe questions in Section C worked well. However, respondents had difficulties understanding

some of the questions. In some cases, it appears that no revision would be necessary, as long as future

versions of the questionnaire are administered by computer assisted survey interviews(CASI). Question

C4 provides an example of this issue: 

29

“C4: How did you get your food stamp benefits in (DATE FROM C3)? Did you get

coupons or credit to an EBT card? 

Probe: By EBT card I mean........

Colorado: EBT card is called Colorado Quest; 

Massachusetts: EBT card is called an EBT card; 

Texas: EBT card is called the Lone Star Card;

No EBT card in Minnesota, North Carolina, or Pennsylvania
82

COUPONS ............................01

EBT CARD ...........................02”

Forty percent of interviewer-respondent interactions were classified as either “medium or high” and

involved several instances in which respondents asked the interviewer for clarifying information. In addition,

a few interviewers incorrectly administered the question or probed responses. These problems were

largely the product of the “hard copy” nature of the questionnaires. Interviewers had to determine (1) what

date to “fill-in” by looking at Question C3, and (2) what location-specific EBT card name to mention.

CASI programming would fill-in this information automatically, thereby eliminating the confusion and

awkwardness respondents were experiencing from interviewers having to figure out dates and locationspecific names of EBT cards.

Question C6 of the nonparticipant questionnaires asked former participants : “Why did you stop

participating in the food stamp program?” Several former participants provided an answer not

covered by the response codes that had to do with “failing to comply with FSP rules or respond to a

request by FSP staff.” We recommend that the questionnaires be revised to include this as a response

category. 

One-third of questionnaires subject to behavioral coding involved respondents asking the interviewer

to repeat Question C13c: 

“C13c When you received food stamps in the past three years, did you ever do things

so that people would not find out you received food stamp benefits? 

Probe: 

For example, some people try to use their food stamp benefits in stores where they are

unlikely to meet anyone they know.”
83

We believe respondents would have less trouble with Question C13c if it is revised so that the text currently

serving as a “probe” is included directly as part of the question as asked. The revised question would read:

“C13c When you received food stamps in the past three years, did you ever do things

so that people would not find out you received food stamp benefits, such as using your

food stamp benefits in stores where you are unlikely to meet anyone you know?”

4. Section D: History of Food Stamp Applications

Section D collects information from respondents about their recent experiences applying for food

stamps: the reasons they applied for food stamp benefits, whether they applied for other benefits at the time

they applied for food stamp benefits, whether they applied for food stamp benefits in-person or via an

authorized representative, and specific problems that they may have encountered when applying.

Nonparticipants who started but did not complete the application process are asked to state the reasons

they did not complete the application process. Participants are asked about factors that helped them

overcome any barriers applying for or using food stamps. Most questions in the section worked as

designed. There were some exceptions. The problem questions, and our recommended solutions, are

discussed in the remainder of this section.

a. Confusion About the Term “Application” 

The focus of Section D is on the respondent’s experiences during his or her most recent application

for food stamps, as opposed to recertification--the periodic renewal of one’s application for food stamps.

Despite the inclusion in Question D1 of the phrase “by applied, I mean have you completed a new

application form,” some respondentshad difficulty understanding that we wanted information about their

application rather than ongoing recertification. Behavioral coding indicated considerable interviewerrespondent interaction categorized in the “medium/high” range in the first two to three questions in Section
84

D, as several respondents needed clarification on what we meant by “applying for food stamps.” We

believe we can reduce confusion on the part of respondents and the resulting lengthy interviewerrespondent interaction by inserting the following introductory text prior to asking Section D questions:

“INTRODUCTION TO SECTION D: My next questions are about applying for food

stamps during the past three years. By applying I mean when you completed a new

application for food stamps. Please do not include the times you were required to

recertify your food stamp eligibility, that is, when you had to go back to renew your

application for food stamp benefits.“

b. Need to Break Some Questions into Multiple Questions

Our examination of pretest data identified four questions in Section D that would benefit from being

broken into two or more questions or components. These are questions D3, D5i, D16i, and D17e. 

“D3 These next questions refer to the last time you applied for food stamp benefits

(MONTH/ YEAR FROM QUESTION D2).

When you applied for food stamp benefits in (MONTH/YEAR FROM D2), did you also

apply for any other kinds of public assistance such as Supplemental Security Income

(SSI), AFDC (FILL STATE WELFARE NAME), Medicaid, or General Assistance?

Colorado: Colorado Works

Massachusetts: Transitional Aid to Families with Dependent Children

Minnesota: MFIP Minnesota’s Family Investment Program

North Carolina: Work First Benefits

Pennsylvania: TANF

Texas: TANF

YES ............................ 01

NO............................... 02

DON’T KNOW........... -1
85

The objective of this question is simple: to determine whether the FSP application was coordinated with

the household’s application for other assistance programs. However, the question as presently worded

is long and complex. Several respondents asked interviewers to repeat the question and/or clarify it.

Automated “fills” of the date and program names under CASI administration will handle some problems.

However, the question could be further improved by separately asking about the fourmain programs that

one may potentially apply for at the same time when applying forfood stamps. The revised question would

read as follows:

“D3 When you applied for food stamp benefits in (MONTH/YEAR FROM D2), did you

also apply for AFDC (FILL STATE WELFARE NAME)? 

Colorado: Colorado Works

Massachusetts: Transitional Aid to Families with Dependent Children

Minnesota: MFIP Minnesota’s Family Investment Program

North Carolina: Work First Benefits

Pennsylvania: TANF

Texas: TANF

YES ............................ 01

NO............................... 02

DON’T KNOW........... -1

D3a When you applied for food stamp benefits, did you also apply for Supplemental

Security Income? 

YES ............................ 01

NO............................... 02

DON’T KNOW........... -1
86

D3b When you applied for food stamp benefits did you also apply for General

Assistance? 

YES ............................ 01

NO............................... 02

DON’T KNOW........... -1

D3c When you applied for food stamp benefits did you also apply for Medicaid? 

YES ............................ 01

NO............................... 02

DON’T KNOW........... -1

Respondents also had difficulty with Question D5i:

“D5: Which of the following reasons led you to apply for food stamp benefits in

(MONTH/ YEAR FROM QUESTION D2)?

..........

D5i: You learned about the program or your eligibility for food stamp benefits?”

Respondents had difficulty with Question D5i because it combined two concepts: (1) learning about the

program, and (2) learning that one’s household was eligible for food stamps. Interviewer-respondent

interaction on this question was high. We suggest revising Question D5i so that it reads as follows:

“D5: Which of the following reasons led you to apply for food stamp benefits in

(MONTH, YEAR FROM QUESTION D2)?

..........

D5i: You learned about the existence of the Food Stamp Program?
87

D5j: You found out you may be eligible for the Food Stamp Program?

D5k: Some other reason (SPECIFY)”

This same fix would also apply to Question D16i. 

Question D17 asks currentparticipants about various attitudes and other circumstances that might have

helped them decide to participate in the FSP, such as inability to get by without food stamps or whether

they got a lot of help with the application process:

“D17. Now I would like to talk to you about some things that may have helped you

decide to use food stamp benefits. 

...............

D17e: Are you uncomfortable getting food from family, friends, charities, or other

programs?” 

 

There are two problems with D17e. Not all respondents have family or friends they potentially could ask

for food. For respondents who do not have family or friends, it simply does not make sense to ask them

whether they are uncomfortable approaching these individuals for help. Second, the question combines

two different types of sources of help: family and friends on the one hand and charities and other formal

programs on the other. The following revision addresses these issues:

“D17. Now I would like to talk to you about some things that may have helped you

decide to use food stamp benefits. 

...............

D17e: Are you uncomfortable getting food from charities or other programs?” 
88

D17f: Do have family or friends close-by that you could approach for food?

YES.....................01

NO.......................02 ----> SKIP TO D18

D17g: Are you uncomfortable asking these family members or friends for food?

YES.....................01

NO.......................02 

5. Section E: Reasons for FSP Nonparticipation

The objective of the questions in Section E of the main questionnaires is to ask nonparticipant

respondents directly why they do not currently participate in the FSP. Respondents are first asked, in a

series of closed-ended questions, whether a specific factor was a reason they did not participate (Questions

E1a to E1p). After each question, the respondent was then asked whether it was an important reason they

do not receive food stamps (Questions E2a to E2p). After all potential reasons are explored and the

respondent is given an opportunity to identify any other reasons that were not asked about by the

interviewer, the interviewer than asks the respondent to identify the most important reason (Question E4).

The remainder of Section E includes more detailed questions about particular reasons for nonparticipation

given in E1. For example, respondents answering that they think they are not eligible for food stamps (E1b)

are asked why they think they are ineligible (E7), whether someone told them they are not eligible, and if

so, who told them and when (E8, E8a, and E8b), or whether they think they are ineligible because they

knowsomeone like them who is ineligible (E8c). Section E is included in the nonparticipant questionnaires

only. Both the long and short versions of the questionnaire contain the complete section. 

Overall, Section E of the questionnaires worked well. However, respondents had some difficulties

with a few of the questions. The problematic Section E questions and recommended revisions are
89

discussed below. Because Section E contains several complex skip patterns based on responses to E1-

series questions, using a hard-copy instrument was difficult and time-consuming for interviewers.

Administration of Section E would be greatly simplified, and fewer interviewer errors made, if it were

administered by CASI. 

a. Recommended Revisions to Question Grid E1 through E4

Respondents had difficulty with two questions in the E1-series. Question E1c asks respondents: “Do

you think it would be hard to get to the food stamp office?” Respondents who did not know where

to go or who to contact in order to apply for food stamp benefits had some difficulty with this question.

One possible fix would be only to ask E1c if respondents know where to apply (that is, they respond “yes”

on E1a). However, we think access to the food stamp office is sufficiently important that all respondents

should be queried on the concept, regardless whether or not they know exactly where to apply. We

recommend that the following interviewer instruction be added to Question E1c:

“E1c INTERVIEWER INSTRUCTION: IF RESPONDENT ANSWERED

“DON’T KNOW” to E1a, THEN READ: Typically you must go to the local

food stamp office in-person in order to apply for food stamps. Depending on

how far you live from the office, you may need to drive, take a taxi or public

transportation, or walk, in order to get there.”

We intended this question to also include difficulties getting to the food stamp office because of constraints

related to employment and problems finding care for a dependent in the home. However, when asked in

E1p for other reasons that the respondent did not participate, some working nonparticipants reported that

an important reasons for not using food stamps was that they could not take time off work. To make this

question clearer, we recommend that the question be revised as follows:
90

“E1c Do you think it would be hard to get to a food stamp office to apply for food

stamps because of transportation problems, health problems, difficulty taking time of

work, or because of the need to find care for someone in your home.” 

We probe in question E10 for the reason the respondent finds it difficult to get to a food stamp office.

 Several respondents asked interviewers to repeat Question E1k:

“E1k Would you dislike relying on government assistance?”

Respondents struggled with the meaning of “government assistance.” We recommend revising this 

question to read as follows:

“E1k Would you dislike having to rely on the government for assistance?”

Respondents had difficulty with the E2-series questions asking whether a particular factor that they

reported as having some role in their decision not to participate was “an important reason” they did not use

food stamps. We have carefully reviewed the benefits and costs of retaining the E2-series questions and

recommend that they be dropped from the questionnaire. The E1-series identifies reasons households are

not participating in the FSP and E4 provides respondents an opportunity to state what they perceive to be

the most important reason for not participating. Thus, the value of the E2-series is that it can tell us which

of the potentially several reasons identified in the E1-series are the more important ones. But this

information comes with a cost. Behavioral coding of the pretest questionnaires indicated that interviewerrespondent interaction for E2-series questions was “medium” or “high” for approximately half of

respondents (10 of 20). Several of respondents asked the interviewer to either clarify or repeat one or

more of the E2-series questions. In addition, the E2-series took several minutes to administer. This is

because the interviewer needs to repeat the question: “Is this an important reason you don’t use food
91

stamp benefits?” each time a respondent gives a response consistent with a reason for nonparticipation

in E1. Knowing which of several reasons given by respondents are the more important ones is not worth

increasing the length of the interview and risking the loss of interest and focus in the survey. Also, as we

will show in Chapter IV, the distribution of responses to the questions about the important reasons that the

respondent did not participate is similar to that of the responses to the questions about whether the factor

played any role in the decision to participate. This suggests that the E2-series does not add much to our

understanding of the reasons for nonparticipation.

The literature review and focus groups we conducted prior to our preparing the study questionnaires

revealed that some participants believed that, as current and/or former taxpayers, they were entitled to

receive food stamps (McConnell and Nixon 1996; Ponza and McConnell 1997). Both participants and

nonparticipants are asked a question to assess whether they have this attitude (nonparticipants are asked

this in Question E19 and participants are asked this in Question D17c). Behavioral coding revealed an

excessive amount of respondent-interviewer interaction to this question. Several respondents asked for

clarification or for the interviewer to repeat the question. The problem is the placement of the question in

the questionnaire. As the last question in Section E, it is nothing like the other questionsin Section E nor

the questions that follow in Section F. We recommend moving the question to Section B for

nonparticipants, and revising it as follows:

“B3 Do you think it’s OK for people who have paid taxes to get food stamps?”

 

We recommend replacing Question D17c with this question in the participant questionnaires.

Twenty working nonparticipants and nine elderly nonparticipants responded that they had “other”

reasons for nonparticipation. In 13 of these cases, the “other” reasons given were reasons that the
92

respondent had already given. One respondent gave an inability to obtain the necessary documentation

as a reason for nonparticipation and one other respondent cited “religious reasons.” The remaining

respondents gave nonsensical responses. We do not recommend adding any questions because of these

“other” responses.

b. Other Recommended Changes to Section E 

All respondents who said they thought they were ineligible for food stamps, were asked why they

thought they were ineligible (Question E7). Three types of responses that were coded as “other-specify”

are worth discussion. First, some nonparticipants said that they “did not need food stamps” in response

to the question about eligibility. As “lack of need” is not a reason for ineligibility, in any future administration

of the survey, interviewers should be instructed to probe for the underlying reason. For example, the

interviewer could probe: “But why do you think you are not eligible? Do you think it is because

your income is too high, you have too many assets, or some other reason?” Second, some

nonparticipants thought that they were categorically ineligible. Many of these nonparticipants thought they

were categorically ineligible because they worked. Others thought they were categorically ineligible for

other reasons, such as there were no children in the household. Third, some respondents thought they

were ineligible because they were students. We recommend adding “student,” “categorically ineligible

because working,” and “categorically ineligible for some otherreason” as additional response categories.

Question E12b asks respondents who thought they would have to wait a long time to be served how

long they thought they would have to wait. There were four response categories: 15 minutes or less, 16

to 30 minutes, 31 minutes to one hour, and more than one hour. As more than half the respondents
Less than 2 percent of respondents received food assistance from any other kind of program. 30

93

responded “more than one hour,” we recommend changing the fourth category to “one to two hours” and

adding an additional category “more than two hours.”

6. Section F: Receipt of Food Assistance from Sources Other Than the FSP

Section F contains questions about household members’ receipt of food assistance from sources other

than the FSP. These sources include: community or senior centers; school breakfast and lunch programs;

WIC; friends or relatives; emergency food network sources; through work; or other sources. Most of the

questions that appear in Section F were taken from the April Food Security Supplement of the Current

Population Survey (CPS). 

Respondents had no trouble with these questions. Interviewer-respondent interaction was normal.

No more than one respondent asked the interviewer to clarify or repeat the question on any single question.

Section F questions worked well and do not need to be revised in any way. 

Section F offers a few possibilities to cut back on the length of the long version of the questionnaire.

First, Questions F9 and F10 could be combined into a single question that asks about receipt of food and

meals from “emergency sources.” We could delete Question F7 that asks whether the household received

food or vouchers to buy food from any other kind of program since the prevalence of this is rare and could

be recorded under “food or meals obtained from any other sources we haven’t already mentioned.” 

30

7. Section G: Food Security

Section G of the questionnaires asks about the food security of the respondents’ households. The

questions in this section were all taken from the April Food Security Supplement Food Security/Hunger
94

Core Section of the CPS. The short-version of the questionnaire contained a shorter version of the section,

containing approximately half as many questions as the full section. 

The food security questions in the main questionnaires appear in other national surveys. For

comparability with other findings, it is desirable that the survey, if implemented on a national basis, include

the same versions as other surveys. Consequently, we are not recommending that any of the questions be

changed. However, thepretest shows that the section is demanding, requiring more interviewer-respondent

interaction than typical. 

Behavioral coding indicated that interviewer-respondent interaction tended to be in the “medium” range

for several Section G questions. This isto be expected given the structure of the questions. Questions in

this series typically start with a description of some dimension of food insecurity and then ask respondents

whether it is “often true,” “sometimes true,” or “never true” fortheir household. Respondents sometimes

forget what dimension is being asked about by the time the interviewer gets to the end of the question,

requiring the interviewer to repeat part or all of the question. 

In addition, several of the questions are complex, requiring the respondent to process multiple

concepts before articulating an answer. A good example of this is Question G9: “In the last 12 months,

did you ever cut the size of your meals or skip meals because there wasn’t enough money for

food?” Respondents need to think about several things before they can give an answer: (1) did I cut the

size of meals?; (2) did I skip meals?; (3) did I do this because there wasn’t enough money for food?; and

(4) did I do this at any time in the last year? 

Respondents had more difficulty with Question G5 than any other question in the sequence. Question

G5 reads as follows:
95

“G5 (I/We) relied on only a few kinds of low-cost foods to feed (my/our) (child/the

children) because (I was/we were) running out of money to buy food.” Was that often,

sometimes, or never true for (you/your household) in the past 12 months?

Probe: By low-cost food we mean rice, beans, macaroni products, bread, or potatoes, or

foods like that.

Often true.................................... 01

Sometimes true...........................02

Never true ..................................03

DON’T KNOW........................... -1”

Respondents in one-quarter of the interviews that were behavioral coded requested that the interviewer

either repeat or clarify this question. FNS may want to consider breaking the question into two

components, similar to G9/G9a:

“G5 In the past 12 months, did (you/your household) rely on only a few kinds of lowcost foods to feed (your/your household’s) (child/children) because (you/we were)

running out of money to buy food.” 

Probe: By low-cost food we mean rice, beans, macaroni products, bread, or potatoes, or

foods like that.

YES ........................................... 01

NO ............................................. 00 SKIP TO G10

DON’T KNOW ......................... -1 SKIP TO G10

G5a Was that often or sometimes true for (you/your household) in the last 12 months?

Often true................................... 01

Sometimes true...........................02

DON’T KNOW.......................... -1”
Asterisks were placed next to response codes of questions that interviewers would need to reference

31

in order to decide on skip patterns. 

96

Finally, the section contains a number of complex skips that require interviewers to process information

from previous questions, often combinations of questions, in order to determine which question to ask next.

We implemented procedures to facilitate this process in the hard-copy administration of the questionnaire.31

Despite these procedures, interviewers were prone to make errors in the complex skip logic and the section

took several minutes to administer. Some interviewers were also making minor changes when administering

certain questions, although they usually did not change themeaning of the questions when doing so. The

administration of the instrument would be much quicker and much less prone to interviewer errorif done

by CASI. 

8. Section H: Employment History 

Section H obtains detailed information on employment of household members. It asks respondents

to report the wage rate and hours worked of each person in the household. It also asks about the current

occupation and the work history of the person in the household who works the most hours. Section H

appears only in the long versions of both working and elderly household questionnaires. (In all other

questionnaires we ask the respondent how many people in the household currently work at a job for pay

and whether the respondent works for pay. These questions are in Section A of the questionnaire,

however). Most of the Section H questions were adapted from other national surveys, such as the survey

developed by MPR for the U.S. Department of Labor’s Trade Adjustment Assistance study.

Section H questionsworked-well. There was modest interviewer-respondent interaction prior to the

respondent answering questions. There was relatively little missing data. Some respondents had difficulty

providing information on the weekly work hours (H2) and wages/salary (H3) of other household members
97

because they might not know it precisely. Some respondents were reluctant to give information on their

wages or salary and that of other household members. However, many of these individuals provided the

information after interviewers reminded them that their responses would be kept confidential and that only

aggregate or summary measures would be reported for the entire sample and not the earnings of individual

households or family members. 

9. Section I: Health 

Section I is a short section which collects information on the general health and physical and cognitive

functioning of respondents in elderly households only. The questions appearing in this section of the

questionnaire were adapted from the questions in the National Health and Nutrition Examination Survey

(NHANES III) and the 1990 Census of Population. Respondents had no difficulty answering these

questions. There was minimal missing data. Interviewer-respondent interaction prior to answering the

question for most respondents was “low.” These questions do not need to be changed. 

10. Section J: Social Supports

Section J of the main questionnaires asks respondents a series of questions about the length of time

they have lived in the neighborhood, the frequency of making social visits or having people to their homes,

and whether relatives live close-by. The questions were adapted from NHANES III. Respondents had

no problems with these questions. 

11. Section K: Income and Expenses 

Questions on income and expenses of the respondents’ household are contained in Section K. These

questions only appear in the long versions of the working and elderly participant and nonparticipant
98

questionnaires. Respondents generally did not have difficulty responding to these questions. However,

there were a few exceptions.

Questions K3 and K4 ask respondents who own their homes whether their mortgage includes

property taxes and insurance:

“K3 (Do you/Does your household) make a separate home insurance payment?”

“K4 (Do you/Does your household) pay a separate property tax bill?”

Several respondents asked interviewersto repeat these questions. We propose revising both questions

so that they get at the underlying issue more directly, as follows:

“K3 (Do you/Does your household) make a separate home insurance payment or is it

included in your mortgage payment?

MAKE A SEPARATE PAYMENT .....................01

INCLUDED IN MORTGAGE PAYMENT .......02

DON’T KNOW ....................................................-1”

“K4 (Do you/Does your household) pay a separate property tax bill or is it included in

the mortgage payment?

MAKE A SEPARATE PAYMENT .....................01

INCLUDED IN MORTGAGE PAYMENT .......02

DON’T KNOW ....................................................-1”

A similar criticism applies to Question K5a:

“K5a (Do you/Does your household) pay separate heating or air-conditioning costs?
99

We recommend that K5a be revised to read:

“K5a (Do you/Does your household) pay separate heating or air-conditioning costs or

are these costs included in your monthly rent payment?

MAKE A SEPARATE PAYMENT .....................01

INCLUDED IN RENT PAYMENT ...................02

DON’T KNOW ....................................................-1”

Respondents had difficulty with question sequence K7b through K9, which asks whether the

household contains disabled members or elderly, and, if so, what were the household’s out-of-pocket

expenses formedical expenses for these individuals. Respondents had difficulty understanding what we

meant by “disabled household member.” Our approach entailed asking whether the household contained

disabled individuals and then defining what we meant by disability. We believe the preferred way to get

at this issue is to ask directly whether the household contains individuals satisfying our definition. Making

this change affects other questions in the series. We recommend series K7b through K9 be revised as

follows:

“K7b: Does anyone in the household receive SSI benefits because of a disability, or

receive social security disability checks, disability retirement pensions, railroad

retirement disability payments, or veteran disability benefits? 

YES ........................................ 01 

NO .......................................... 00

DON’T KNOW ...................... -1
100

K7c Is there anyone in your household who is age 60 or older?

YES ........................................ 01 

NO .......................................... 00 

DON’T KNOW ...................... -1 

K7d INTERVIEWER: CHECK K7b and K7c. IS ANYONE IN THE HOUSEHOLD

DISABLED (K7b EQUALS 01) OR AGE 60 OR OLDER (K7c EQUALS 01)?

YES ........................................ 01 

NO .......................................... 00 ---> SKIP TO K10

K8 Last month, did (you/your household) pay health insurance premiums or make

payments to belong to an HMO (CIRCLE ALL THAT APPLY)?

PAY HEALTH INSURANCE PREMIUMS ................ 01

MAKE PAYMENTS TO BELONG TO AN HMO ....... 02

DO NOT MAKE PAYMENTS ...................................... 00 ---> SKIP TO K9

DON’T KNOW .............................................................. -1 ---> SKIP TO K9

K8a Now thinking about those individuals receiving disability benefits or are age 60

or older, how much did (you/your household) pay last month for health insurance

premiums and payments to belong to an HMO? Please tell me only the amount that you

and members of your household pay out-of-pocket.

$|___|, |___|___|___|

DON’T KNOW ........................ -1

K8b NO QUESTION THIS VERSION
101

K8c INTERVIEWER: CHECK K7b and K7c. IS ANYONE IN THE HOUSEHOLD

DISABLED (K7b EQUALS 01) OR AGE 60 OR OLDER (K7c EQUALS 01)?

YES ........................................ 01 

NO .......................................... 00 ---> SKIP TO K10

K9 Now think about the people in your household who receive disability benefits or are

age 60 or older. Last month how much were their out-of-pocket medical expenses?

Please include doctor and hospital bills, prescription drugs, lab tests, or X-rays, and any

other medical expenses you paid out-of-pocket. Please exclude anything for which you

will be reimbursed.

 $|___|, |___|___|___|

DON’T KNOW ........................ -1

One omission from the questionnaire was a question to elicit how many household members were

either elderly or disabled. This information is needed to determine how many people in the household can

use the medical deduction. To obtain this information we recommend adding two questions after K7b,

K7b1 Is any disabled person in your household not elderly?

YES ............................. 01

NO ............................. 00 SKIP TO K7c

K7b2 How many people in your household are not elderly?

An interviewer check can be included as K7a so that households containing only elderly persons can

skip the questions about disabilities:
Prices of used vehicles are available on several internet sites. To value the vehicles reported in the

32

pretest, we used the internet site: http://www.autopricing.com.

102

K7a INTERVIEWER CHECK. 

Is everyone in the household elderly?

YES .......................... 01 SKIP TO K8

NO ........................... 00

K7c could also be an interviewer check.

Question grid K10.1 through K14.11 asks respondents aboutincomes sources and amounts received

by all members in the household. This is a demanding sequence of questions. Behavioral coding of

responses indicated interviewer-respondent interaction in the “medium” to “high” range for several

respondents for at least one income source or amount. Several respondents needed interviewers to repeat

or clarify questions. A few were reluctant to provide information on amounts of income received.

However, we do not think we can make this series of questions easier without sacrificing some of the

information collected. The administration of this sequence of questions would be greatly improved ifit was

administered by CASI.

Questions K15 through K17 were designed to provide information to calculate the value of each

vehicle owned by household members. Question K15 asks the respondent whether anyone in the

household owns a vehicle. Questions K16 asks households with vehicles to provide the year, make, and

model of each vehicle (up to three). These data can be used to estimate the value of the vehicles using

published data on the prices of used vehicles. If respondents are unable to answer Question K16, 32

respondents are asked in Question K17 for the approximate value of the vehicles.
103

Some nonparticipant respondents who reported in the screening interview that no one in their

household had any vehicles, were asked about their vehicles again in Question K15. This was because

with a hard-copy questionnaire it was too difficult for the interviewer to go back and check the response

to the vehicle question in the screening interview before asking K15. Although no respondents complained

about this question, we recommend that if the main questionnaire is to be administered by CASI that the

vehicle questions are not asked of respondents who report in the screening interview that they have no

vehicles.

Some respondents had difficulty answering the question about the year, make, and model of their

vehicles (Question K16.). Approximately 20 percent of respondents who owned one or more vehicles

either responded that they “didn’t know” or did not provide enough information for us to estimate the value

of the household’s vehicles. It may be possible to substantially reduce missing data by inserting probes.

For example, several respondents knew the year and make, but not the model of the vehicle. In a CATI

survey, the computer could be programmed to insert a probe that helped respondents recall the model.

Respondents sometimes gave the make instead of the “model.” Again, in a CATI survey, the computer

could be programmed to not accept this answer, prompting the interviewer to re-ask the question.

Even when respondents answered Question K16,we did not receive enough information to make an

accurate determination of the vehicle price. No respondent in the pretest gave the exact model of the

vehicle. In nearly all cases, the respondents gave a one-word answer such as “Camry” or “Corolla.”

However, there are many versions of each model. For example, in 1990 there were seven versions of the

Toyota Camry, ranging from an average retail price of $3,650 for the Base Sedan to $5,950 for the LE
33Obtained from the internet site: http://www.autopricing.com

104

ALL-TRAC Sedan 4-speed AT. The prices also vary with the condition of the vehicle and any

33

nonstandard equipment it may have.

 We recommend dropping question K17 that probes for the value of the vehicle. We believe that

respondents who do not know the year, make, and model of a vehicle are unlikely to know the

approximate value of the vehicle. In the pretest, the instructions to the interviewers were to ask about the

value of the vehicles only ifinformation on all the household vehicles was missing. If FNS wishes to retain

this question, the instructions should be changed so that the interviewers ask about each vehicle for which

there is any missing data on year, make, or model of a vehicle. 

12. Section L: Demographic Information

Questions that obtain information on the demographic characteristics ofthe respondent, such as age,

race/ethnicity, education, marital status, and citizenship appear in Section L. Respondents had little

difficulty with these questions. Review of the 40 questionnaires subject to behavioral coding showed no

more than one or two respondents asked for clarification or for the interviewer to repeat the question for

each question. Interviewer-respondent interaction prior to answering was consistently “low” for all

questions in this section. No item had more than 5 percent nonresponse most items had nonresponse in

the 1 to 2 percent range.

E. SUMMARY AND IMPLICATIONS 

Overall, the main questionnaires worked well and were well-received by respondents. The overall

completion rate for the mixed-mode survey questionnaire, once a household was determined eligible by

the screening process, was approximately 85 percent. The overallresponse rate for the pretest survey,
105

when one takes into account the screening interview completion rate, was 51 percent. Questionnaire

completion rates varied by interview mode: they were substantially higher for questionnaires administered

by telephone than in-person (88 percent versus 70 percent). Many of the respondents who refused to

complete an in-person interview reported that they would have completed the interview if they had been

surveyed over the telephone. For households with telephones, this suggests that the best strategy for

fielding the questionnaires on a national level would be by telephone. Completion rates were lower for

longer versions of the questionnaires. 

We recommend that the requirement that the respondent to the main questionnaire is the person who

would or did apply for food stamps be relaxed to any adult in the household who might apply for food

stamps. This would increase the likelihood that the respondentto the main questionnaire would be the same

person who is administered the screening questionnaire. We expect that this would raise the completion

rates to the main questionnaires and we would still talk with someoneknowledgeable about the household’s

experiences and decision-making. 

For telephone interviews, the “long” version of the questionnaire took on average 26 minutes to

administer and the “short” version 15 minutes. In-person interviews took approximately 10 minutes longer

to administer in each case. In addition, many respondents, especially elderly ones, complained about the

length ofthe long version of the questionnaires. We recommend a final version of the questionnaire slightly

shorter than the long versions of the questionnaire used in the pretest. We recommend ways to shorten the

long questionnaires in the next chapter. 

The main questionnaires were difficult to administer using hard-copy because of complicated skip

patterns and fills. It would be much more efficient to administer the entire survey by CASI; administering

the main questionnaire by CASI could shorten the length of the interview. We administered the main
106

questionnaire using hard copy in the pretest to avoid the cost of programming the instruments for such a

small sample. We recommend that CASI be used for both the screening interviews and the main

questionnaires, should the survey be implemented nationally. 

As expected, respondents had difficulty with some questions. However,most of these problems can

be remedied by revising questions. In most cases the revisions are straightforward: change a word or

phrase, simplify language, sharpen probes, include probes as part of the question, add interviewer

instructions, or expand response categories. Some fixes will require adding questions or breaking a

complex question into two or more questions or components. 
IV. THE INFORMATION COLLECTED BY THE QUESTIONNAIRES

The questionnaires were designed so that if administered on a larger, national scale they would enable

FNS to collect sufficient data to ascertain the reasons working and elderly households have low rates of

FSP participation. Care was taken to ensure that FNS would have sufficient information to be able to

assess whether the low rates of FSP participation are a cause for concern and, if they are, be able to

recommend the necessary policy changes. This chapter assesses the ability of the questionnaires to collect

sufficient information about the reasons for nonparticipation to make policy recommendations. 

The pretest provided information on the experiences and attitudes of samples of FSP participants and

nonparticipants from working and elderly households residing in ten U.S. counties. Because the samples

were purposively selected and the sample sizes are small, it is not appropriate to use the data to make

inferences about the reasons households containing working and elderly members do not participate in the

FSP nationally. However, we do present some of the findings from the pretest in this chapter for two

reasons. First, doing so illustrates the breadth of information that would be obtained from fielding the

survey on a national level. Second, it provides an opportunity to assess whether the questionnaires collect

the appropriate data. By carrying out some descriptive and comparative analyses and displaying results,

we will be able to determine whether there are specific topics or questions that need to be added and/or

superfluous questions that may be deleted.

The chapter is organized into six sections. Section A provides an overview of research objectives and

questions that may be addressed with data collected from the main questionnaires and describes the analytic

approaches to address them. Section B discusses how information from the long questionnaires can be

used to make a better prediction of FSP eligibility than was made by the screening interview. Section C

shows the range of analyses and findings on the reasons reported by respondents from working and elderly
108

households for not participating in the FSP. Section D demonstrates how the characteristics and

experiences of participants and nonparticipants may be compared to gain additional insight into the reasons

some working and elderly households participate while others do not. Section E compares the data

collected by mode of interview administration. Finally, Section F summarizes our main findings and

discusses their implications for a survey of nonparticipation that would be administered on a larger scale.

A. OVERVIEW OF RESEARCH OBJECTIVES AND ANALYTIC APPROACHES

We designed the questionnaires to allow two lines of inquiry into the reasons for nonparticipation: (1)

a direct approach--asking persons who were probably eligible for food stamps why they did not participate

in the FSP, and (2) an indirect approach--making statistical comparisons of the characteristics and

experiences of FSP participants and nonparticipants that can be used to infer reasons for nonparticipation.

Data collected from the questionnaires would support a broad-based analysis of FSP nonparticipation

by working and elderly households that, at a minimum, would enable FNS to address the following specific

research questions:

1. What reasons are given by FSP-eligible nonparticipants for not participating in the FSP?

2. Do the reasons given for nonparticipation differ for working and elderly households? 

3. How do the characteristics and past experiences of nonparticipants who give certain reasons for

nonparticipation differ from those who do not give the reasons?

4. How do the characteristics of FSP participants differ from those of nonparticipants?

5. What past experiences have participants and nonparticipants had with the FSP that may have

affected their household’s decision to participate?

6. Do participants have certain attitudes, motivations, or resources that nonparticipants do not

have that enable them to overcome perceived barriers to FSP participation? 
109

7. Are the low participation rates by working and elderly households a cause for concern? If

so, for which subgroups?

8. What program or policy changes are needed to increase participation by working and elderly

households?

Three types of analysis could be conducted with the data collected from the questionnaires: 

1. Descriptive tabular analyses. These involve presenting means and frequencies of the

characteristics or past experiences of either FSP nonparticipants or participants, and of the

reasons given for nonparticipation (nonparticipants only). 

2. Comparative tabular analyses. These involve comparing means and frequencies of

characteristics or past experiences of (1) FSP-eligible nonparticipants and FSP participants

or (2) different subgroups of nonparticipants, such as nonparticipants who give a specific

reason for nonparticipation. 

3. Multivariate regression analyses. These involve regressing outcomes, such as the

household’s decision whether to participate in the FSP or specific reasons reported for

nonparticipation, on individual and household characteristics, attitudes, and past experiences

with the FSP. 

B. INFORMATION TO DETERMINE FSP ELIGIBILITY

The RDD screening interview was designed to screen out respondents that are not eligible for the FSP.

However, because the screening interview needs to be short, some ineligible respondents will still pass the

tests in the screening interview and be administered a main questionnaire. Including respondents who are

ineligible for food stamps in samples of FSP-eligible nonparticipants will bias the findings on the reasons

for nonparticipation. For example, respondents who are ineligible are more likely to say they do not need

food stamps and that they think (correctly) they are ineligible.

The long versions of the questionnaires included questions on income by source, expenses required

to determine net income, vehicles, and citizenship information. This information can be used to better

ascertain whether nonparticipants in the sample are likely to be eligible for the FSP. We chose not to ask
110

any questions about financial assets other than the ones in the screening interview, because questions about

financial assets are lengthy, sensitive, and difficult to answer. Because of the time constraints, we did not

collect information on income, expenses, vehicles, or citizenship on the short questionnaires. 

In analyzing the data from the national survey, the sample should be restricted to only those that are

determined FSP eligible based on the more detailed information about income, expenses, and vehicles.

We found in the pretest samples that over one-third of nonparticipants who passed the screening interview

were found to be ineligible based on data collected by the main questionnaires. Because so many

households were found ineligible based on the more detailed income, expense, and vehicle information, it

is important that this information is collected in any survey on nonparticipation. Also, in designing a national

survey, the sample sizes should be inflated to take into account that most of the analyses of nonparticipation

will be conducted on only respondents that are determined eligible for food stamps using the more detailed

available income, expense, and vehicle data.

As over 40 percent of working nonparticipant households reported in the main questionnaires usual

wage rates and weekly hours worked that would be inconsistent with a monthly household income below

130 percent of poverty, we recommend adding an additional question and an interviewer check after the

existing earnings questions in the main questionnaires. After the respondent has given the “usual” hours

worked and hourly wage rate, we would ask whether each working household member worked these

hours over the past month. If they reply that they did not, we would ask how many hours in total they

worked over the past month. Using CASI, the computer could calculate the implied monthly household

earnings from the previous responses. If the total earnings exceeds 130 percent of poverty, the interviewer

could ask the respondent to reconcile the reported earnings with the reported monthly income in the

screening interview.
Questions about why the respondent did not complete the application process and why they did not

34

use food stamps they received are also asked of participants.

This question was not included in the short versions of the questionnaires. 35

111

C. REASONS REPORTED BY RESPONDENTS FOR NOT PARTICIPATING IN THE

FOOD STAMP PROGRAM 

The most obvious way to collect information on the reasons for nonparticipation is to ask FSP-eligible

nonparticipants directly why they do not participate in the program. The questionnaires ask nonparticipants

about their participation decisions in two ways. First, each nonparticipant questionnaire includes a section

that asks respondents directly about reasons they currently do not participate. Second, the questionnaires

ask nonparticipants who had received food stamps sometime in the previous three years why they had

stopped participating in the program; nonparticipants who had begun the FSP application process but not

completed it, why they did not complete it; and nonparticipants who had received food stamps but not used

them, why they had not used them. The rest of this section discusses the information collected from these

34

two sets of questions.

1. Reasons for Currently Not Participating in the FSP 

Previous surveys have found that when asked in one or two questions why they do not participate,

respondents tend to give answers that are too vague to use as a basis for policy recommendations

(McConnell and Nixon 1996). To avoid this, the questionnaires ask a series of structured closed-ended

questions about the reasons for nonparticipation. 

We begin by asking whether the respondent had heard of food stamps or the FSP before the survey

(Question B0). If they had not heard of food stamps or the FSP, we assumed that this was the most

35
112

important reason why they did not participate and did not ask the respondent any more questions about

the reasons why they did not participate. 

For respondents who had heard of food stamps or the FSP, for 15 separate reasons, we asked the

respondents whether the reason was applicable to them, and if it was, whether it was an important reason

they did not participate. At the end of this series of questions, we asked whether there were any other

reasons for their nonparticipation that we had not covered. We then asked which was the most important

reason they did not participate. 

The reasons for nonparticipation fall into five broad categories:

1. Lack of information, including an unawareness of the existence of the FSP, lack of

knowledge about where or howto apply for food stamps, and misperceptions about eligibility.

2. Perceived lack of need, including a perception that the respondent “could get by” without

food stamps, the belief that other households are more deserving, and a belief that the

respondent’s need is only temporary.

3. Size of the FSP benefit is so low that the respondent does not think it is worthwhile to apply

for food stamps.

4. Program features and administration including the complexity of the application process,

problems getting to the FSP office, program requests for personal information, and

perceptions of discourteous staff and unpleasant offices. 

5. Psychological reasons including the stigma related to applying for and using food stamps,

an attitude of not wanting help from the government, or the belief that family and friends would

not be supportive of the decision to participate.

Table IV.1 presents the responses to these questions about nonparticipation given by the

nonparticipants in the pretest. The findings presented in Table IV.1 and other tables in this chapter are

illustrative and should not be used as a basis on which to make inferences about the reasons for

nonparticipation. The columns entitled “Applicable Reason” in Table IV.1 show the percentage of
113

TABLE IV.1

REASONS REPORTED BY NONPARTICIPANTS FOR NOT PARTICIPATING IN THE FOOD STAMP PROGRAMa 

Percent of Nonparticipantsb

Working Elderly

Reason Reason Reason Reason Reason Reason Reason

Applicable Important Most Important Applicable Important Most Important 

Lack of Information

Don’t know FSP exists 2 2 2 7 7 7

c d d d e e e

Don’t know where to go or who to contact to apply 36 9 2 46 8 2

Don’t think eligible for FSP benefits 41 29 11 33 23 7

Perceived Lack of Need

Can get by on my own without FSP benefits 79 73 24 84 55 25

Feel others need FSP benefits more 80 52 13 75 50 14

Need is only temporary 63 36 5 30 16 3

Expected FSP Benefits Too Low

Think eligible for only a low benefit amount 45 24 5 35 23 8

Problems Related to Program Administration

Hard to get to FSP office 11 6 3 25 13 3

Application process is too long and complicated 23 12 3 27 15 2

Questions are too personal 21 8 2 24 16 2

FSP office staff are disrespectful 24 15 4 9 3 1

FSP office is unpleasant or unsafe 12 7 2 14 9 2
TABLE IV.1 (Continued)

Percent of Nonparticipantsb

Working Elderly

Reason Reason Reason Reason Reason Reason Reason

Applicable Important Most Important Applicable Important Most Important 

114

Psychological Reasons

Other family members or friends would not approve of 

respondent receiving food stamps 11 5 0 1 1 0

Feel embarrassed applying for FSP benefits 25 14 3 27 14 2

Would feel embarrassed using FSP benefits 16 12 1 25 18 2

Dislike relying on the government for assistance 44 29 3 37 21 3

Other Reasons 5 10 1 7 4 1

No reason given 12 12 12 14 14 14

Missing data 0 0 1 0 0 2

Sample Size 177 177 177 126 126 126

NOTES:

Data in this table should not be used to make inferences about the reasons households do not participate in the FSP nationally. Because the data were collected

a

in a pretest, the samples are small and not nationally representative. 

Includes nonparticipants who, based on information collected in the main questionnaire, were probably ineligible for food stamp benefits. b

This is the percentage of nonparticipant respondents to the long questionnaires who said they had not heard of food stamps or the FSP before we interviewed them.

c

These respondents were not asked about the reasons they did not participate; it was assumed that they were not participating because they were unaware of the

existence of the program. For these respondents, we counted an unawareness of the FSP as both an “important reason” and “the most important reason” for not

participating.

Three percent of the nonparticipants administered long questionnaires responded that they had not heard of food stamps or the FSP. d

Twelve percent of the nonparticipants administered long questionnaires reported that they had not heard of food stamps or the FSP. e
115

nonparticipants reporting that a given reason has at least some role in the decision not to participate. The

columns entitled “Important Reason” show the percentage of nonparticipants reporting that the reason was

an important factor. The columns entitled “Most Important Reason” show the percentage of

nonparticipants who report, after all reasons have been discussed, that the reason is the most important

reason why they do not participate in the program. To maintain sufficient sample sizes, the data presented

in Table IV.1 and the other tables in the chapter do not exclude respondents that we determined were not

eligible for food stamps. 

All the reasons for nonparticipation asked about in the questionnaire were relevant. For each of the

16 reasons we asked about, the reason was applicable for more than 5 percent of all respondents. We

do not recommend removing any of the direct questions about the reasons for nonparticipation (Questions

E1a to E1p).

The most frequently cited reasons for nonparticipation by respondents from both working and elderly

households were related to a perceived lack of need for food stamps. Substantial proportions of

nonparticipants gave “can get by on my own,” “feel others need food stamps more,” and “need is only

temporary” as reasons for nonparticipation. Because of the frequency that reasons related to a lack of need

for nonparticipation are given, it is important that a questionnaire about nonparticipation include questions

to ascertain:

C Whether the lack of need is real or whether the respondent has a need that they do not admit

to, perhaps because of embarrassment or other factors

C If the respondent does not need food stamps, whether this is because they receive food

assistance from other sources. 
These nine respondents comprise 7 percent of all elderly nonparticipants interviewed. 36

116

Hence, it is important that the questionnaire include questions about both food security and sources of other

food assistance.

A belief that they are ineligible for food stamps is an important reasonworking and elderly households

do not participate in the program. This underlines the importance of collecting data to determine whether

the households are correct in their belief that they are ineligible. We also recommend retaining the followup questions that ask why respondents think they are ineligible and whether they were told by someone

in a FSP office that they were ineligible, and if they were, how long ago they were told. 

We asked in the long questionnaires whether the respondent had heard of food stamps or the FSP

before they were interviewed for the pretest. We included this question in the long questionnaire only. We

were surprised to find that 12 percent of respondents from elderly households asked this question reported

that they did not know about the existence of the FSP. This is an important enough reason that the

36

question should be included in any questionnaire about the reasons for nonparticipation. 

The pretest findings shown in Table IV.1 highlight the importance of asking respondents for “the most

important reason for not participating” from among the factors that play a role in the decision not to

participate. Over 70 percent of all nonparticipants gave more than one reason for not participating. 

Asking the respondent for the most important reason provides some information about the importance of

each reason. As an example, 45 percent of nonparticipants from working households expect that they are

eligible for only a small amount of benefits, and nearly 24 percent cite this as a reason they do not

participate, but only 5 percent say it is the most important reason they do not participate.

The findings in Table IV.1 also confirm that it is not necessary to ask respondents whether a given

factor was an “important reason” they did not participate (Question sequence E2a through E2p). The
117

distributions for whether the reason was applicable and “whether it was an important reason” are similar

for most reasons. We recommend deleting sequence E2a through E2p from the questionnaire since its

inclusion does not add much information. 

a. Characteristics of Pretest Nonparticipants Giving Particular Reasons for Currently Not

Participating in the FSP

To obtain a deeper understanding of the reasons for current nonparticipation, both descriptive tabular

analysis and logit regression analysis can be used to assess whether some reasons are more important than

others for certain subgroups of nonparticipants. First, comparative tabular analysis can be used to

contrast the characteristics and past experiences of nonparticipants who give a particular reason for

nonparticipation with those of nonparticipants who do not give it as a reason. It is also useful, but not

essential, to provide the distribution of the characteristics of FSP participants as a benchmark.

Logit regression can be used to identify subgroups of nonparticipants most likely to give particular

reasons for nonparticipation. Using the example above, variables such as age, gender, education level,

whether the respondent had previously received food stamps, the physical and cognitive functioning of the

respondent, and other characteristics and experiences are included in the regression equations as

independent variables. The difference between the tabular and logit analyses is that the latter identifies the

independent effect that a given respondent characteristic or experience has on the likelihood of giving a

particular reason for nonparticipation, controlling for other measured respondent characteristics and

experiences.

Figure IV.1 summarizes some of the key characteristics and experiences that could be considered in

the tabular and regression analyses designed to explain how various reasons for nonparticipation differ

across nonparticipants, assuming that data will be collected using the long 
118

 FIGURE IV.1

ANALYSES OF REASONS FOR FOOD STAMP PROGRAM NONPARTICIPATION BY 

KEY CHARACTERISTICS AND EXPERIENCES OF NONPARTICIPANTS

Dependent variable: “Don’t know about existence of the FSP” 

Key Characteristics:

C Don’t know anyone who applied for FSP benefits

C Had not received food stamps as a child (long version only)

C Had not applied in the past

C Education

C Cognitive functioning (long version, elderly only)a

C Age, gender, race/ethnicity 

C Citizenship (long version only)

Dependent variable: “Don’t know where or who to contact about how to apply for FSP benefits”

Key Characteristics:

C Don’t know anyone who applied for FSP benefits

C Had not received food stamps as a child (long version only)

C Had not applied in the past

C Education

C Cognitive functioning (long version, elderly-only)

C Age, gender, race/ethnicity

C Citizenship (long version only)

Dependent variable: “Don’t think eligible for FSP benefits”

Key Characteristics: 

C Have been found ineligible in past

C Amount of FSP benefit (expected) 

C Income (long version only)

C Home ownership (long version only)

C Vehicle ownership (long version only)

C Presence of working person (working only)

C Age, gender, and race/ethnicity

C Citizenship (long version only)

Dependent variable: “Can get by on my own without FSP benefits”or “Feel others need food stamp benefits more

than me”

Characteristics:

C Whether food secure (more measures available on long version than short version)

C Whether receive food assistance from other sources (long version only)

C Whether have social supports (long version only)

C Household composition (more details in long version)

C Whether believe others need food assistance more than their household does

C Whether receive benefits from other government programs, such as SSI (long version only)

C Income (long version only)

C Age, gender, race/ethnicity

C Citizenship (long version only) 
FIGURE IV.1 (continued)

119

Dependent variable: “Need is only temporary”

Characteristics: 

C Whether think will be working at same job three months from now (long version only)

C Earning more or less three months from now (long version only)

C Measures of stability of job (long version only)

C Whether believe others need food stamps more than their household does

C Age, gender, race/ethnicity

C Citizenship (long version only)

Dependent variable: “FSP benefits are too low”

Characteristics:

C Potential benefits a

C Expected benefits b

C Perceive it is hard or costly to get to food stamp office

C Perceive application process is long and complicated

C Age, gender, race/ethnicity

C Citizenship (long version only)

Dependent variable: “Hard to get to the FSP office”

Characteristics:

C Whether live in rural area

C Whether own vehicle (long version only)

C Whether have physical mobility limitations

C Whether need to take time off work

C Whether lose pay when apply

C Whether have health problems (long version, elderly only)

C Whether have dependents in household

C Age, gender, race/ethnicity

C Citizenship (long version only)

Dependent variable: “Application process too long and complicated”a

Characteristics: 

C Cognitive functioning (long version, elderly only)

C Whether have health problems (long version, elderly only)

C Whether have past experience applying for food stamps and perceived there to be problems with application

process

C Age, race/ethnicity, and gender

C Citizenship (long version only)

C Education

Dependent variable: “Questions too personal”a

Characteristics: 

C Household composition (more details in the long version)

C Amount and sources of income (long version only)

C Age, race/ethnicity, and gender

C Citizenship (long version only)

C Education
FIGURE IV.1 (continued)

120

Dependent variable: “FSP office staff disrespectful”

Characteristics:

C Age

C Urban vs. rural location

C Income (long version only)

C Race/ethnicity

C Gender

Dependent variable: Psychological reasons for nonparticipation

Characteristics:

C Age, gender, race/ethnicity

C Whether received food stamps as a child

C Education

C Income (long version only)

C Residential location (urban vs. rural)

C Whether participated in the past

NOTES: 

 Analyses depicted above assume that data will be collected using the long versions of the questionnaires. If FNS a

opts for the shorter versions, then not all of the subgroups or independent variables can be constructed and

included in the comparative tabular and regression analyses. We have noted in the table those variables available

only in the long version of the questionnaires.

 The amount of benefits the respondent would receive if they participated in the FSP. It is calculated from b

household size and income information collected in the questionnaires.

 The amount of benefits the respondent thinks they would get if they participated in the FSP. c
This replicates the index used by Hamilton et al. (1997). 37

versions of the questionnaires. Not all of the subgroups or independent variables can be constructed and

included in the comparative and regression analyses if the short versions of the questionnaires are used.

We have noted in the figure those variables available only in some versions of the questionnaires. 

Table IV.2 illustrates how we would present the tabular analysis for the reasons related to a lack of

need for food stamps. We present some distributions of characteristics of all participants, all

nonparticipants, and all nonparticipants who give one or more of three reasons related to a lack of need

(“can get by on my own,” “others need them more,” or “my need is only temporary”) as the most important

reason for nonparticipation. Similar analyses could be performed for nonparticipants who gave a lack of

need as an applicable factor.

The food security questions in the questionnaires can be used to construct a “food security” scale.37

 Households are classified as either food secure or falling into one of three categories of food insecurity:

food insecure without hunger, food insecure with moderate hunger, and food insecure with severe hunger.

Of the nonparticipants from working households who reported that they did not receive food stamps

for a reason related to a lack of need, a significant proportion seem to be food insecure. These pretest

findings emphasize the importance of collecting food security data.

Respondents who are food insecure over a 12-month period may not be food insecure over a shorter

period. For example, respondents may have been food insecure six months ago, but since then, they have

had sufficient food. Hence, we cannot conclude that respondents who perceive they don’t currently need

food stamps are currently food insecure based on the data collected by the questionnaires. Given the

policy relevance of the food security questions, we recommend that the questions cover a shorter period
122

TABLE IV.2

SELECTED CHARACTERISTICS OF NONPARTICIPANTS WHO REPORTED THAT THE MOST IMPORTANT REASON FOR

NONPARTICIPATION WAS RELATED TO A LACK OF NEED FOR FOOD STAMPSa

(Percentage Distributions)

Working Elderly

Nonparticipants Nonparticipants b b

Participants Nonparticipants Important Reason Participants Nonparticipants Important Reason

All Gave it as the Most All Gave it as the Most

Nonparticipants Who Nonparticipants Who

c c

Food Securityd

Food secure 33 47 65 69 77 92

Food insecure

without hunger 28 27 23 22 15 4

Food insecure with

moderate hunger 22 11 3 9 1 0

Food insecure with

severe hunger 0 6 5 0 3 0

Missing 17 10 5 0 4 4

e

FSP Participation

History

Received FSP

benefits in past three

years 100 19 11 100 10 6

Applied for benefits

in past three years 100 19 7 100 6 2

Relatives, friends,

neighbors, or

coworkers receive

FSP benefits 27 24 21 8 11 11
TABLE IV.2 (Continued)

Working Elderly

Nonparticipants Nonparticipants b b

Participants Nonparticipants Important Reason Participants Nonparticipants Important Reason

All Gave it as the Most All Gave it as the Most

Nonparticipants Who Nonparticipants Who

c c

Family received FSP

benefits when

respondent was 54 36 31 22 26 11

child

Age of Respondent

Less than 30 34 33 39 1 3 2

31 to 59 60 58 53 12 13 11

60 to 69 0 2 1 39 34 32

70 to 79 3 2 3 39 29 32

80 and older 0 0 0 8 18 21

Missing data 3 5 4 1 2 2

Gender

Male 11 29 33 19 27 30

Female 89 71 67 81 73 70
TABLE IV.2 (Continued)

Working Elderly

Nonparticipants Nonparticipants b b

Participants Nonparticipants Important Reason Participants Nonparticipants Important Reason

All Gave it as the Most All Gave it as the Most

Nonparticipants Who Nonparticipants Who

c c

124

Education

Primary or less 5 3 5 15 18 17

Some high school 15 20 21 32 25 17

High school 58 41 41 36 39 47

Vocational degree 0 2 0 0 2 4

One to three years

of college 15 20 21 11 9 6

At least 4 years of

college 6 12 11 3 5 8

Other 0 1 0 3 1 0

Missing data 1 1 0 1 2 2

Citizenship

Household all U.S.

citizens 91 89 90 97 100 100

Household includes

some non U.S.

citizens 6 5 2 0 0 0

No citizens in

household 3 6 8 3 0 0
TABLE IV.2 (Continued)

Working Elderly

Nonparticipants Nonparticipants b b

Participants Nonparticipants Important Reason Participants Nonparticipants Important Reason

All Gave it as the Most All Gave it as the Most

Nonparticipants Who Nonparticipants Who

c c

125

Difficulty

Managing Money

No difficulty n.a. n.a. n.a. 72 78 88

Some difficulty n.a. n.a. n.a. 22 15 12

A great deal of

difficulty n.a. n.a. n.a. 6 5 0

Unable to do n.a. n.a. n.a. 0 1 0

Sample Size 73 177 75 75 126 53

NOTES: 

Data in this table should not be used to make inferences about the characteristics of these nonparticipants nationally. Because the data were collected in

a

a pretest, the samples are small and not nationally representative. 

Includes nonparticipants who, based on information collected in the main questionnaire, were probably ineligible for food stamp benefits. b

All nonparticipants who reported that the most important reason for nonparticipation was either that they could get by on their own, that other people needed c

food stamps more than they did, or that their need was only temporary.

Coded from the long questionnaires only. d

Most of the missing data was a result of interviewer error administering the complex skip logic. e

n.a. = not asked
126

of time. The CPS used both a 12-month and 30-day time period. We recommend changing the time frame

from “12-months” to the “past 30 days” in future administrations of the questionnaires.

b. Additional Data on Underlying Reasons for Nonparticipation

For six of the potential reasons for nonparticipation, the questionnaires ask additional follow-up

questions about the underlying reasons for nonparticipation. Additional details are asked about the reasons

respondents thought:

1. Their household was ineligible. Respondents were asked why they thought they were

ineligible. They were also asked about how they arrived at that perception--whether staff at

a FSP office told them they were ineligible, and if so, when they were told, whether someone

at another program thought they were ineligible, or whether they formed that opinion based

on the circumstances of someone they knew who was like them.

2. It was hard or costly to get to the food stamp office to apply. In a series of closedended questions, respondents were asked why they thought it would be hard to get to the

food stamp office. Specific reasons asked about include: transportation difficulties or

expenses, physical difficulties, difficulties getting time off work, loss of pay when visiting the

FSP office, and the need to care for someone in their home.

3. The application process was too long and complicated. In a series of closed-ended

questions, respondents were asked whether they thought they would have to wait a long time

to be served, whether they thought the application form was too long and complicated, and

whether they thought it would be difficult to get all the necessary paperwork. For those who

said they thoughtthey would have to wait a long time, the questionnaires asked how long they

thought they would have to wait.

4. The FSP office is an unpleasant place. Respondents were asked in closed-ended

questions whether they thought the office was unpleasant because of the inside of the building,

the other people in the waiting room, or because of the neighborhood the office was in.

5. The benefits they were entitled to receive are too low. Respondents who thought the

benefits were too small were asked whether they were told they were eligible for only a small

amount of benefits by someone at a food stamp office, and if they were, how long ago they

were told this; whether they were told they were eligible for only a small amount of benefits

by someone at another program; and whether they based their opinion on a comparison with

someone else they knew who was like them. We also asked all nonparticipants who thought
127

they were eligible for food stamps the amount of food stamp benefits they thought they were

eligible for.

6. The questions on the application form were too personal. In an open-ended question,

respondents were asked what types of questions they thought were too personal.

Tables IV.3 through IV.8 provide examples of how these data can be summarized. 

In a large-scale survey, for most reasons for nonparticipation, there would probably be a sufficient

number of respondents who say that the reason is applicable to be able to analyze the responses to the

more detailed questions about the reasons. More than 10 percent of nonparticipants would be asked each

of these more detailed questions. For all questions except those that follow-up on the reasons why it is

hard or costly to get to the food stamp office, more than 20 percent of nonparticipants are asked the

questions. We do not recommend dropping the questions about why the FSP office is an unpleasant place

because they provide information useful to the FSP.

Given that so many respondents gave reasons related to a lack of need for food stamps, we

recommend that a future survey would include closed-ended follow-up questions about this perceived lack

of need. For example, it may be informative to ask whether they don’t need food stamps because they

receive food assistance from family or friends, because they receive other benefits, or because they go

without medications or paying bills. 

2. Reasons Current Nonparticipants Stopped Receiving Food Stamps, Started But Did Not

Complete an FSP Application, and Received But Did Not Use Food Stamps

Additional perspective can be gained on the reasons some households do not participate in the FSP

by examining any previous experience nonparticipants may have had with the program. So that the

respondents can recall their experiences, we ask about only the previous three years. Section C of the

nonparticipant questionnaires provides information on the reasons former participants stopped 
128
129

TABLE IV.3

REASONS NONPARTICIPANTS THINK THEY ARE INELIGIBLEa

(Percentage Distributions)b

Nonparticipants Who Thought They Were 

Ineligible or Did Not Know If They Were Eligiblec

Reason for Perception Working Elderly

Income too high 73 41

Assets too high 4 4

Missing or incomplete paperwork 0 0

Do not meet citizenship requirements 0 0

Do not satisfy work requirements 0 1

On strike from job 0 0

Student 1 0

Thinks they are categorically ineligible

because they work 6 3

Thinks they are categorically ineligible for

some other reason 8 6

d

Other 15 21 e

Don’t know 7 27

Sample Size 113 78

NOTES:

Data in this table should not be used to make inferences about the reasons nonparticipants think they are ineligible

a

nationally. Because the data were collected in a pretest, the samples are small and not nationally representative.

Totals exceed 100 percent because respondents could give multiple responses. b

Includes nonparticipants who, based on information collected in the main questionnaire, were probably ineligible c

for food stamp benefits.

Includes persons who said they did not know why they were categorically ineligible. d

Nearly all these responses did not appropriately answer the question. For example, when asked why they thought e

they were not eligible, some respondents answered “I don’t need food stamps.”
130

TABLE IV.4

REASONS NONPARTICIPANTS THINK IT IS HARD OR COSTLY 

TO GET TO THE FSP OFFICEa

(Percentage Distributions)b

Nonparticipants Who Thought It Would be Hard

or Costly to Get to the FSP Officec

Reason for Perception Working Elderly

Difficult or expensive to get transportation 45 55

Physical difficulties/mobility limitations 55 72

Difficult to take time off from work/school 50 3

Would lose pay going to the food stamp office 35 14

Would have to arrange for someone to take care of

someone in your home 25 14

Don’t know the location of the FSP office 0 3

Concerns about safety 0 3

Other 1 0

Don’t know 0 7

Sample Size 20 29

NOTES:

Data in this table should not be used to make inferences about the reasons nonparticipants think it is hard or

a

costly to get to the FSP office nationally. Because the data were collected in a pretest, the samples are small and

not nationally representative. 

Totals exceed 100 percent because respondents can give multiple responses. b

Includes nonparticipants who, based on information collected in the main questionnaire, were probably ineligible c

for food stamp benefits.
131

TABLE IV.5

REASONS NONPARTICIPANTS THINK THE FSP APPLICATION

 PROCESS IS TOO LONG AND COMPLICATEDa

(Percentage Distributions)b

Nonparticipants Who Thought the Application Process

Was Too Long and Complicatedc

Reason for Perception Working Elderly

Have to wait a long time to be served 73 56

Thought would have to wait:d

15 minutes or less 0 5

16-30 minutes 7 16

31 minutes to one hour 35 11

More than one hour 59 42

Don’t know 0 26

Application form too long and complicated 71 74

Difficult to get all the necessary paperwork 49 61

Application process too long 2 0

Disabilities 0 6

Other 0 3

Don’t know 0 3

Sample Size 41 31

NOTES:

Data in this table should not be used to make inferences about the reasons nonparticipants think the FSP

a

application process is too long and complicated nationally. Because the data were collected in a pretest, the

samples are small and not nationally representative. 

Totals exceed 100 percent because respondents can give multiple responses. b

Includes nonparticipants who, based on information collected in the main questionnaire, were probably ineligible c

for food stamp benefits.

Calculated only for the respondents who thought they would have to wait a long time to be served. d
132

TABLE IV.6

REASONS NONPARTICIPANTS THINK THE FSP OFFICE IS UNPLEASANT OR UNSAFEa

(Percentage Distributions)b

Nonparticipants Who Perceive the FSP Office As

Unpleasant or Unsafec

Reason for Perception Working Elderly

Inside of building is physically unpleasant 38 31

Don’t like waiting with the other applicants 29 37

FSP office is in unsafe neighborhood 43 56

Staff are unpleasant or disrespectful 19 19

Lack of parking 0 6

Other 10 19

Don’t know 5 6

Sample Size 21 16

NOTES:

Data in this table should not be used to make inferences about the reasons nonparticipants think the FSP office a

is unpleasant or unsafe nationally. Because the data were collected in a pretest, the samples are small and not

nationally representative. 

Totals exceed 100 percent because respondents can give multiple responses. b

Includes nonparticipants who, based on information collected in the main questionnaire, were probably ineligible c

for food stamp benefits.
133

TABLE IV.7

SOURCES OF PERCEPTIONS OF ELIGIBILITY FOR ONLY SMALL BENEFITa

(Percentage Distributions)

Nonparticipants Who Think They are Eligible for

a Small Benefit Amountb

Source of Perception Working Elderly

Someone at the FSP office told the respondent that

his/her household was eligible for only small amount 14 37

Told by FSP staff:c

Within last 3 months 10 7

Between 3 and 12 months ago 36 27

More than one year ago 45 66

Don’t know/missing 9 0

Told by someone at another program that the

household was ineligible 3 2

Thought eligible for small amount because know

someone like them that receives only a small amount

of benefits 14 17

Sample Size 80 41

NOTES:

Data in this table should not be used to make inferences about the reasons some households think they are eligible

a

for only a small benefit amount nationally. Because the data were collected in a pretest, the samples are small

and not nationally representative. 

Includes nonparticipants who, based on information collected in the main questionnaire, were probably ineligible b

for food stamp benefits.

Distribution calculated for only those respondents who were told by the FSP office that they were eligible for

c

a small amount of food stamp benefits.
134

TABLE IV.8

REASONS NONPARTICIPANTS THINK THE FSP APPLICATION IS TOO PERSONALa

(Percentage Distributions)b

Nonparticipants Who Think The FSP Application

Is Too Personalc

Questions/Subjects That are Too Personal Working Elderly

Composition of household 17 18 d

Citizenship 3 0

Disabilities 3 0

Resources/assets 39 21

Income sources 33 29 e

“FSP wants to know everything about one’s life” 0 11

Other 17 25

Sample Size 36 28

NOTES:

Data in this table should not be used to make inferences about the reasons nonparticipants think the FSP

a

application is too personal nationally. Because the data were collected in a pretest, the samples are small and not

nationally representative. 

Totals exceed 100 percent because respondents can give multiple responses. b

Includes nonparticipants who, based on information collected in the main questionnaire, were probably ineligible c

for food stamp benefits.

The child’s father’s residence was frequently cited as especially personal by respondents in nonelderly

d

households.

Whether the household receives child support payments was frequently cited as especially personal by

e

respondents in nonelderly households.
135

receiving food stamp benefits; Section D provides data on the reasons individuals who contacted the FSP

office or began the application process did not complete an application. It also asks respondents who

applied for and were found eligible to receive food stamps why they did not use their food stamps. 

The percentages of nonparticipants who have had previous experiences with the FSP are reported

in Table IV.9. About 19 percent of respondents from working households and 10 percent of elderly

respondents had received food stamps in the previous three years. Thus the sample sizes of nonparticipants

who have previously received food stamps in a national survey would be large enough to support an

analysis of the these nonparticipants’ experiences with the FSP.

The samples of nonparticipants who, in the previous three years, began an application for food stamps

without completing it are smaller, comprising only 6 percent of respondents from working households and

1 percent of respondents from elderly households. A further 2 percent of respondents from working

households and 2 percent of respondents from elderly households had contacted the FSP office but not

completed the application. We recommend dropping the questions about why persons who have been

found eligible for food stamps did not use them as only 2 percent of respondents from working and elderly

households had not used food stamps they had received.

We illustrate how we would present data on the reasons nonparticipants stopped receiving food

stamps in Table IV.10. The questionnaires contain questions that would allowa more in-depth examination

of the reasons former participants discontinued participation. For those saying they were told they were

ineligible by FSP staff, we can examine the reasons they were found ineligible. We can also examine when

they were told they were ineligible. This is relevant because if nonparticipants were told they were no

longer eligible many months ago, it is possible that their circumstances may have changed and they are now

eligible. Similarly, we can examine th e
136

TABLE IV.9

NONPARTICIPANTS’ FSP EXPERIENCES OVER THE PREVIOUS THREE YEARSa

(Percent Distributions)

Nonparticipantsb

Experience Working Elderly

Applications

Have contacted FSP office to ask about benefits, but did not apply 2 2

Have begun the application process but did not complete it 6 1

Have completed the application process 19 6

Have completed the application process but was found ineligible 7 1

Have been found eligible but did not use food stamps 2 1

Participation

Have received food stamp benefits 19 10 c

Have stopped receiving food stamps because found ineligible 7 8

Sample Size 177 126

NOTES:

Data in this table should not be used to make inferences about FSP nonparticipants nationally. Because the data a

were collected in a pretest, the samples are small and not nationally representative. 

Includes nonparticipants who, based on information collected in the main questionnaire, were probably ineligible b

for food stamp benefits.

The percentage of households receiving food stamps is not necessarily equal to the percentage completing

c

applications because the household may have been found ineligible at the application or the application may have

occurred more than three years ago. 
137

TABLE IV.10

REASONS REPORTED BY NONPARTICIPANTS FOR DISCONTINUING 

FOOD STAMP RECEIPT IN THE PREVIOUS THREE YEARSa 

(Percentage Distributions)b

Reason Stopped Receiving Food Stamps Working Elderly

Not Eligible

Notified by the FSP that no longer eligible 33 33

Perceived Ineligibility

Thought no longer eligible for FSP benefits 50 21

Perceived Lack of Need

Thought no longer needed food stamps 23 25

Thought situation would improve 5 0

Other people needed them more 0 0

FSP Benefits Too Low

Think not worth the effort to continue participating because

benefit level is too low 14 25

Program Features and Administration

Hard to get to FSP office to do paperwork to continue

receiving benefits 0 13

Process needed to go through to continue to receive

benefits too long and complicated 5 13

Questions needed to answer to continue to receive benefits

too personal 0 13

Not treated well by FSP staff 5 0

Office is very unpleasant 0 0

Office located in an unsafe neighborhood 0 13
TABLE IV.10 (Continued)

Reason Stopped Receiving Food Stamps Working Elderly

138

Stigma

Other family members no longer approved of respondent

receiving food stamps 0 0

Felt embarrassed using food stamp benefits 0 0

Did not like relying on government for assistance 0 0

Other Reasonsc 32 8

Sample Size 22 12

NOTES:

Data in this table should not be used to make inferences about the reasons nonparticipants discontinued food

a

stamp receipt nationally. Because the data were collected in a pretest, the samples are small and not nationally

representative. The nonparticipants include nonparticipants who, based on information collected in the main

questionnaire, were probably ineligible for food stamp benefits.

Percentages sum to more than 100 percent because respondents could give more than one reason for

b

discontinuing FSP participation.

 

The majority of “other reasons” given by respondents had to do with the household’s failure to comply with FSP c

rules or staff requests or an increase in the household’s income that meant they were no longer eligible. 
139

reasons respondents who said they thought they were no longer eligible felt that they were ineligible. For

respondents who said they quit participating because benefits were too low, we can tabulate the amount

of benefits they reported receiving the last month of their food stamp spell. 

Data on the reasons why some nonparticipants had begun the application but not completed it, and

the reasons why some nonparticipants received food stamps but did not use them could be presented in

tables similar to Table IV.10.

D. COMPARISONS OF CHARACTERISTICS AND EXPERIENCES OF FSP

PARTICIPANTS AND NONPARTICIPANTS

The analyses described in the previous section are based on the direct reports by respondents about

why they did not participate in the FSP. The questionnaires also collect information on the characteristics

and experiences of both FSP participants and nonparticipants. This allows a more indirect approach to

analyzing the reasons for nonparticipation--comparing the characteristics and experiences of participants

and nonparticipants. The remainder ofthis section discusses how the questionnaires allow a comparison

of participants and nonparticipants on (1) personal and household characteristics; (2) past experiences with

the FSP; and (3) attitudes and other factors that may facilitate or hinder program participation. 

1. Comparisons of Characteristics of Participants and Nonparticipants

Datafrom the questionnaires will support comparisons of FSP participants and nonparticipants across

several personal and household characteristics. Table IV.11 provides an example of simple descriptive

tabular comparisons of participants and nonparticipants for selected demographic characteristics and the

receipt of food assistance from other sources. The distributions are presented 
140

TABLE IV.11

SELECTED CHARACTERISTICS OF PARTICIPANT AND NONPARTICIPANT HOUSEHOLDSa

(Percentage Distributions)

Working Elderly

Characteristic Participants Nonparticipants Participants Nonparticipants b b

Respondent Characteristics

Age

Less than 30 27 32 0 2

30 to 59 67 59 14 14

60 to 69 0 2 39 34

70 to 79 3 2 39 29

80 and older 0 0 8 19

Missing data 3 5 1 2

Mean 34 37 66 67

Median 33 36 68 69

Gender

Male 11 29 19 27

Female 89 71 81 73

Race/Ethnicity

Nonhispanic Black 38 32 31 27

Nonhispanic White 48 49 61 65

Hispanic 8 12 5 5

Other 5 3 3 1

Missing data 0 4 0 2

Education

Primary or less 5 3 15 18

Some high school 15 20 32 25

High school 57 41 36 39

Vocational training program certificate 0 2 0 2

Some college 15 20 11 9

Two-year or four-year college degree 6 12 3 5
TABLE IV.11 (Continued)

Working Elderly

Characteristic Participants Nonparticipants Participants Nonparticipants b b

141

Other 0 0 3 1

Missing data 1 1 1 2

Marital Status

Married or living as married 34 37 7 27

Divorced/separated 28 25 34 37

Widowed 4 6 36 18

Never been married 34 35 24 18

Household Characteristics

Household Size

1 person 0 15 73 52

2 16 27 10 29

3 23 21 10 10

4 14 15 1 5

5 or more 47 23 5 5

Mean 4.5 3.2 1.6 1.9

Median 4 3 1 1

Children Present

Yes 90 59 13 14

No 10 41 87 85

Citizenshipc

Household members all U.S. citizens 91 89 98 97

Household includes some members

who are non U.S. citizens 6 4 1 0

No members of household are U.S.

citizens 3 7 1 3

Receipt of Food Assistance from

Other Sources

Congregate or home-delivered meals 0 0 4 5

Day-care or Head Start program 11 3 0 4
TABLE IV.11 (Continued)

Working Elderly

Characteristic Participants Nonparticipants Participants Nonparticipants b b

142

School Lunch Program 36 12 4 5

School Breakfast Program 26 8 1 2

Women, Infants, and Children (WIC) 8 1 1

Program 23

Vouchers to get food 7 3 3 1

Food or money for food from friends

or relatives 12 15 7 9

Emergency food from church or food

pantry 12 7 11 8

Emergency food from soup kitchens 1 1 0 2

Meals from work 5 3 0 0

Food from garden 8 8 5 10

Food from animals raised by

respondent 0 3 0 1

Food from hunting or fishing 8 7 1 6

Other sources 0 2 1 2

Sample Size 73 177 75 126

NOTES:

Data in this table should not be used to make inferences about the characteristics of FSP participant and nonparticipant a

households nationally. Because the data were collected in a pretest, the samples are small and not nationally

representative. 

Includes nonparticipants who, based on information collected in the main questionnaire, were probably ineligible for

b

food stamp benefits.

Calculated for respondents to long versions of the questionnaires only. c
143

separately for working and elderly participants and working and elderly nonparticipants. The characteristics

of the respondents that we could present in tables similar to Table IV.11 and the rationale for collecting

data on these characteristics are described below.

a. Demographic Characteristics

Both the long and the short questionnaires contain questions on demographic characteristics. All

questionnaires ask about the age, gender, race/ethnicity, education, and marital status of respondent. The

screening interviews collect information on household size. The long questionnaires also contain a full

household roster that asks the age and relationship to the respondent of everyone in the household. They

also ask about the citizenship of household members. 

Comparing demographic characteristics of participants and nonparticipants may be informative

because some potential reasons for nonparticipation may be related to household composition. For

example, respondents with less education may find it difficult to find out about the program and how to

apply. Households with children may be more likely to participate because adults may be willing to go

without food but not willing for their children to do so. Household size may be important for two reasons:

smaller households are eligible for lower benefits than larger households, but they face the same costs of

applying for and obtaining food stamps; second, larger households can buy food at a lower unit cost, hence,

food stamps may be of greater value to them than to smaller households.

b. Economic Characteristics

Economic characteristics are collected in the long questionnaires only. The characteristics collected

by the questionnaires include: the amount and sources of household income, including earnings; medical

expenses for elderly/disabled members of household; dependent-care expenses; home ownership; and
144

vehicle ownership. There are at least two reasons for nonparticipation to be connected to the household’s

total income and other economic resources. First, households with greater resources may perceive that

they do not need food stamps. Second, such households may not know that they are eligible or may

believe that they are ineligible. 

While data on vehicle ownership and expenses are collected mainly to make a determination of FSPeligibility for nonparticipants, wealso collected these data for participants. The rationale for collecting these

data was that there may be interesting differences between participants and nonparticipants. We believe

that these data are of secondary importance, and the questions on expenses and vehicle ownership could

be dropped from the participant questionnaires.

c. Receipt of Food Assistance Other Than Food Stamps

Only the long versions of the questionnaires collected data on other sources of food assistance. Types

of other food assistance asked about include: congregate or home-delivered meals (elderly households

only); free or reduced priced breakfasts or lunches from SchoolBreakfast, National School Lunch, or day

care or HeadStart programs;food through the WIC or other programs; meals or food from food pantries,

food banks, or soup kitchens; food or meals from friends or relatives; and food or meals received as part

of employment. Comparisons of the receipt of food assistance from sources other than food stamps

betweenparticipants and nonparticipants may provide information about whether nonparticipants have less

of a need for food stamps than participants because they have access to more sources of food assistance

and whether access to one food assistance program facilitates access to another.
The person in the household who works the most number of hours per week. 38

145

d. Food Security 

The responses to questions about Food Security, in both the long and short versions of the

questionnaires, can be used to determine whether the respondent’s household is food secure, and if not,

the degree of food insecurity. Comparisons of food security between participants and nonparticipants may

shed light on whether respondents who say they do not need food stamps are food secure and, more

generally, whether those who do not participate are in need of food assistance.

e. Characteristics of Employment

Information on employment is collected by both the working and elderly long questionnaires. No

information on employment is collected by the short questionnaires except how many persons in the

household work and whether the respondent works. The elderly long questionnaire only collects

information on earnings for each member of the household. As well as information on earnings, the long

working questionnaires also collect information on:

C The occupation of the principal earner in the household38

C How long the principal earner has been working at his or her present job

C How likely it is that the principal earner will still be at the same employment in three months

C Whether the principal earner expects his or her earnings to change 

C The number of different jobs held by the principal earner in the past year

C The number of months the principal earner was unemployed over the past year
146

Each of these pieces of information provides some information on the stability of employment. One

difference between participants and nonparticipants may be that nonparticipants have more stable

employment. Persons who have employment that is likely to end shortly may be more likely to participate

than persons with the same income who have more stable employment. 

f. Health Characteristics 

Only the long elderly questionnaires collect information on the respondent’sself-assessed health status

and cognitive and physical functioning. Questions related to health were included because it is sometimes

argued that elderly persons have low participation rates because of health problems that make it difficult

for them to apply for and/or use benefits. 

g. Social Supports 

Questions on the social supports available to the respondent are included in all the long questionnaires.

 Three questions are asked to assess the extent of the social supports available to the respondent. These

collect information on the:

C Length of time the respondent has lived in his or her neighborhood

C Frequency the respondent meets with friends

C Distance from the respondent’s nearest relative

These characteristics are included because social supports, by indicating the extent to which family and

friends can act as a safety net to the respondent, may be an important factor in determining participation.
147

h. FSP Benefit Level 

Data on the actual amount of food stamp benefits received by participants are collected in the

screening interviews. Using data collected by the long questionnaires, we can also estimate the amount of

food stamp benefits nonparticipants would receive if they participated (the potential benefits). Using these

data, we can compare the benefits of participants with the benefits of nonparticipants to see whether

nonparticipants on average would receive a smaller benefit than participants.

2. Comparisons of Participant and Nonparticipants on Their Past Experiences with the FSP

If issues related to the administration of the FSP are reasons why some persons eligible for food

stamps do not participate, we might expect participants and nonparticipants to have different experiences

with the FSP. The questionnaires ask both participants and nonparticipants about their experiences

applying for and using food stamps over the past three years. 

For respondents who have previously completed an application form for food stamps, we can

compare their experiences applying and using food stamps along the following dimensions:

C Reasons the respondent applied for food stamps or contacted the FSP office

C The logistics of applying for food stamps: where the respondent filed the application form;

whether the respondent applied for other benefits at the same time; whether the respondent

applied or whether an authorized representative applied for him or her; where the certification

interview was held; whether the respondent took time off work to apply; the types of help

with the application process the respondent received; and whether the help with the

application was received from FSP staff or others 

C Whether the respondent has been found ineligible in the previous three years, and the reasons

he or she was found ineligible

C Experiences and difficulties applying, such as being treated disrespectfully by FSP staff;

difficulties meeting FSP caseworkers; losing wages from missing work; and feeling humiliated

applying for food stamps
148

C Experiences and difficulties receiving or using food stamps, such as being treated

disrespectfully by store personnel or other customers

C Whether the respondents changed their shopping habits or other actions so that people would

not find out they received FSP benefits 

We illustrate how these participant and nonparticipant experiences would be tabulated in Tables IV.12 and

IV.13.

In the pretest, nonparticipants who did not complete an application form were not asked the full

sequence of questions about their experiences applying (Questions D4a through D7k). However, these

people may have had particular difficulties with the application process. Hence, we recommend asking all

nonparticipants who have contacted the FSP or begun an application form the full sequence of questions

about their experiences with the application process. In the pretest, these questions were skipped for the

4 percent of nonparticipants who had begun an application form but not completed it.

For those individuals who in the recent past have contacted the FSP but never applied, applied but

did not complete the application, or who have been found eligible but did not use food stamps, we can

compare FSP participants and nonparticipants on the:

C Reasons they did not complete an application form after contacting the FSP

C Reasons they did not complete the application process 

C Reasons they were found ineligible at application 

C Reasons households found eligible did not use food stamps 
149

TABLE IV.12

SELECTED EXPERIENCES OF RESPONDENTS WHEN APPLYING FOR FOOD STAMPSa 

(Percentage distribution of households applying for food stamps during the past three years)

Working Elderly

Experience Participants Nonparticipants Participants Nonparticipants b b

Difficult or expensive getting transportation to the FSP office 15 21 13 *

Health or disability made it difficult to get to FSP office 15 33 24 *

Difficult to take time off work to apply for food stamp benefits 19 12 1 *

Lost wages when took time off work to apply for food stamps 19 27 1 *

Needed to arrange for dependent care to apply for food stamps 21 15 3 *

Had to wait a long time to be served at the food stamp office 31 55 17 *

Food stamp office staff were disrespectful 16 27 11 *

Application form was too long and complicated 21 33 20 *

Caseworker asked questions that were too personal 4 18 11 *

It was difficult to get all the necessary paperwork for the application 26 39 16 *

Felt embarrassed having to apply for food stamps 27 45 15 *

Missing data 3 0 1 *

Sample Size 73 33 75 7

NOTES:

Data in this table should not be used to make inferences about the experiences of households applying for food stamps nationally. Because the data were collected

a

in a pretest, the samples are small and not nationally representative. 

Includes nonparticipants who, based on information collected in the main questionnaire, were probably ineligible for food stamp benefits. b

* = insufficient number of cases to calculate distribution. 
150

TABLE IV.13

SELECTED EXPERIENCES OF RESPONDENTS USING FOOD STAMPSa

(Percentage distribution of respondents participating in the FSP during the previous three years)

Working Elderly

Experience Participants Nonparticipants Participants Nonparticipants b b

Sometimes treated disrespectfully by either employees or other customers

when using food stamp benefits in stores 29 21 17 17

Embarrassed to use food stamp benefits 16 18 12 33

Had difficulties obtaining monthly food stamp benefits 10 15 8 17

Felt needed food stamps to make it through the month 96 97 92 92

Had difficulties doing all the paperwork needed to keep getting food 21 36 20 25

stamps

Had difficulties arranging meetings with caseworker at convenient times 22 39 12 25

Sometimes treated disrespectfully by food stamp office staff 27 42 8 17

Other 7 15 8 8

Missing data 0 0 1 8

Sample Size 73 33 75 12

NOTES:

Data in this table should not be used to make inferences about the experiences of households using food stamps nationally. Because the data were collected in a

a

pretest, the samples are small and not nationally representative. 

Includes nonparticipants who, based on information collected in the main questionnaire, were probably ineligible for food stamp benefits. b
151

The percentages of respondents who would answerthe questions about the reasons for not completing

the application and not using food stamps are small (see Table IV.9). Hence, we caution that questions

about the reasons respondents did not complete the application process should only be included if the

sample sizes are large enough to support the comparisons of these reasons between participants and

nonparticipants. As noted in Section C, as so few nonparticipants in either working or elderly households

did not use food stamps after being found eligible, we recommend dropping the questions about the reasons

respondents did not use food stamps.

3. Comparisons of Participants and Nonparticipants on Factors that May Influence Barriers

to FSP Participation

The questionnaires ask participants and nonparticipants who have previously applied for food stamps

about factorsthat helped them overcome real or perceived barriers to participation. These factors include:

C They had lots of help with the FSP application process

C Family and friends were supportive of the decision to participate

C They feel it is alright to receive FSP benefits because they pay taxes

C They need FSP benefits or they won’t have enough to eat

C They are uncomfortable getting food fromfamily, friends, charities, or other food assistance

programs

Table IV.14 illustrates how the findings about factors that influence barriers to participation could be

presented.

The question about the role of family and friends in the decision to participate is worded differently in

the participant and nonparticipant questionnaires. Participants were asked “Did otherfamily members or

friends encourage you to get food stamp benefits?” whereas nonparticipants were
152

TABLE IV.14

FACTORS THAT MAY HELP POTENTIAL PARTICIPANTS OVERCOME 

BARRIERS TO PARTICIPATIONa

(Percent Distribution of Households)b

Working Elderly

Factor Participants Nonparticipants Participants Nonparticipants c c

Had lots of help with

application process 15 12 33 43 d d

Family and friends were

(would be) supportive of

decision to participate 26 83 16 91 e

Feel it’s O.K. to receive FSP

benefits because pay taxes 63 67 67 52

Need FSP benefits or won’t

be able to get enough food 60 21 60 16

Was (would be)

uncomfortable getting food

from family, friends, charities,

or other programs 47 28 17 n.a.

Other 11 n.a. 15 n.a.

Sample Size 73 177 75 126

NOTES:

Data in this table should not be used to make inferences about the factors that help participants overcome

a

barriers to participation nationally. Because the data were collected in a pretest, the samples are small and not

nationally representative. 

bTotals do not add to 100 percent because respondents can indicate that more than one factor helped them

overcome barriers to participation.

Includes nonparticipants who, based on information collected in the main questionnaire, were probably ineligible c

for food stamp benefits.

Only nonparticipants who completed FSP applications within the past three years were asked this question. d

This question was worded differently for participants and nonparticipants. Participants were asked “Did other

e

family members or friends encourage you to get food stamp benefits?” whereas nonparticipants were asked

“Would other family members or friends discourage you from using food stamps?”

n.a. = question not asked
We recommend that question D17b be deleted and that a question “Did other family members or

39

friends discourage you from using food stamp benefits?” be added as C14h in the participant

questionnaires.

153

asked “Would other family members or friends discourage you from using food stamp benefits?” Because

of the differences between the wording of the questions, direct comparisons should not be made between

the responses of participants and nonparticipants. To allow the comparison, we recommend that

participants are asked the same question as nonparticipants.39

E. COMPARISON OF FINDINGS BY ADMINISTRATION MODE

In the pretest 15 percent of the interviews were administered in-person. We found no significant

differences in the responses to questions by mode of administration. Table IV.15 presents the reasons

reported by nonparticipants for not participating in the FSP by whether the interview was administered by

telephone or in-person. The distributions of reported reasons are similar for each mode of administration. 

F. SUMMARY AND IMPLICATIONS

This chapter illustrated the depth of information that could be obtained from fielding the questionnaires

nationally. The questionnaires, incorporating the recommended revisions identified in this chapter and

Chapter III, would collect a wealth of information about the reasons for nonparticipation. 

1. Recommended Modifications to the Questionnaires

The chapter indicated some questions that could be added, some that could be dropped, and some

that could be changed:
154

TABLE IV.15

REASONS REPORTED BY NONPARTICIPANTS FOR CURRENTLY NOT PARTICIPATING 

IN THE FSP, BY INTERVIEW MODEa

(Percentage Distributions)

Nonparticipants Who Reported Reason

as Applicableb

Reason Telephone In-Person

Lack of Information

Don’t know FSP exists 8 0

Don’t know where to go or who to contact 42 29

Don’t think eligible for food stamps 37 47

Perceived Lack of Need

Can get by on my own without FSP benefits 82 75

Feel others need FSP benefits more 78 81

Need is only temporary 50 47

Expected FSP Benefits Too Low

Think eligible for only a low benefit amount 43 31

Problems Related to Program Administration

Hard to get to FSP office 17 14

Application process is too long and complicated 24 25

Questions are too personal 23 17

FSP office staff are disrespectful 18 22

FSP office is unpleasant or unsafe 13 11

Psychological Reasons

Other family members or friends would not approve of

respondent receiving food stamps 7 8

Would feel embarrassed applying for FSP benefits 27 19
TABLE IV.15 (Continued)

Nonparticipants Who Reported Reason

as Applicableb

Reason Telephone In-Person

155

Would feel embarrassed using FSP benefits 20 31

Dislike relying on the government for assistance 40 47

Other Reasons 9 14

Sample Size 153 24

NOTES:

Data in this table should not be used to make inferences about the reasons households do not participate in the

a

FSP nationally. Because the data were collected in a pretest, the samples are small and not nationally

representative. 

Includes nonparticipants who, based on information collected in the main questionnaire, were probably ineligible b

for food stamp benefits.
156

C A question about whether the respondent had ever heard of food stamps or the FSP before

the survey interview should be included in all questionnaires about the reason for

nonparticipation.

C It is not necessary to include the series of questions that ask whether a reason is an important

reason why a respondent did not participate (Questions E2a to E2p) if the respondent is

asked for the most important reason why he or she does not participate.

C The questions about food security should refer to a 30-day period rather than a 12-month

period.

C Follow-up questions should be added for nonparticipants who give a reason related to a lack

of need for food stamps, such as whether the respondents do not need food stamps because

they receive food assistance from family or friends, because they receive other benefits, or

because they go without medications or paying bills.

C Questions about the reasons why some people who are found eligible for food stamps do

not use them should be deleted as they are applicable to less than 2 percent of

nonparticipants from working or elderly households.

C The nonparticipant questionnaires should ask all respondents who have contacted or begun

the application process about their experiences applying for food stamps.

C Questions about whether theworkers in the household worked their “usual” hours over the

past month should be added to the questions on earnings. Interviewer checks for whether

the earnings exceed 130 percent of poverty should also be included.

C The questions about expenses and vehicles could be deleted from the participant

questionnaires. 

2. The Information Obtained from the Direct Questions Will Be More Informative than the

Information Obtained from Comparisons between Participants and Nonparticipants

An analysis of the responses to the direct questions about why nonparticipants do not participate will

yield sufficient information on which to base policy recommendations. In contrast, comparisons of

participants and eligible nonparticipants yield findings that are suggestive of reasons for nonparticipation but

rarely provide evidence that a specific reason is important. 
157

3. The Long Versions of the Questionnaires Could Be Shortened

We believe that FNS could get a wealth of information about the reasons for nonparticipation from

a questionnaire slightly shorter than the long versions used in the pretest. The shortening would help raise

response rates and reduce survey costs. Specifically, we recommend revising the long-version of the

questionnaires to exclude:

C Questions indirectly related to the respondent’s knowledge of the FSP, such as whether the

family ever received food stamps when the respondent was a child and whether any

relatives, friends, neighbors, or coworkers receive food stamps

C Questions about employment, other than earnings

C Questions about health and cognitive and physical functioning

C Questions about social supports, such as how long the respondent has lived in the

neighborhood

These questions collectinteresting information, but the information is less informative about the reasons for

nonparticipation than the othermore direct questions about the reasons for nonparticipation. If collected,

we would use the information to compare the characteristics of participants and nonparticipants, and the

characteristics of nonparticipantswho give different reasons for nonparticipation. However, it is difficult

to infer reasons for nonparticipation from these types of comparisons--most differences between

participants and nonparticipants would be consistent withmore than one reason for nonparticipation. We

think that a questionnaire without these questions could yield sufficient information to make policy

recommendations.
158

We expect that a questionnaire without these questions and the modifications described in Chapter

III would take about 20minutes to administer by telephone. Including the screening interview, the whole

interview would take less than 24 minutes. 
159

4. The Minimum Set of Information that Should Be Collected by a Survey on the Reasons for

Nonparticipation

FNS may consider fielding thissurvey as an additional module to an existing survey rather than a

stand-alone survey. In this case, the length of the questionnaire will be limited. We believe that any survey

on the reasons for nonparticipation should include at a minimum: 

C The Direct Questions about the Reasons for Nonparticipation. We recommend that

a survey on the reasons for nonparticipation include the questions contained in Section E of

the questionnaires used in the pretest.

C The Food Security Questions. It is critically important that any survey on the reasons for

nonparticipation collect data on food security. If the majority of nonparticipants are

estimated to be food insecure, than this suggests nonparticipation is a problem, as those

needing food assistance are not being reached by the program. However, if most are food

secure, then the FSP may be meeting its mission of providing food assistance to those who

need it and low rates of nonparticipation are not a cause for concern.

C Questions on Income. It is important that detailed information on income and earnings is

collected in a survey of the reasons for nonparticipation. This will allow us to conduct the

analyses on samples of respondents who not only pass the screens in the short screening

interview but also when asked detailed questions about income, have household income

consistent with eligibility in the FSP. Including persons in the sample who are ineligible can

bias the results. Estimates of the proportion of nonparticipants who are not participating

because they think they are ineligible and because they don’t need food stamps will both be

biased upwards if we include persons who are ineligible for food stamps in the sample. 

C Questions on the Receipt of Food Assistance from Sources other than the FSP.

Given the high proportion of nonparticipants who gave reasons for nonparticipation related

to a lack of need for food stamps, it is important to ascertain whether nonparticipants who

say they don’t need food stamps have access to other sources of food assistance.

These questions could be administered in 15 minutes. If the complete screening interview was also

administered, the whole module on nonparticipation would take less than 20 minutes to administer.
Specific recommendations about changes to individual questions are provided at the end of Chapters

40

II, III, and IV and in Appendix A.

160

V. CONCLUSIONS AND RECOMMENDATIONS

Conducting a survey about the reasons for nonparticipation in the FSP poses three main challenges.

First, it is difficult to locate persons to interview who are eligible for food stamps but do not receive them.

Because no lists exist of these eligible nonparticipants, a random-digit-dialing (RDD) frame is required.

Second, the questions at the beginning of the interview that screen out persons not eligible for the survey

need to strike a balance between collecting detailed and sensitive information to accurately determine

eligibility and minimizing nonresponse. Third, the questionnaires need to collect sufficient information about

the reasons for nonparticipation to inform policy decisions. The purpose of the pretest was to investigate

whether these challenges could be met.

This chapter presents the main conclusions from the pretest and our recommendations for fielding the

survey on a national scale.40

A. IT IS FEASIBLE TO CONDUCT A STAND-ALONE SURVEY ON THE REASONS FOR

NONPARTICIPATION

The pretest showed that it is feasible to conduct a stand-alone survey about the reasons for

nonparticipation. However, the survey would require considerable survey resources, mainly because of

the difficulties identifying survey respondents. Using RDD, we called nearly 17,000 telephone numbers to

identify 484 eligible nonparticipants and 92 participants from working or elderly households. We estimate

that it would take over 18,000 hours of interviewer labor to identify a sample of about 1,000 eligible

nonparticipants from working households and 1,000 eligible nonparticipants from elderly households.
Massey et al. (1997). 41

161

B. A LIST FRAME IS NEEDED IF THE SURVEY IS TO INCLUDE FSP PARTICIPANTS

If RDD isbeing used to identify nonparticipants, identifying participants at the same time requires little

additional interviewer time. However, because we found participants to be rarer than FSP-eligible

nonparticipants among working and elderly households, it would take more calls to identify a FSP

participant than itwould to identify a FSP-eligible nonparticipant. Unless the survey design calls for a ratio

of nonparticipants to participants of above six for working households and above four for elderly

households, the target for the number of nonparticipants in the sample would be reached before the target

for the number of participants. Once the target for nonparticipants is reached, identifying additional

participants using RDD will be many times more costly than identifying the participants using a list frame.

Hence, if it is decided that the survey should include participants, a mixed-frame design would be the most

efficient one.

C. A FINAL RESPONSE RATE OF ABOUT 65 PERCENT COULD BE ACHIEVED ON A

STAND-ALONE SURVEY ON THE REASONS FOR NONPARTICIPATION

Nonresponse is a concern because the persons who do not respond to a survey may differ from those

who do in ways that are related to the reasons for nonparticipation. If the factors that determine whether

a person responds are related to the reasons for nonparticipation, the survey findings will be biased--the

observed findings will differ from the findings that we would have observed if there had been no

nonresponse. 

The response rate to a stand-alone survey about the reasons for nonparticipation is unlikely to be high

for three reasons. First, response rates to RDD surveys are typically low. Use of answering machines,

call-forwarding, and telephone solicitation all contribute to low response ratesto RDD surveys. Obtaining

41
Massey et al. (1997). 42

162

a response rate above 70 percent for an RDD survey is rare. Second, the RDD screening interview

42

includes questions about household income andassets--questions that are both difficult and sensitive. Third,

nonparticipants may be uninterested in topics related to a program that they have chosen not to participate

in.

The overall response rate to the pretest survey was about 51 percent--the response rate to the

screening interviews was just under 60 percent; the completion rate to the main questionnaires was 85

percent. One factor that lowered the response rates in the pretest was that two of the eight pretest sites

were large northeastern metropolitan cities that typically have low survey response rates. With the

recommended changes to the survey discussed below, we think the response rate to the screening

interviews in a national survey could be as high as 70 percent and the completion rate to the main

questionnaire as high as 90 or 95 percent, yielding an overall response rate of 63 to 67 percent. 

To improve the response rates, we recommend the following changes to the pretest.

1. Change the Order of the Questions on the RDD Screening Interview. The first

interview questions should be nonthreatening questions that are related to the topic of the

questionnaire. Instead of asking first about household size and income, we recommend

asking about the respondent’s participation in the FSP and then whether his or her household

contains a working or an elderly person. The interviewer could then ask the respondent

whether he or she has ever received food stamps. Only after these questions should the

interviewer ask about the respondent’s income. 

2. Add More Interviewer Probes to the Screening Interview. We found in the pretest that

including interviewer probes after an initial nonresponse to the income question was

successful at eliciting responses. We recommend adding similar probes after the asset

questions. We also suggest probing respondents about inconsistent responses to questions

about the receipt of food stamps before counting them as nonresponders.
163

3. Send an Advance Letter to Persons on the RDD Sample Frame. We recommend that

before calling, persons on the RDD sample frame who have listed telephone numbers and

addresses are sent a letter that explains the study and encourages their participation.

4. Lengthen the Field Period. Our experience has shown that increasing the length of the

field period can increase the response rate primarily by increasing the number of telephone

numbers for which the residential status can be determined.

5. If a List-Frame is Used, Obtain Current Lists of FSP Participants As Quickly as

Possible from the FSP Offices. Contact information on FSP participants can get out-ofdate fast. The importance of obtaining the data quickly should be emphasized to FSP

agency staff.

6. Conduct In-Person Follow-Up to Locate Persons on the List Frame. Some persons

who cannot be located by telephone may be located by an interviewer going in-person to

the respondents’ addresses.

7. Use Commercial Services to Obtain More Locating Information on Persons Listed

on the List-Frame. Commercial services can provide telephone numbers, changes of

addresses, and telephone numbers of neighbors for some persons on the list-frame.

8. Decrease the Frequency of Changing Respondents Between the Screening

Interview and the Main Questionnaire. Response rates are lower if the main

questionnaire is administered to someone in the household other than the person who was

administered the screening interview. In the pretest, we administered the main questionnaire

to the person in the nonparticipant households who would apply for food stamps if the

household decided to participate and the person who last applied for food stamps in the

participant households. For 17 percent of the interviews, this person was not the person

who responded to the screening interview. We recommend relaxing this definition, and

administering the main questionnaire to any adult in the household who may apply for food

stamps.

9. Shorten the Main Questionnaire. The completion rate for the short-version of the

questionnaires conducted by telephone was 91 percent compared with 85 percent for the

long questionnaires. We recommend a main questionnaire to be used in a stand-alone

survey about the reasons for nonparticipation that would be about 5 minutes shorter than the

long questionnaire which took on average 26 minutes to administer. The short

questionnaires took an average of 15 minutes to administer.
164

D. THE RDD SCREENING INTERVIEWUSED IN THE PRETEST STRIKES THE RIGHT

BALANCE BETWEEN DETERMINING ELIGIBILITY AND MINIMIZING

NONRESPONSE

Most previous studies of the reasons for nonparticipation in the FSP used crude screening rules to

create samples of nonparticipants who were likely to be eligible for food stamps (McConnell and Nixon

1996). The RDD screening interview used in the pretest used more sophisticated screening rules that

required respondents to answer questions about their income, vehicles, and assets. Even so, we estimate

that 38 percent of the respondents found eligible by the RDD screening interview seem to be ineligible for

food stamps based on information given later in the interview. 

When designing a screening interview, there is a fine line between developing an interviewthat makes

a good determination of FSP eligibility and one that asks so many detailed and sensitive questions that its

response rate is unacceptably low. We believe the screening interview used in the pretest, with the

modifications described in Chapter II, hits about the right balance between the two objectives of keeping

the interview short and simple and doing a good job of predicting FSP eligibility.

E. THE SCREENING INTERVIEWS AND MAIN QUESTIONNAIRES SHOULD BE

ADMINISTERED USING COMPUTER ASSISTED SURVEY METHODS

In the pretest, the screening interviews were conducted using CATI, but because of the small sample

sizes, the main questionnaires were administered using hard-copy instruments. A national survey about

nonparticipation should use computer assisted survey interviews (CASI) for both the screening interviews

and the main questionnaires. In the RDD screening interview, CASI supports sample management and

scheduling, aids the interviewer in conducting complex skip logic, and automatically determines whether

the respondent is eligible for the sample. CASI also helps in the administration of the main questionnaires,

which include complex skip logic, with some questions being asked only of persons with past experience
Resource constraints prevented including an in-person screening component in the pretest. 43

165

in the FSP and other questions being asked only of persons who report specific reasons for

nonparticipation.

F. A SURVEY ABOUT THE REASONS FOR NONPARTICIPATION SHOULD INCLUDE

A SMALL SAMPLE OF HOUSEHOLDS WITHOUT TELEPHONES AND ASK ABOUT

TELEPHONE INTERRUPTIONS 

The pretest sample included only households with working telephones. However, estimates from the

Censussuggest that about 6 percent of all households and about 20 percent of low-income households do

not have working telephones. Because households without telephones may have different reasons for not

participating than households with telephones, a national survey of the reasons for nonparticipation should

include a small sample of respondents without telephones. 

Respondents without telephones would be administered both a screening interview and the main

questionnaire in-person. Interviewers would go door-to-door and administer both the screening interviews

and the main questionnaires in-person. We foundin the pretest that the main questionnaires worked equally

well either by telephone orin-person. The screening interviews were successfully cognitively tested in43

person (Ponza et al. 1997).

Considerable survey resources would be needed to identify householdswithout telephones that meet

our criteria for inclusion in the sample. Working and elderly households that are eligible for food stamps

but do not receive them comprise less than 8 percent of the population. As non-telephone households tend

to have lower income than households with telephones, the proportion of eligible nonparticipants in nontelephone households may be even lower. Hence, the proportion of working or elderly nonparticipant
166

households without telephones in the population is probably less than 2 percent of the population. It would

take considerable interviewer time to find even a small number of eligible non-telephone households. 

Because of the cost of identifying eligible non-telephone households, the sample of non-telephone

households would need to be small and designed to yield a national estimate of the reasons for

nonparticipation by non-telephone households,rather than regional estimates. To reduce survey costs,

screening for non-telephone households would take place in areas with a high concentration of nontelephone households. The cost of the in-person survey would be lower if it is conducted when the 2000

Census data are available and areas with a high concentration of non-telephone households can be

identified with more up-to-date information. 

The required sample size of non-telephone households can be reduced by collecting data about past

interruptions of telephone service from households bothwith and without telephones. Many households

lose and gain telephone service during the year--having telephone service when they can afford it and

having it disconnected when they cannot. Some households that do not have telephones at a specific point

of time have recently had telephone service and, conversely, some households with telephone service at

a point of time may have had interruptions in the past. In the 1993 National Household Education Survey

between 9 and 12 percent of households with telephones at the time of the interview reported interruptions

of telephone service of one day or more in the previous year (Brick et al. 1996). We would expect higher

percentages of households with interruptions in their telephone service in low-income populations. 

Under the assumption that non-telephone households and households with interruptions in telephone

service have similar reasons for not participating in the FSP, data on interruptions in telephone service can

be used to statistically adjust for the under-representation of non-telephone households (Keeter 1995).

We recommend including questions in the main questionnaires that ask whether the respondent’s household
167

has experienced an interruption in telephone service over the previous 12 months of more than one day and

if so, how long the interruption lasted.

We also recommend that both FSP participants and FSP-eligible nonparticipant non-telephone

households be administered the main questionnaires. Our concern is that it will be particularly difficult to

find nonparticipants among FSP-eligible non-telephone households because non-telephonehouseholds tend

to have lower income and lower-income households are more likely to participate in the FSP (McConnell

and Nixon 1996). By interviewing both participant and nonparticipant respondents in non-telephone

households information would be collected to determine the FSP participation rate among non-telephone

households. It may be that the participation rate is so high among non-telephone households, that

nonparticipation among non-telephone households is not a concern.

G. A QUESTIONNAIRE ABOUT NONPARTICIPATION SHOULD INCLUDE CLOSEDENDED STRUCTURED QUESTIONS ABOUT NONPARTICIPATION

In previous surveys, the questions about nonparticipation were typically broad and open-ended and

elicited responses that were too vague to inform policy decisions (McConnell and Nixon 1996). Hence,

we designed questionnaires in which there was a series of direct closed-ended questions about the reasons

for nonparticipation. Each question asked whether a particular reason was applicable to the respondent.

At the end of the series of questions, we asked whether there were other reasons that the respondent did

not participate in the FSP. We also asked which was the most important reason that the respondent did

not participate. For some reasons, more detailed follow-up questions were asked. These closed-ended

questions worked well. On their own, they could provide much detailed information about the reasons for

nonparticipation.
168

H. A QUESTIONNAIRE ABOUT NONPARTICIPATION SHOULD INCLUDE QUESTIONS

TO DETERMINE FSP ELIGIBILITY, FOOD SECURITY, AND SOURCES OF OTHER

FOOD ASSISTANCE

It is important in any analysis of the reasons for nonparticipation that the sample contains only persons

who are eligible for food stamps. Including persons who are not eligible for food stamps will bias the

findings. For example, when asked why they do not participate in the FSP, persons who are ineligible for

food stamps are more likely to say that they think (correctly) that they are ineligible for food stamps and

that they do not need food stamps. 

Because of concerns about the response rate, the screening interview cannot ask all the detailed

questions required to accurately determine FSP eligibility. However, some of these questions can be asked

later in the main interview when the interviewer has established rapport with the respondent. When

analyzing the survey data, this information can be used to make a better determination of FSP eligibility and

sample members who do not seem to be FSP eligible can be removed from the analysis sample. 

It is important that any survey on the reasons for nonparticipation collect data on food security. The

purpose of the FSP is to provide food assistance to all those who need it. If the majority of nonparticipants

are found to be food insecure, this suggests that some persons needing food assistance are not being

reached by the program. However, if most nonparticipants are food secure, the FSP may be meeting its

mission of providing food assistance to those who need it, and low rates of participation are not a cause

for concern.

The findings from the pretest suggested that the most important reason that both working and elderly

households do not participate in the FSP is a perception of a lack of need for food stamps. From a policy

perspective, it would be interesting to determine whether the lack of need is because nonparticipants are
169

receiving food assistance from other sources, such as other government programs, charities, family, or

friends.

I. THE QUESTIONNAIRESDEVELOPED FOR THIS STUDY SHOULD BE REVISED TO

ADDRESS FSP NONPARTICIPATION ISSUES ARISING FROM RECENT WELFARE

REFORM

The recent decline in FSP participation, from 11 million households in 1996 to 8 million households

in 1998, cannot be fully explained by a decrease in the number of households in poverty. This suggests that

changes associated with the welfare reform provisions of the Personal Responsibility and Work

Opportunity Reconciliation Act of 1996 may have decreased FSP participation rates. Of the two

population groups of interest in this study, welfare reform is more likely to have affected working

households, who because they have often have children, may have received TANF. Elderly households

are not directly affected by welfare reform but may have been affected indirectly, perhaps through changes

in attitudes toward welfare. Although a one-time survey cannot fully explore the reasons for changes in

the FSP participation rate, we recommend that the questionnaires be revised to include questions that

explore the link between FSP participation and welfare reform.

Welfare reform may have affected the FSP participation rate in three main ways. First, FSP

participants who stop receiving TANF benefits because they find work, reach the time limit, or are

sanctioned for not meeting work requirements may discontinue receipt of food stamp benefits even though

they may still be eligible to receive them. This may be because they think incorrectly that their households

are no longer eligible for food stamp benefits, they no longer think it is worth the “hassle” to receive just

food stamp benefits, or they want to be free of receiving any government assistance. Caseworkers may

also not be informing clients that they are still eligible for food stamp benefits, and in some cases, may even

be incorrectly terminating benefits. We recommend that when respondents are asked about why they
170

stopped receiving food stamps benefits in the survey, they should be asked whether they stopped receiving

TANF benefits at about the same time. If they did, follow-up questions can explore how the cessation of

TANF affected their FSP participation. The respondents can be asked whether they thought their

households were no longer eligible for food stamp benefits, and if so, whether this perception was based

on information provided by the caseworker or some other source. 

Second, welfare reform may have reduced the likelihood that a person applying for TANF would also

apply for food stamp benefits. Although food stamp applications can still be made at the same time as

TANF applications, local caseworkers may be less likely to tell applicants that they are eligible for food

stamp benefits. Many states have developed diversion programs that discourage people from going onto

TANF. Some diversion programs offer TANF applicants a one-time lump-sum payment instead of

monthly TANF benefits; others require applicants to engage in job search as a condition to be met before

they are eligible for TANF. Applicants diverted from TANF may not be aware that they are still eligible

for food stamp benefits. Caseworkers may not be informing them about food stamp benefits and may even

be improperly holding up the processing of FSP applications until the applicant has completed preapplication proceduresfor TANF. This suggests that the questionnaires should ask respondents not only

whether they have recently applied for food stamp benefits, but also whether they have recently applied

for TANF benefits. Respondents who have recently applied for TANF benefits should be asked follow-up

questions about whether they were told about their eligibility for food stamp benefits when they applied for

TANF benefits and if they were encouraged by eligibility workers to apply for food stamp benefits also.

Third, welfare reform may have increased the importance of psychological reasons for not participating

in theFSP. Welfare reform transformed AFDC from an entitlement program which provided monthly cash

assistance to a transitional assistance program (TANF) oriented toward getting people off welfare and into
171

employment. This transformation may have increased working and elderly peoples’ desire for selfsufficiency and increased the stigma of applying for and using food stamp benefits. The questionnaires

already include questions about whether the respondent did or would feel embarrassed applying for and/or

using food stamp benefits. For those respondents who report feeling a stigma associated with applying for

and/or using food stamp benefits, follow-up questions in the questionnaires could ask whether the

respondents feel that the stigma associated with food stamp benefits has increased in recent years.

Nonparticipants could be asked whether their desire to be self-sufficient was important in their decision to

not participate in the FSP.

 

J. A 20-MINUTE MAIN QUESTIONNAIRE WOULD COLLECT SUFFICIENT

INFORMATION ABOUT NONPARTICIPATION TO MAKE POLICY DECISIONS

Ideally, a survey of nonparticipation would include all the questions included in the long versions of the

questionnaires. However, because of concerns of respondent burden and response rates, we recommend

instead using a slightly shorter version of the long questionnaires that would take about 20 minutes to

administer. The questionnaire would still include the full set of questions about the reasons for

nonparticipation (with the modifications suggested in Chapters III and IV), questions about food security

and sources of food assistance, questions about the demographic composition of the household, and

questions about income, expenses, and vehicles. It would also include the questions about previous

experiences respondents have had applying for and using food stamps. We would delete from the long

versions questions about topics that do not add significantly to our knowledge about the reasons for

nonparticipation, including:

C Questions indirectly related to the respondent’s knowledge of the FSP, such as whether the

family ever received food stamps when the respondent was a child and whether any

relatives, friends, neighbors, or coworkers receive food stamps
Although a survey of non-telephone households should include participants. 44

172

C Questions about employment, although we would retainthe questions about earnings needed

to determine FSP eligibility 

C Questions about health and cognitive and physical functioning

C Questions about social supports, such as how long the respondent has lived in the

neighborhood

K. CONSIDERATION SHOULD BE GIVEN TO EXCLUDING PARTICIPANTS FROM

THE SURVEY

If the resources available to conduct a survey onFSP nonparticipation are limited, we recommend

that FNS consider conducting a telephone survey of only nonparticipants. Conducting a survey of 44

nonparticipants only would provide sufficient information to provide guidance on the appropriate policy

response to the nonparticipation. While including participants in the survey would add a richness to the

data collected, the information collected from the participants may not justify the additional survey

resources required to include them in the survey.

We designed the questionnaires to allow two lines of inquiry into the reasons for nonparticipation: (1)

a direct approach--asking nonparticipants why they do not participate in the program, and (2) an indirect

approach--making statistical comparisons of the characteristics of participants and nonparticipants. The

direct approach--asking people directly why they do not participate--is the more informative and does not

require that any participants are surveyed. Comparisons of participants and nonparticipants yield findings

that are suggestive of reasons for nonparticipation but rarely provide firm evidence that a particular reason

is important. Some reasons that respondents give for nonparticipation, such as the view that other people

may need food stamp benefits more, would not be uncovered by just comparing the characteristics or
173

experiences of participants and nonparticipants. The direct approach also allows a ranking ofthe relative

importance of each reason for nonparticipation.

Ifresources permitted including participants in a survey of nonparticipation, including them would yield

two main benefits. First, comparing the characteristics and experiences of participants and nonparticipants

provides information about the type of people that are more likely to participate. This may be useful in

targeting efforts to increase participation to specific population groups. Second, including participants

allows the survey to explore what factors help participants overcome perceived or real barriers to applying

for or using food stamp benefits that the nonparticipants do not overcome. For example, by comparing the

experiences of participants and nonparticipants who began the application process but did not complete

it, we could investigate whether completing the application process at a place other than the FSP office

increases the likelihood that the application process is completed. 

Many of these comparisons between participants and nonparticipants can, however, be made with

existing survey data. Comparisons between participants and nonparticipants of household demographic

and economic characteristics, sources of other food assistance, and food security can be made using the

SIPP and the Food Security Supplement to the CPS (McConnell and Nixon 1996). Comparisons of

experiences applying for and using food stamp benefits between participants and nonparticipants can be

made using the National Food Stamp Program Survey. Collecting information on participants and

nonparticipants in the same survey has the advantage, however, that the data are directly comparable. This

would be important if the reasons for nonparticipation are changing over time because of changes in the

FSP or other assistance programs, changes in the economy, or changes in attitudes toward welfare.
174

L. CONSIDERATION SHOULD BE GIVEN TO ADDING A MODULE ABOUT FSP

NONPARTICIPATION TO ANOTHER HOUSEHOLD SURVEY

An alternative to conducting a stand-alone survey would be to add a short module about the reasons

for FSP nonparticipation to another household survey. The module would begin with the screening

questions, and only persons who meet the criteria in the screening interview would then be asked questions

about nonparticipation. At a minimum, we recommend a 15- to 20-minute module that includes: 

C The screening questions

C The direct questions about nonparticipation included in Section E of the questionnaires

C Questions about whether the respondent has previously applied for or used food stamps

C Questions about food security, sources of food assistance, and income, by source

Adding a module to an existing household survey would significantly reduce the costs of collecting the

data since only the additional costs associated with a longer interview would be incurred. Depending on

the survey to which the module is added, the response rate to the questions may also be higher. Also, if

the main household survey contains detailed questions about income disaggregated by source, the screener

for the add-on module could use this information rather than the respondent’s estimate of aggregate

household income to determine whether the respondent is likely to be eligible for food stamps. This would

allow the screening questions to determine FSP eligibility more accurately, as income is less likely to be

underreported when the respondent is asked about income by source (Citro and Michael 1995). 

The main disadvantage of an add-on module rather than a stand-alone survey is that the module must

be short. Moreover, the sample size of FSP-eligible nonparticipants in working or elderly households could

be small. Also, the survey design and data collection procedures that were designed for another survey

may not be optimal for the module on the reasons for nonparticipation. 
175

A module about the reasons for nonparticipation could be added to any nationally-representative

household survey that contains a large-enough sample of low-income households to ensure sufficient

numbers of FSP nonparticipants who are in either working or elderly households. Ideally the household

survey would also collect data on monthly income, including earnings, disaggregated by source and data

on food sufficiency and sources of other food assistance.

The SIPP is a good example of an existing nationally-representative household survey to which a

module on the reasons for FSP nonparticipation could be added. Periodically, modules of special interest

are attached to the SIPP. For example, the Extended Well-Being Module was administered to Wave 6

of the 1991 SIPP panel and Wave 3 of the 1992 SIPP panel. McConnell and Nixon (1996) estimated

that this module was administered to over 3,700 FSP nonparticipants with income less than 130 percent

of poverty and nearly 1,900 FSP participants. Of these, there were about 1,500 working nonparticipants,

1,500 elderly nonparticipants, 600 working participants, and 400 elderly participants. The samples of

nonparticipants are large enough to conduct an analysis of the reasons for nonparticipation given by

nonparticipants.

Another advantage of the SIPP is that it already collects detailed income information, including

earningsinformation, information on FSP participation, and information on the age of household members.

A nonparticipation module would need to include screening questions only about assets and vehicles.

Interviewers could use information reported earlier in the interview to determine whether income exceeded

130 percent of poverty, whether the household was participating in the FSP, and whether the household

contained an elderly or a working person. The SIPP also collects information about receipt of food

assistance from other government programs. However, its core files do not include questions about food
The March supplement to the CPS contains asset information and detailed income information. It

45

could be linked to the nonparticipation module (even if administered in another month) to make a more

accurate determination of FSP-eligibility for households that were in both samples.

176

security. So a nonparticipation module would also need to include a set of questions to determine the

household’s level of food security.

The CPS is another example of a nationally-representative survey to which a module on the reasons

for nonparticipation could be added. The sample sizes of eligible nonparticipants are quite large.

McConnell and Nixon (1996) found that the 1995 CPS Food Security Supplement contains 10,000

respondents with household income less than 130 percent of poverty. Of these, over 7,000 were

nonparticipants, over 4,000 working nonparticipants, and nearly 3,000 elderly nonparticipants. One

disadvantage of the CPS is that the core interview only collects income information by asking respondents

to report their aggregate household income in categories of $2,499 or more. A second disadvantage

45

of the CPS is that its core questionnaires do not collect data on FSP participation, food security, or other

sources of food assistance.
177

REFERENCES

Brick, J. M., J. Waksberg, and S. Keeter. “Using Data on Interruptions in Telephone Service as

Coverage Adjustments.” Survey Methodology, vol. 22, no. 2, December 1996.

Brunner, G. Allen, and Stephen J. Carroll, Jr. “The Effect of Prior Telephone Appointments on

Completion Rates and Response Content.” The Public Opinion Quarterly, vol. 31, 1969.

Citro, Constance, and Robert T. Michael. Measuring Poverty: A New Approach. Washington, DC:

National Academy, 1995.

Cohen, Rhoda, Donna Eisenhower, Sheena McConnell and Michel Ponza. “Reaching the Working

Poor and Poor Elderly: Cognitive Test Planning Memorandum.” Washington, DC: Mathematica

Policy Research, Inc., December 1996.

Cohen, Steven B. and Barbara Lepidus Carlson. “An Analysis of the Characteristics of Reluctant

Respondents in the National Medical Expenditure Survey.” Paper presented at the Annual Meeting

of the American Statistical Association. Boston, 1992.

Council of American Survey Research Organizations (CASRO). “Special Report: On the Definition

of Response Rates.” Report of the CASRO Completion Rates Task Force, New York, Audits and

Surveys Co. Inc., unpublished paper, 1982.

 

Dillman, Don A., Jean Gorton Gallegos, and James H. Frey. “Reducing Refusal Rates for Telephone

Interviews.” The Public Opinion Quarterly, vol. 40, no. 1, Spring 1976.

Dillman, Don A. Mail and Telephone Surveys. The Total Design Method. Pullman, WA: John

Wiley and Sons, 1978.

Frey, James H. Survey Research by Telephone. Newbury Park, CA: Sage Publications, 1989.

Gleason, Philip,Peter Schochet, and Robert Moffitt. “The Dynamics of Food Stamp Program Participation

in the Early 1990s.” InCurrent Perspectives on Food Stamp Program Participation. Alexandria,

VA: U.S. Department of Agriculture, Food and Nutrition Service, April 1998.

Hamilton, William L., John T. Cook, William W. Thompson, Lawrence F. Buron, Edward A. Frongillo,

Jr., Christine Olson, and Cheryl A. Wehler. “Household Food Security in the United States.”

Technical Report on the Food Security Measurement Project. Washington, DC: Food and Consumer

Service, U.S. Department of Agriculture, 1997.

Keeter, S. “Estimating noncoverage bias from a phone survey.” The Public Opinion Quarterly, vol.

59, 1995.
178

Kristal, Alan R., Emily White, Julie R. Davis. “Effects of Enhanced Calling Efforts on Response

Rates, Estimates of Health Behavior, and Costs in a Telephone Health Survey Using Random-Digit

Dialing.” Public Health Reports, vol. 108, no. 3, May-June 1993, p. 372-379.

Massey, James T., Dan O’Connor, and Karol Krotki. “Response Rates in Random Digit Dialing

(RDD) Telephone Surveys. Proceedings of the American Statistical Association, Section on Survey

Research Methods, forthcoming 1997.

McConnell, Sheena. “Predicting Food Stamp Program Eligibility UsingSurvey Data: What Information

is Needed?” Washington, DC: Mathematica Policy Research, Inc., July 1997.

McConnell, Sheena and Lucia Nixon. “Reaching the Working Poor and the Poor Elderly: Report

on Literature Review and Data Analysis.” Washington, DC: Mathematica Policy Research, Inc.,

March 1996.

Ponza, Michael and Sheena McConnell. “Reaching the Working Poor and Poor Elderly: Interim

Report on Focus Groups.” Princeton, NJ: Mathematica Policy Research, Inc., December 1996.

Stavrianos, Michael. “Food Stamp Program Participation Rates, January 1994.” In Current Perspectives

on Food Stamp Program Participation. Alexandria, VA: U.S. Department of Agriculture, Food

and Nutrition Service, March 1997.

Traugott, Michael W., Robert M. Groves, and James M. Lepkowski. “Using Dual Frame Designs

to Reduce Nonresponse in Telephone Surveys.” The Public Opinion Quarterly, vol. 51, 1987.

Trippe, Carole, and Julie Sykes. “Food Stamp Program Participation Rates, January 1992.” In

Current Perspectives on Food Stamp Program Participation. Alexandria, VA: U.S. Department

of Agriculture, Food and Nutrition Service, October 1994.
APPENDIX

OTHER RECOMMENDED REVISIONS

TO THE QUESTIONNAIRES
A-2

In addition to the revisions to the questionnaires identified in Chapter III of the main text, we

recommend some additional, minor changes to the questionnaires. In general, these are revisions to correct

errors in skip logic or oversights in coding structure that we caught during the pretest. The revisions are

as follows:

C The Questions D4d to D4i do not work if the response to Question D4c is that an authorized

representative appointed by the household applied forfood stamps. We recommend skipping

questions D4d to D4i for respondents who said that an authorized representative applied for

them (answered 02 to D4c).

C In the short elderly nonparticipant questionnaire “be difficulty” should be changed to “be

difficult” in Question E12c

C If no-one in the respondent’s household worked (as reported in the screening interview), then

the respondent should not be asked Questions D7c and D7d which ask about difficulties in

applying for food stamps related to working. We could not implement this skip pattern in the

pretest because the main questionnaire was administered by hard-copy.

C In the short elderly participant questionnaire, the skip in D4d (codes 01 through 06) should be

changed from “skip to D7” to “skip to D5.”

C In the short working nonparticipant questionnaire, the skip for “00” and “-1” codes in Question

D1 should be to D14 and not D8.