1 

Program Confusion in the 2014 SIPP: 

Using Administrative Records to Correct False Positive SSI Reports*

Katherine Giefer 

Abby Williams 

Gary Benedetto 

Joanna Motro 

U.S. Census Bureau

4600 Silver Hill Road 

Suitland, MD 20746 

Proceedings of the 2015 Federal Committee on Statistical Methodology (FCSM) Research Conference 

* This paper is released to inform interested parties of ongoing research and to encourage discussion of work in progress. The 

views expressed on methodological or operational issues are those of the author and are not necessarily those of the U.S. Census 

Bureau. Any error or omissions are the sole responsibility of the author. All data are subject to error arising from a variety of 

sources, including sampling error, non-sampling error, model error and any other sources of error. For further information on 

SIPP statistical standards and accuracy, see http://www.census.gov/programs-surveys/sipp/tech-documentation/source-accuracystatements.html.

Introduction 
2 

The 2014 Survey of Income and Program Participation (SIPP) asks questions about two programs administered by 

the Social Security Administration (SSA): Supplemental Security Income (SSI) and Old-Age, Survivors and 

Disability Insurance (OASDI or Social Security). To qualify for SSI a person must have limited income and assets, 

while to qualify for OASDI a person must have worked in the paid labor force. Both SSI and OASDI provide 

benefits to people who are blind, disabled, or over a specified age. Individuals may receive payments from both SSI 

and OASDI at the same time, further blurring the distinction between these programs. Matching SIPP data to SSA 

administrative records revealed that nearly one-half of respondents who reported SSI receipt in the first wave of the 

2014 SIPP panel did not have a corresponding administrative record indicating receipt of SSI payments. Further 

analysis of the linked data suggested that “program confusion” between SSI and Social Security likely contributed to 

the high level of false positive SSI reports. 

In this paper, the term “program confusion” refers to respondents reporting receipt of SSI payments when 

administrative records do not corroborate that response. In other words, a respondent said “yes” they received SSI 

but administrative records do not show that benefits were paid. In the 2014 SIPP, program confusion occurred in 

two ways. Some respondents “double reported” benefits by indicating receipt of both SSI and OASDI when

administrative records indicated only Social Security receipt. Others misreported benefits or “program s wapped” by 

indicating only SSI receipt when administrative records indicated only Social Security receipt. To a lesser extent, 

respondents double reported both SSI and OASDI or misreported OASDI when they were only receiving SSI. The 

outcomes presented in this paper focus on program confusion associated with false positive SSI reports. 

The paper begins by providing background on the SSI program and the rates of false positive SSI reports in prior 

SIPP panels. Next, the data section provides an overview of the wave 1 2014 SIPP data and SSA administrative 

records used for the analysis. It also describes the procedure used to link the SIPP data to administrative records . 

The methodology section explains topic flags, which indicate whether a respondent received SSI in at least one 

month of the reference period. It also describes the method used to impute SSI topic flags and the approach used to 

correct cases of program confusion. The results section describes the evaluation of the initial or “uncorrected” SSI 

topic flag, discusses the solutions considered and presents the results for the “corrected” topic flag.

Background 

The federal government sets requirements that determine whether an individual is eligible for a federal SSI payment. 

Additionally, most states choose to supplement federal SSI benefits with a state SSI payment. States that provide a 

supplement define their own eligibility requirements, which typically differ somewhat from federal eligibility 

requirements, and payment rates. Among the states that offer state supplements, some choose to have their

supplemental payments distributed by the SSA – referred to as federal administration states. Others distribute the 

supplemental payments themselves through state and local agencies – referred to as state administration states. A 

small subset of these states distribute payments using dual administration, meaning some supplements are 

distributed by SSA and some supplements are distributed by the state (Understanding Supplemental Security Income 

SSI Benefits, 2015). To further the complexity, some states offer SSI supplements to people who do not meet the 

eligibility standards for federal SSI, enabling individuals to receive state SSI payments despite not qualifying for 

federal SSI payments (State Assistance Programs for SSI Recipients, January 2011, 2011)1

. In combination, these 

factors make the administration and distribution of SSI payments a complicated process for both program enrollees 

and researchers. 

While SSI in itself is a complex program, its similarity to OASDI drives the issue of program confusion. The 

disability component of OASDI – the Social Security Disability Insurance program or SSDI – provides benefits to 

disabled individuals. The old-age and survivors component of Social Security – commonly called Social Security – 

provides benefits to retirees and their dependents. SSI, SSDI and OASDI all have the word “security” in their name, 

making it easy for survey respondents to respond “yes” to a question about receipt of a program benefit with a 

similar name to the one they are receiving. Moreover, SSI and SSDI have nearly identical acronyms, making it 

difficult to distinguish between these two programs both in writing and in conversation. As mentioned previously, in 

some instances SSI and OASDI serve populations with similar characteristics. The real differences between these 

 

1This report provides detailed descriptions of the state SSI programs in each state as of 2011. The “scope of coverage” sections 

describe the categories of individuals eligible for state SSI receipt in each state. 
3 

programs are in the eligibility requirements related to financial resources and prior labor force participation, which 

unlike age and certain disabilities, are not always apparent on the surface. Finally, because recipients of SSI, SSDI 

and Social Security retirement benefits often stay on these programs for long periods of time, respondents may 

forget the specifics of program names and acronyms. These distinctions have the potential to be even harder for 

proxy respondents to sort out. 

During the latter half of the 2008 SIPP panel, the Census Bureau concurrently conducted two annual field tests 

covering calendar years 2010 and 2011 in preparation for the 2014 SIPP redesign. While the presence of false 

positive reports in the 2008 SIPP panel and the field tests indicate that program confusion may is not an entirely new 

issue, the false positive rate was exacerbated in the first wave of the 2014 SIPP panel. In the 2008 panel, 

approximately 23 percent of the reports of monthly SSI receipt for calendar years 2010 and 2011 were false 

positives (see Table 1). False positive rates of monthly SSI receipt from the field tests were 33 percent for calendar 

year 2010 and 31 percent for calendar year 2011, both of which were higher than the corresponding rates from the 

2008 panel. Few modifications were made to the SSI and Social Security questions between the 2008 panel and field 

tests, however, the topic order was reversed. The 2008 SIPP panel first asked respondents about Social Security 

receipt in the general income section and later asked respondents about SSI and other means -tested programs. The 

field tests (and the 2014 SIPP panel) ask respondents about SSI and other means-tested programs in the event 

history calendar (EHC) portion of the instrument followed by Social Security in the post-EHC general income 

section. 

The design of the 2014 SIPP survey instrument incorporated a number of changes from the field tests in an effort to 

reduce false positive responses and confusion among both interviewers and respondents. The first change was a 

modification to the initial SSI screen from “Now I’m going to ask about SSI” to “Now I’m going to ask about SSI. 

SSI is also called Supplemental Security Income”. This change provides field representatives an opportunity to 

clarify the program by including both the full name and the acronym in the screen text. The other two changes were 

screen notes providing additional guidance to field representatives. One change was a note at the bottom of the SSI 

introduction screen informing interviewers, “SSI is not the same as Social Security. Income from Social Security is 

asked about later.” The other change coincided with the first SSI question (“Are you currently receiving SSI?”) and 

informed interviewers “Do NOT include disability, retirement, or survivor income from Social Security”. These 

changes are only useful when interviewers read and understand the notes on their screen and conduct interviews 

exactly as written. SIPP interviewer training reinforces these standards, but without observing a large number of 

interviews it is hard to determine whether these standards were consistently implemented in the field. 

Literature 

This section addresses two topics: matching administrative data with survey data and imputing topic-level nonresponse in survey data. Both of these subject matters contribute to better understanding to the approach used to 

identify and correct the high rate of false positive SSI reports in wave 1 of the 2014 SIPP. 

Matching survey data with administrative records 

There is a track record of matching SIPP data with SSA administrative records, primarily to evaluate the accuracy of 

survey responses. Research consistently shows that SIPP respondents over report receipt of OASDI, while the 

outcomes for SSI receipt are more nuanced depending on the timeframe being evaluated and whether the analyses 

use monthly or annual estimates (Davies and Fisher, 2009). Research has also established a pattern of program 

confusion among survey respondents between SSI and OASDI (Huynh, Rupp, & Sears, 2002). Looking at data from 

1993, 1995, 1996 and 1998, findings show that administrative records corroborate SIPP reported receipt of OASDI 

between 94 and 96 percent of the time when respondents are receiving only OASDI. The rates of corroboration 

begin to drop, however, when respondents are receiving only SSI, or both SSI and OA SDI. In January of 1993, 79 

percent of the SIPP respondents receiving only SSI reported only SSI in the SIPP. About 8.7 percent reported only 

OASDI, another 6.6 percent reported neither SSI nor OASDI and 6.0 percent reported both OASDI and SSI. For 

those receiving both OASDI and SSI, 76 percent correctly reported this in the SIPP while 3.5 percent reported 

neither program, 15 percent reported only OASDI 

 and 5.9 percent reported only SSI. Similar trends were observed in 1995, 1996 and 1998. Gathright and Crabb 

(2014) extended this work by examining the 1996, 2001, 2004 and 2008 SIPP panels. They found increased rates of 

misreporting in the 2008 SIPP panel. For instance, of respondents receiving only SSI, the proportion reporting only 
4 

OASDI in the SIPP increased from 8.6 percent in the 1996 panel to 14.8 percent in the 2008 panel (Gathright & 

Crabb, 2014). 

Although evaluating survey data matched to administrative records helps answer questions about data quality, some 

researchers have voiced concerns over the extent of bias in matched survey data. Huynh, Rupp, & Sears (2002) 

evaluated match-bias in the 1993 SIPP panel by looking at OASDI and SSI benefit amounts in January of 1993 and 

August of 1995. For both programs, they find that OASDI and SSI benefit amounts among non-matched SIPP

respondents were lower than the payments for matched SIPP respondents. They relate this finding to “a pattern of 

selectivity that may have important implications for combining survey and administrative data for various analytic 

purposes.” Citing a similar notion that, “sample members who are not included in the administrative records of SSA 

are systematically different from the sample members who are included,” Czaijka, Mabli, & Cody (2008) evaluated 

match-bias in both the SIPP and the Current Population Survey (CPS). After analyzing the 1996 and 2001 SIPP 

panels, they found no evidence of increased bias in the 2001 panel despite an increase in the number of non -matches 

due to sample loss. This was true across a number of topics in the SIPP, including SSI. 

Imputing item non-response in survey data 

Prior to the 2014 SIPP panel, missing responses associated with item non-response were imputed using a hot-deck 

imputation method (SIPP Users’ Guide, 2001). This method involved constructing a multi-dimensional matrix 

across a set of respondent characteristics (e.g. age, sex and marital status) and filling the cells of the matrix with 

“hot” values reported from survey respondents. Respondents with mis sing data were matched to a “donor” from the 

hot-deck based on the specified characteristics and assigned the donor’s value. If no donor was available, 

respondents data were imputed using cold-deck, or predefined values, based on historical data. Although this is a 

widely used method for imputing data, a number of assumptions are needed to ensure its functionality. Most 

notably, “the stratified matrix of donors must have reasonable cell sizes in order to get good donors” (Benedetto and 

Stinson, 2009). Each additional dimension added to the hot-deck matrix could both increase the accuracy of the 

imputations and reduce the number of respondent-donor matches (the tradeoff being potentially more cold-deck 

imputations). More recently, researchers have been developing imputation methods that use regression models to 

account for the complex nature of survey data. 

As part of the 2014 SIPP redesign, a model-based imputation approach was used to impute missing topic-level data 

(“topic flags”), while hot-deck imputations remained the main imputation method for most cases of item nonresponse. This model-based procedure allowed analysts to incorporate more variables into the imputation process 

without the problems associated with too many dimensions in a hot-deck matrix. Raghunathan et. al. (2001), after 

recognizing the interdependencies between variables in a dataset, developed a sequential regression multiple 

imputation (SRMI) method for imputing missing data. Their approach uses a series of regressions that impute 

missing values in a hierarchical structure such that each model’s regressors are constantly updated to include 

previously imputed data. Moreover, unlike the hot-deck method, the sequential regressions approach can produce 

different outcomes during each run because of its probabilistic nature. 

Data 

This section provides an overview of the SIPP survey data and SSA administrative records. The SIPP is a 

longitudinal survey that produces estimates from a sample of the civilian, non-institutionalized population of the 

U.S. Historically, panels typically spanned two to four years, interviewing households once every four months on a 

rotating basis. Interviewers asked respondents about a variety of demographic and economic changes that may have 

occurred over the past four months since they were last interviewed (SIPP Users’ Guide, 2001). 

The survey went through a major redesign for the 2014 panel. Changes include reducing interviews to once per year 

and incorporating the use of an event history calendar (EHC) that acts as a visual aid when collecting monthly data 

for a number of topics covering the reference year and the months in the interview year (e.g., if an interview takes 

place in February the interview year months would include January and February). The EHC allows respondents to 

easily identify changes occurring simultaneously or associated with each other. For example, a change in marital 

status from unmarried to married may coincide with a change of residency or health insurance coverage. SSI 

questions are asked as part of the EHC, along with other means-tested programs including Temporary Assistance for 

Needy Families (TANF) and the Supplemental Nutrition Assistance Program (SNAP). Questions about Social
5 

Security receipt are asked later in the interview in what is called the post-EHC section. An additional source of 

information from the SIPP provides insight into better understanding cases of program confusion, which are 

interview notes entered by the field representatives. 

The Supplemental Security Record (SSR) and the Master Beneficiary Record (MBR) are extracts provided by SSA 

to the Census Bureau for the purposes of improving the quality of the data in the SIPP. The SSR is a d atabase 

maintained by SSA to record and track SSI applications, entitlement and benefits paid. The SSR extract has monthly 

benefits paid from January 1974 to July 2014 (with further updates to follow) and can be used to determine the date 

of first payment, as well as the presence of payments in any particular year. The MBR is a database used by the 

SSA to administer the OASDI program. Like the SSR extract, the MBR extract contains information about 

applications, entitlement and monthly benefits paid, although the time series is longer, beginning in January 1962 

and ending in July 2014. 

During SIPP interviews, respondents have the option to consent to administrative data linkage using personal 

information such as their name and date of birth. Those who consent and match to a record in the SSR or MBR are 

given a Protected Identity Key (PIK) that acts as the link between individuals’ SIPP data and administrative records. 

The PIK is then used to identify matches and create the data extracts. Cases with a PIK are referred to as “linked 

records” because the PIK acts as a bridge between SIPP survey data and SSA’s administrative records. Although 

respondents without a PIK are not matched to actual administrative data, it is possible to impute their administrative 

data, similar to how data is imputed for respondents missing SSI data in the SIPP. In this paper, these cases are 

referred to as “imputed records.” For both the linked and imputed records, SIPP SSI receipt is either reported or 

imputed. This generates four types of matched records: 

1. Reported SIPP data and available administrative data (linked records). 

2. Reported SIPP data and imputed administrative data (imputed records). 

3. Imputed SIPP data and available administrative data (linked records). 

4. Imputed SIPP data and imputed administrative data (imputed records). 

While the first three types have at least one piece of reported data, the last type relies entirely upon imputed data. 

The matched data for this analysis is based on survey data from wave 1 of the 2014 SIPP covering 2013 and records 

from the SSR and MBR covering the same period. Of the 72,065 respondents in universe for SSI during wave 1 of 

the 2014 SIPP, 64,721 (90 percent) of the cases had a PIK that allowed for a match between SIPP data and SSA 

administrative records (Table 2). PIKs were unavailable for the remaining 10 percent of the cases either because 

respondents did not consent to record linkage, or because their personal information from the SIPP did not match to 

an individual in the administrative records. 

The SIPP collects a more detailed level of information than the SSA about SSI payment type. SIPP respondents can 

specify whether their payments were from the federal government, state government, or both. The SSR only has 

records for the payments administered by SSA, which includes federal SSI payments and federally administered 

state supplements from federal and dual administration states. SSI state supplements from state and dual 

administration states are not included in SSA’s administrative records because they did not distribute the payments. 

Consequently, SIPP respondents may correctly report state SSI receipt, but there will be no administrative record 

corroborating this report if it was a state administered payment. 

Data describing state policy and providing details on the number of people receiving supplements is outdated and 

difficult to find. The most recent and comprehensive report on state SSI receipt was published by SSA in 2011.2

Although it is possible to identify which states changed the way they administer SSI payments between 2011 and 

2014, it is much more difficult to determine whether they currently offer the option to receive state administered 

supplements when eligibility requirements for federal SSI are not met, and how often this happens. The lack of 

administrative data on these types of payments and poorly documented state policies regarding state SSI 

supplements makes SIPP reports of state SSI receipt difficult to verify. 

This presented a challenge in identifying cases of program confusion in the SIPP. Individuals reporting SSI in the 

SIPP may be incorrectly identified as a false positive or program confusion case despite actually receiving SSI in the 

 

2 Unfortunately, the report will not be updated because of “diminishing resources and increasingly unreliable data.” 

(http://www.ssa.gov/policy/docs/progdesc/ssi_st_asst/).
6 

form of a state administered supplement. Few respondents receive state SSI payments without also receiving a 

federal SSI payment. In January of 2013 – the first month of the reference year for wave 1 – of all SSI recipients 

receiving payments from the federal government only 2.7 percent were receiving a federally administered state 

supplements and no federal SSI payment (SSI Monthly Statistics, 2013,January 2014). Assuming that eligibility 

requirements for state administered state supplements are similar to those for federally administered state 

supplements, receipt of only state administered state supplements and no federal SSI payments should be equally 

rare – making this a small, but important, issue to note. For these reasons, when respondents reported only receiving 

a SSI payment administered by the state their topic flag value of “yes” was retained even though there was no 

administrative record indicating federally distributed payments. 

Methodology 

This section explains the model-based imputation method used to assign topic flags, as well as the approach used to 

identify and correct cases of program confusion. Topic flags are created for several content areas, including 

programs, health insurance and paid employment. They indicate a yes/no status covering the reference period. Like 

all programs in the EHC, the SSI section begins with a set of screener questions that ask respondents whether they 

are currently receiving benefits, or if they received benefits at any time since the beginning of the reference period. 

Respondents who report receiving SSI are asked to identify the months receipt started and stopped for each spell (or, 

each set of months with continuous receipt). During the edit process, these screener questions are used in 

conjunction with the beginning and end months to assign a topic flag value. Respondents who received benefits in at 

least one month between January and December of the reference period are given a value of “yes” for the SSI topic 

flag, while respondents who did not report receiving benefits or only reported benefit receipt during the interview 

year are given a value of “no.” 

When responses to the screener questions are missing, a value is imputed using SRMI (see Appendix B).3

 This is a 

departure from previous SIPP panels, where all item non-response was assigned using hot decks. A major advantage 

of SRMI over other imputation methods is being able to control for characteristics of respondents and other 

household members. Moreover, the process of imputing topic flags incorporates administrative data to mitigate 

problems caused when survey data are not “missing at random.” The result of the SRMI procedure is a “yes” or “no” 

value for each in universe respondent, derived from respondents’ screener values (when data was reported) or 

imputed values (when data was missing). 

For all respondents, the SRMI SSI model controls for: basic demographic characteristics (age, sex, race and 

ethnicity), topic flags from other content areas (disability), administrative SSI record and administrative estimated 

total household earnings. For respondents aged 18 or older at the time of the interview, the model additionally 

controls for: educational attainment, topic flags from other content areas (employment and Social Security) and 

administrative Social Security record. For respondents under age 18 at the time of the interview, additional 

regressors include: presence of parents in the household – and if so their educational attainment and value of their 

topic flags from other content areas (employment and Social Security). With the SRMI procedure, upstream changes 

to any of these variables will be accounted for when modeling SSI receipt. 

For the purposes of the analysis in this paper, the SSR and MBR data extracts were used to create administrative 

indicators for SSI and OASDI receipt. Similar to the topic flags, these receipt indicators are designed to hold a “yes” 

or “no” value for all in universe respondents. For respondents who were not linked to the SSA data extracts, SSI and 

OASDI administrative records were imputed during the SRMI process. 

Together, the SSI topic flag, the OASDI topic flag and the administrative receipt indicators allow for the 

identification of program confusion cases in the SIPP. During the program confusion correction, SSI topic flags for 

non-imputed respondents with an uncorrected value of “yes” whose administrative records indicate OASDI 

payments and not SSI payments are changed to “no” with one exception: the “yes” is retained for respondents 

 

3 This paper focuses on the conceptual issue of program confusion and the results from the administrative data edit correction to 

the 2014 SIPP SSI data. See the FCSM paper by Martha Stinson, Gary Benedetto and Joanna Motro titled “Introducing 

Parametric Models and Administrative Records Into 2014 SIPP Imputations” for an in depth discussion of the methodological 

approach used to impute topic flags.
7 

reporting only state administered payments.4 Similarly, SSI topic flags for imputed respondents with an uncorrected 

value of “yes” whose administrative records indicate OASDI payments and not SSI payments are changed to “no.” 

The correction, in other words, is targeted to a specific type of misreport where it appears respondents were 

cooperating and accidentally reported information in the wrong part of the survey. 

 

Results5

The high rate of SSI false positive reports was initially identified while reviewing the SSI topic flag for wave 1 of 

the 2014 panel. Note that weights were not available at the time of analysis, so the results presented from this point 

forward are unweighted. 

Initial topic flag values 

Table 2 presents the results from a comparison of the values for the pre-correction SSI topic flag after the modelbased imputation to respondents’ corresponding SSI administrative record. Of the 3,485 respondents with a topic 

flag indicating SSI receipt during the reference period: 1,803 had an administrative record showing benefit payment 

and 1,682 had an administrative record showing benefits were not distributed. The number of respondents with a SSI 

topic flag indicating receipt, but no administrative record of SSI payments being distributed represents the total 

number of likely false positive cases and potential program confusion cases. This amounted to nearly one-half (48 

percent) of the respondents who indicated SSI receipt, much higher than the false positive rates seen in both the 

2008 SIPP panel and the EHC field tests. The rate of false positive reporting for sample members under age 18 at 

the time of interview was lower than it was for those aged 18 and older: 38.4 percent (165 of 430) compared with 50 

percent (1,517 of 3,055).6

 

Respondents who reported not receiving SSI had a much higher rate of corroboration in the administrative records. 

There were 68,580 respondents with a SSI topic flag indicating that no benefits were received during the reference 

period: 67,959 (99 percent) with an administrative record showing benefits were not distributed and 621 (1 percent) 

with an administrative record showing benefit payment. The rate of corroboration was nearly 100 percent for 

respondents under age 18 at the time of interview and those aged 18 and older, although the rate was slightly lower 

among adults (99 percent compared with 97 percent). For the small portion of cases where administrative records 

indicate SSI payments were distributed, but receipt of SSI or OASDI was not reported in the SIPP, no correction 

was made to the topic flag. Simply put, administrative records were only used to modify responses when a 

respondent reported receipt in the wrong section of the instrument. 

Among respondents with false positive SSI reports, the OASDI topic flag can be examined for people age 18 and 

older in the last month of the reference year.7 This includes 1,512 (90 percent) of the 1,682 false positive cases. 

Table 3 shows that 1,268 (84 percent) of the cases fall under the program confusion label. Respondents were twice 

as likely to “double report” (860 or 68 percent), or report both SSI and Social Security receipt when administrative 

records indicate they were only receiving Social Security than to “program swap” (408 or 32 percent), or only report 

SSI receipt when administrative records indicate they were only receiving Social Security. 

Table 4 shows the portion of program confusion cases receiving SSDI. Of the program confusion cases, 579 (46 

percent) had an administrative record showing SSDI receipt. Interestingly, 28 percent of those who double reported 

had an administrative record showing SSDI receipt (238 out of 860), while 84 percent (341 out of 408) of those 

who program swapped had such a record. 

Notes entered by field representatives administering the survey reinforce the findings. Entries from the SSI section 

of the instrument indicate respondents meant to report OASDI instead of SSI. Some notes explained that the 

 

4

 The exception included 169 respondents. 

5

 All comparative statements have undergone statistical testing and are significant at the 90 percent confidence level. 

6

 Receipt of SSI for children under age 18 is collected by a proxy respondent aged 18 or older, while receipt of 

Social Security for children under age 18 is collected on a parent or guardian’s record. 

7Note that age restrictions differ slightly between the SSI topic flag and the OASDI topic flag – for SSI age is 

measured at the time of the interview, while for OASDI it is measured in the last month of the reference year. 
8 

respondent did not receive SSI, but did receive Social Security. Others said that respondents were confused about 

which benefits they received, or that the field representative made the mistake themselves. 

Addressing the program confusion cases 

After the high rate of false positive reports was identified, there was concern that the SSI data would need to be 

suppressed for the duration of the 2014 panel. This was an undesirable option given the level of detail provided by 

the SIPP data. Valuable information on monthly SSI receipt, benefit amounts and payment types, as well as spell 

level information on reasons for starting and stopping receipt and the initial year receipt for left censored cases 

would not be available for researchers on the public use file. 

Two solutions to the address the false positive reports were explored. The first was based on incorporating a series 

of logic checks into the SSI edit programs that would identify likely program confusion cases and correct their SSI 

responses accordingly. Some cases would remain uncorrected, leaving noise in the data while still addressing the 

program confusion issue. In order to implement this solution, the program confusion cases needed to have a defined 

set of demographic or economic characteristics that set them apart from the SSI population. These characteristics 

would then be used to define a series of checks used to classify cases as either valid SSI reports or program 

confusion cases. 

After comparing select characteristics, there were no differences between the program confusion cases that could 

reliably set them apart from the respondents who correctly reported SSI receipt (Table 5). With regard to age8

, it was 

initially thought that the program confusion respondents may be more likely to be older since OASDI primarily 

distributes retirement payments. One-half of the program confusion respondents were age 65 or older compared with 

27 percent of the valid SSI respondents. With regard to disability status, it was initially thought that the program 

confusion cases may be less likely to have a disability than the valid SSI respondents. Among respondents identified 

as program confusion cases, 55 percent had a SIPP reported disability compared with 83 percent of respondents who 

correctly reported SSI. With regard to income, it was initially thought that the program confusion responde nts may 

be more likely to have higher income since SSI is means -tested and OASDI is not. Using W-2 earnings, which were 

the best available proxy for income, resulted in statistically different proportions of respondents with low earnings: 

94 percent of program confusion cases compared with 91 percent of valid reports. However, it should be noted that 

among people aged 65 and older earnings is not a particularly reliable predictor of income since many people have 

retired or reduced work effort. For both age and disability status, while the hypotheses were correct the differences 

between the groups were not stark enough to separate the valid responses from program confusion cases. For 

income, again the hypothesis was correct, but there was only three percentage points separating the two groups, 

which would in no way provide guidance in identifying program confusion cases. 

The other solution, which was ultimately implemented, involved using the SSR and MBR extracts to identify cases 

where the SSI topic flag indicated program receipt but the administrative data indicated only Social Security receipt. 

In a similar fashion to the other solution, this correction “fixed” the likely program confusion cases while still 

leaving some noise in the data by ignoring the false positives that exhibited no evidence of program confusion. In 

cases of double reporting, the SSI topic flag was changed to be in accordance with administrative data while the 

Social Security topic flag was left unchanged. For cases involving a program swap, both the SSI and Social Security 

topic flags were changed to be in accordance with administrative data. 

Table 6 presents the distribution of SSI responses in the SIPP before and after the edit correction (see Appendix 

table A-1 for a detailed description of the changes resulting from the edit correction). The edit correction resulted in 

the proportion of people with a “yes” SSI topic flag decreasing from 3,485 (4.8 percent) to 2,238 (3.2 percent) 

respondents. The change was largely driven by a drop in receipt among reported cases falling from 3,322 (4.9 

percent) to 2,095 (3.2 percent). By contrast, there was no statistical change in imputed SSI receipt, which included 

163 respondents (3.6 percent) pre-correction and 233 respondents (3.9 percent) post-correction. 

This same process was executed in reverse to correct cases of program confusion in the Social Security section of 

the SIPP. The Social Security topic flag was first compared to the MBR extract to identify cases of non-receipt. 

These cases were then compared to the SSR extract to identify the extent to which double reporting or prog ram 

 

8

 Age is measured at the time of the interview. 
9 

swapping with SSI occurred in the Social Security section of the instrument. While program confusion was still an 

issue in the OASDI section, it was to a much smaller extent than in the SSI section. 

Conclusions 

The proportion of cases where program confusion was identified as an issue in the first wave of the 2014 SIPP panel 

was concerning enough to warrant suppressing the SSI data for public use in the absence of an edit correction. 

Because the 2014 SIPP panel was well underway when the level of program confusion was identified, fixes were 

limited to edit corrections and minor instrument changes for the remainder of the panel. By linking the SIPP data 

with SSA administrative records, cases of program confusion could be identified and corrected during the edit 

process. This fix reduced the number of false positive SSI reports without extensive, mid-panel changes to the 

survey instrument. One minor wording change in the SSI section was approved and will be implemented in wave 3 

of the 2014 panel. This change expanded the introduction to the SSI section from “Now I’m going to ask about SSI. 

SSI is also called Supplemental Security Income” to “Now I’m going to ask about Supplemental Security Income, 

also called SSI. SSI is different from Social Security, which will be asked about later.” 

Although no major changes were made to the 2014 survey instrument, a new question was proposed for the 2018 

SIPP panel to control for the issue of program confusion and eliminate the need for an edit correction. Prior to the 

SSI and OASDI sections, this question would explain to respondents that it is easy to confuse SSI and OASDI and 

ask whether they were receiving SSI, OASDI, both SSI and OASDI, or neither program. A flashcard outlining the 

major differences between the programs would enable respondents to better identify the benefits they were 

receiving. Important differences such as eligibility requirements, categories of recipients and the health insurance 

associated with each program would be outlined side by side to help respondents compare the two programs. 

Answers to this question would determine which sections come on path later during the survey. Currently, changes 

to the 2018 panel are still pending. 

This project provided some insights into the potential benefits and uses of matched survey and administrative data. 

The matched data revealed the extent of false positive reporting and program confusion in the SIPP, bringing to light 

a concerning data quality issue that would not have been discovered without access to administrative records. 

Moreover, it allowed for the implementation of a relatively straightforward edit correction that improved the quality 

of the data by reducing a specific type of false positive reporting in the SIPP. 

 
10 

Table 1. 

Rate of False Positive SSI Reporting in the SIPP 

Calendar 

Year 

2008 SIPP1

2011 SIPPEHC12012 SIPPEHC1 T-statistic2

Percent Percent Percent 

2010 0.23 0.33 - 3.90 

2011 0.23 - 0.31 2.85 

 

 1 Pooled calendar year estimate based on months of receipt during the year.

 2 This column presents the results of tests of significance between false positive rates for the same year. In both instances, the rates observed in 

the SIPP-EHC field tests were significantly higher than those in the 2008 SIPP. 

 3

 There are no tenths place values because these numbers were taken from a report that presented the results as proportions rather than 

percentages. 

 Source: U.S. Census Bureau, Survey of Income and Program Participation, 2008 Panel, 2011 and 2012 SIPP-EHC Field Tests and the 2014 

Panel (Wave 1) and the Social Security Administration's Supplemental Security Record (SSR).

 
11 

Table 2. 

Pre-Correction Values for the SSI Topic Flag by SSI Administrative Record Among in Universe 

Respondents: 2013 

 X Not applicable. 

 Note: Age is measured at the time of the interview (consistent with the universe for the SSI topic flag). 

 Source: U.S. Census Bureau, Survey of Income and Program Participation, 2014 Panel (Wave 1) and the Social Security Administration's 

Supplemental Security Record (SSR). 
12 

Table 3. 

Pre-Correction Value for the OASDI Topic Flag by OASDI Administrative Record Among Respondents 

Aged 18 and Older with False Positive SSI Reports: 2013 

 X Not applicable. 

 Note: Age is measured in the last month of the interview (consistent with the OASDI topic flag universe). 

 Source: U.S. Census Bureau, Survey of Income and Program Participation, 2014 Panel (Wave 1) and the Social Security Administration's 

Master Beneficiary Record (MBR). 

Table 4. 

Pre-Correction Value for the OASDI Topic Flag by SSDI Administrative Record Among Respondents Aged 

18 and Older with False Positive SSI Reports: 2013

 X Not applicable. 

 Note: Age is measured in the last month of the interview (consistent with the OASDI topic flag universe). 

 Source: U.S. Census Bureau, Survey of Income and Program Participation, 2014 Panel (Wave 1) and the Social Security Administration's 

Master Beneficiary Record (MBR). 

 

Number

Margin 

of Error Percent

Margin of 

Error Number

Margin 

of Error Percent

Margin of 

Error

Total number................................................ 1,512 1,268 X 100.00 X X 100.00 X 244

Administrative Record Imputed....... 53 8 2.05 0.65 26 8 11.07 3.30 27

Reported and Imputed

Yes.............................................................. 935 860 27 67.82 2.16 75 12 30.74 4.86

No................................................................ 577 408 27 32.18 2.16 169 12 69.26 4.86

Reported

Yes.............................................................. 905 834 27 68.53 2.19 71 11 31.70 5.11

No................................................................ 536 383 27 31.47 2.19 153 11 68.30 5.11

Imputed

Yes.............................................................. 30 6 50.98 11.52 4 26 3 20.00 14.71

No................................................................ 41 6 49.02 11.52 25 16 3 80.00 14.71

SIPP OASDI Topic Flag

Administrative Record

OASDI Receipt: "Yes"

Administrative Record

OASDI Receipt: "No" Total

Number

Margin of 

Error Percent

Margin of 

Error Number

Margin of 

Error Percent

Margin of 

Error

Total............................................................... 1,512 579 X 100.00 X 933 X 100.00 X

Administrative Record Imputed....... 53 40 10 6.91 1.73 13 6 1.39 0.63

Reported and Imputed

Yes.............................................................. 935 238 19 41.11 3.36 697 22 74.71 2.34

No................................................................ 577 341 19 58.89 3.36 236 22 25.29 2.34

Reported

Yes.............................................................. 905 232 19 42.03 3.46 673 21 75.70 2.37

No................................................................ 536 320 19 57.97 3.46 216 21 24.30 2.37

Imputed

Yes.............................................................. 30 6 4 22.22 13.16 24 5 54.55 12.35

No................................................................ 41 21 4 77.78 13.16 20 5 45.45 12.35

SIPP OASDI Topic Flag

Administrative record

SSDI Receipt: "Yes"

Administrative Record

SSDI Receipt: "No" Total
13 

Table 5. 

Characteristics of Respondents Aged 18 and Older with a Pre-Correction SSI Topic Flag 

Indicating Receipt, by SSA Administrative Record Type: 2013

 X Not applicable. 

 1 Age is measured at the time of the interview (consistent with the universe for the SSI topic flag). 

 2

 Self-reported disability from Wave 1 of the 2014 SIPP 

 3 Person-level earnings (W-2 income) from SSA's Detailed Earnings Record 

 Source: U.S. Census Bureau, Survey of Income and Program Participation, 2014 Panel (Wave 1) and the Social Security 

Administration's Master Beneficiary Record (MBR) and Supplemental Security Record (SSR). 

Table 6. 

Pre- and Post-Correction Values for the SSI Topic Flag: 2013 

 X Not applicable. 

 Source: U.S. Census Bureau, Survey of Income and Program Participation, 2014 Panel (Wave 1). 

Percent Margin of Error Percent Margin of Error

Total number.................................. X 1,269 1,538 X

Age Group1

 18-61.......................................... 68.14 1.95 37.19 2.23

 62-64.......................................... 4.88 0.90 12.61 1.53

 65 and Older............................. 26.96 1.86 50.20 2.31

Disability2

Yes............................................. 82.57 1.59 55.16 2.05

No............................................... 17.43 1.59 44.84 2.05

Annual Earnings 3

Missing..................................... 5.59 0.96 3.94 0.90

Under $25,000........................... 93.89 1.00 90.70 1.34

$25,000 and Above.................. 0.52 0.30 5.36 1.04

Valid Reports

Administrative Record

 SSI Receipt: "Yes"

Characteristics of SIPP 

Respondents

Program Confusion Cases

Administrative Record

OASDI Receipt: "Yes"

Number Percent Margin of 

Error

Number Percent Margin of 

Error

Number Percent Margin of 

Error

Before Correction

Total number......... 100.00 72,065 X 67,492 100.00 X 4,573 100.00 X

Yes........................ 3,485 4.84 0.11 3,322 4.92 0.14 163 3.56 0.40

No......................... 68,580 95.16 0.11 64,170 95.08 0.14 4,410 96.44 0.40

After Correction

Total number......... 100.00 72,065 X 66,139 100.00 X 5,926 100.00 X

Yes........................ 2,328 3.23 0.13 2,095 3.17 0.11 233 3.93 0.42

No......................... 69,737 96.77 0.13 64,044 96.83 0.11 5,693 96.07 0.42

SIPP SSI Topic 

Flag

Allocation Value

All Reported Imputed
14 

References 

Benedetto, G. and Stinson, M. (2009, November 2-4). Testing New Imputation Methods for Earnings collected by 

the Survey of Income and Program Participation. Paper presented at the 2009 Federal Committee on Statistical 

Methodology (FCSM) Research Conference, Washington D.C. 

(https://fcsm.sites.usa.gov/files/2014/05/2009FCSM_Stinson_VII-C.pdf).

Gathright, G. and Crabb, T. (2014, August 2-7). Reporting of SSA Program Participation in SIPP. Paper presented 

at the 2014 Joint Statistical Meetings (JSM), Boston, MA. 

Huynh, M., Rupp, K.and Sears, J. (2002). The assessment of Survey of Income and Program Participation benefit 

data using longitudinal administrative records (SIPP Report No. 238). U.S. Department of Commerce, U.S. Census 

Bureau. 

Mathematica Policy Research, Inc. (2008). Sample loss and survey bias in estimates of Social Security beneficiaries: 

A tale of two surveys (Reference No. 6193-501). Washington, DC: Czajka, John L., Mabli, J., & Cody, S. 

Raghunathan T.E., Lepkowski J.M., VanHoewyk J., & Solenberger P., (2001). A multivariate technique for multiply 

imputing missing values using a sequence of regression models. Survey Methodology 85-95.

Social Security Administration. (2014, January). “SSI Monthly Statistics, 2013”, 

(https://www.ssa.gov/policy/docs/statcomps/ssi_monthly/2013/index.html ). 

Social Security Administration. (2011), “State Assistance Programs for SSI Recipients, January 2011”, 

(http://www.socialsecurity.gov/policy/docs/progdesc/ssi_st_asst/2011/index.html). 

Social Security Administration. (2015), “Understanding Supplemental Security Income SSI Benefits”, 

(http://www.socialsecurity.gov/ssi/text-benefits-ussi.htm).
15 

Appendix A 

Table A-1.

Detailed Description of Changes Resulting from the Corrected SSI Topic Flag 

 Source: U.S. Census Bureau, Survey of Income and Program Participation, 2008 Panel, 2011 and 2012 SIPP -EHC Field 

Tests and the 2014 Panel (Wave 1) and the Social Security Administration's Supplemental Security Record (SSR). 

 Notes: There were 6 cases where SSI was imputed to be “no” and 124 cases where SSI was reported to be “no” but there 

was evidence of program confusion in the OASDI report resulting in the OASDI topic flag being changed to “no” and the SSI 

topic flag being changed to “yes.” Additionally there are 169 cases where we did not change the SSI report even though there was 

disagreement with the administrative data because the SIPP respondent said that the SSI payment was state-only. 

 

Reported 

SSI 

Receipt

Imputed 

SSI 

Receipt

Yes

Original total............................................................................................................................... 3,322 163

Corrected total............................................................................................................................ 2,095 233

 Difference between uncorrected and corrected total..................................................... -1,227 +70

 Reported program confusion: OASDI receipt in linked administrative data............ -1,301 n.a.

 Reported program confusion: OASDI receipt in imputed administrative data........ -98 n.a.

 Reported state only SSI payment................................................................................... +169 n.a.

 Imputed program confusion: OASDI receipt in linked administrative data............ n.a. -45

 Imputed program confusion: OASDI receipt in imputed administrative data........ n.a. -19

 Reported program confusion: SSI receipt in administrative data.............................. +124 n.a.

 Imputed program confusion: SSI receipt in administrative data............................... n.a. +6

No

Uncorrected total...................................................................................................................... 64,170 4,410

Corrected total............................................................................................................................ 64,044 5,693

 Difference between uncorrected and corrected total..................................................... -126 +1,283

 Reported program confusion: OASDI receipt in linked administrative data............ n.a. +1301

 Reported program confusion: OASDI receipt in imputed administrative data........ n.a. +98

 Reported state only SSI payment................................................................................... n.a. -169

 Imputed program confusion: OASDI receipt in linked administrative data............. n.a. +45

 Imputed program confusion: OASDI receipt in imputed administrative data......... n.a. +19

 Imputed program confusion: SSI receipt in administrative data................................ n.a. -124

 Reported program confusion: SSI receipt in administrative data.............................. -6 n.a.

SSI Topic Flag

Number of 

Respondents