Evaluating the 2013 CPS ASEC 

Income Redesign Content Test

Proceedings of the 2013 Federal Committee on Statistical Methodology (FCSM) Research Conference

Jessica L. Semega

Edward Welniak, Jr. 

Disclaimer: These data are being released on request, despite concerns about their quality. The 

Census Bureau’s policy is not to withhold data that are available, unless releasing such data 

would violate confidentiality requirements. The Census Bureau recommends using these data 

only for research or evaluation purposes, and not to make statements about characteristics of the 

population or economy, because they do not meet the criteria outlined in the Census Bureau’s 

Statistical Quality standard: Releasing Information Product. The views expressed in this 

research, including those related to statistical, methodological, technical, or operational issues, 

are solely those of the authors and do not necessarily reflect the official positions or policies of 

the Census Bureau. The authors accept responsibility for any errors. This paper reports the 

research and analysis undertaken by Census Bureau staff. It has undergone more limited review 

than official publications. 
Abstract 

The Annual Social and Economic Supplement to the Current Population Survey (the CPS ASEC) 

is one of the most widely used socioeconomic surveys publishing national level medians and 

means. The CPS ASEC is the official source of the US national poverty statistics. The ASEC 

asks each person detailed questions categorizing income into over 50 sources. As one of the

nation’s longest running surveys, it has been over 30 year since the last major questionnaire 

redesign.1 In an effort to take better advantage of an automated environment and to update 

questions on retirement income and health insurance, the Census Bureau conducted a field test in 

March 2013 of a redesigned instrument using a retired ASEC sample of 23,000 households. This 

paper looks into the outcomes of the field test and the specifics of the redesigned income portion 

of the ASEC instrument, including the use of income range brackets for respondent question 

refusals to income amounts (currently the instrument will move on to the next source of income) 

and a new “dual-pass” approach that asks all income sources first, followed by income amount

collection. 

Introduction 

The Annual Social and Economic Supplement to the Current Population Survey (CPS ASEC) is

the one of the most widely used surveys conducted by the government, it serves as the basis of 

national income measurement and the source of official poverty estimates. The CPS ASEC is 

mostly conducted in March but also February and April and asks income questions of the prior 

calendar year, as well as health insurance coverage (DeNavas-Walt et al., 2013). This paper only 

discusses the redesigned income questionnaire portion of the ASEC.2 Previous research shows 

the ASEC suffers from misclassification of certain types of income, general underreporting of 

income, and historically under-reported enrollment in means-tested government programs 

(Meyer et al., 2009). To address these issues the Census Bureau contracted Westat Inc. and 

Mathematica in 2011 to evaluate the CPS ASEC questionnaire and to improve the instrument. In 

April of that year, Westat Inc. issued a report entitled “Cognitive Testing of Potential Changes to 

1

In 1994 the CPS ASEC paper questionnaire transitioned to a computer environment.

2 This paper is one among a series of reports that describe the setup and results of that March 2013 CPS ASEC 

Content Test. Bee and Cantu (2013) evaluate the interview time difference between the redesigned ASEC and the 

production ASEC. Hornick (2013) describes the sample selection process for the Content Test. Medalia et al. 

(2013) describe the changes to the health insurance coverage section of the ASEC and summarizes the health 

insurance coverage estimates. Brault (2013) evaluates potential non-response bias from the March 2013 CPS ASEC 

Content Test.

1 

 
the Annual Social and Economic Supplement of the Current Population Survey” which 

suggested potential changes to income questions to correct income errors of varying magnitudes 

across income types, and by particular subpopulations based on the results of conducting

cognitive test. A nation-wide (sample of 23,000 households) content test was conducted by 

telephone interview of the redesigned ASEC in March of 2013 based on their suggestions. The 

content test included a redesigned health insurance section (see Medalia et al., 2013). 

The objective of the redesigned income section of the ASEC instrument is to improve data 

quality by addressing misreporting of income amounts and item nonresponse, and to reduce error

resulting from respondent fatigue. It is also an effort for the ASEC to address the changing 

retirement account environment. While retirement income is still dominated by Social Security 

and traditional pensions, the aggregate holdings in newer types of retirement accounts (such as 

tax-advantaged IRAs and 401k plans) already exceed those of traditional pension plans by a 

substantial margin. Retirement, pensions, and annuities are historically under-reported in the 

ASEC (see Czajka and Denmead 2008). Lastly, one of the largest aggregate shortfalls in 

measured ASEC income is in asset income. The redesigned ASEC better clarifies questions on 

asset income such as interest and dividends. 

Summary of Changes 

Changes to the ASEC questionnaire include: 

1) the removal of the family income screener, 

2) the dual-pass approach, 

3) tailored skip patterns, 

4) income range follow-up for “Don’t know” or “Refusals” to income amounts, 

2 
5) clarification on retirement accounts, pensions and other interest earning assets, 

6) new questions on withdrawals and distributions from retirement accounts and 

7) revised series of questions on social security disability income. 

A discussion of each of these follows. 

- Removal of the Family Income Screener

Currently the production ASEC asks only households that reported less than $75,000 in 

combined family income about means-tested transfer programs such as food stamps and 

Temporary Assistance to Needy Families (TANF). The redesigned ASEC removes this family

income screener because there is evidence that the ASEC instrument inappropriately screens out 

some households that would have been eligible for participating in one or more transfer 

programs.3 In the redesigned ASEC all households are asked all questions regardless of family 

income (including those above $75,000). 

- Dual-Pass

The production ASEC uses an interleaf design, for example, if a respondent answered 

affirmative to receiving a source of income then immediately a follow-up question is asked on 

the amount of the income or the easiest way to report the income (i.e. weekly, monthly, annually, 

etc). The redesigned ASEC uses a dual-pass approach. The “first pass” or first series of 

questions identifies all sources of income received by members of the household. After the “first 

pass” identifies all sources of income received by all members of the household, the “second 

pass” collects the income amounts from each identified source. Earnings from jobs, 

3 Tabulations of 2008 American Community Survey (ACS) data show that 12% of SNAP recipient households and 

20% of public assistance income recipients would have failed the family income screen test if it had been applied to 

the ACS.

3 

 
unemployment income, and worker’s compensation questions are unchanged in the redesigned 

ASEC. They do not follow the dual-pass approach and are always the first series of questions in 

the ASEC asked of each person in the household.4

- Tailored Skip Patterns 

The redesign ASEC tailors the order of the questions asked based on known household 

characteristics. Using tailored skip patterns allows prioritization of more relevant questions to 

help reduce respondent fatigue by presenting the most likely sources of income for that 

household earlier in the interview. Because of the removal of the family income screener, the 

redesigned ASEC asks all questions regardless of household composition, just in different orders. 

The three orders are:

1) Low income (family income less than $75,000): prioritizes questions on public 

assistance, food stamps, WIC, and public housing; 

2) Senior (householder/spouse age 62+): prioritizes questions on disability and retirement 

income; and

3) Default: presented to households that do not qualify as low income or senior, closely 

reflects the current instrument order. 

- Income Range Brackets

The redesigned ASEC presents new range questions anytime a respondent “Doesn’t know” or 

“Refuses” to provide a dollar amount for an income source they (or someone in the household) 

indicated as having received. The income amounts presented in the range questions depend on 

the source of income. The redesigned ASEC uses high, middle, and low ranges developed and 

assigned based on the type of income, as shown below. The objective of the income range 

4

In a 2014 ASEC for a split-sample, all income questions except earnings (asked at the person-level) will be asked 

at the household-level and follow the dual-pass approach.

4 

 
questions is to reduce item nonresponse by allowing respondents to provide a less precise 

amount. The three levels of income range follow-up questions were: 

 High-range income follow-up brackets:

• Less than $45,000 

• Between $45,000 and $60,000 

• $60,000 or more 

If the respondent selects the lowest bracket (less than $45,000), the instrument 

presents the following ranges: 

• Less than $15,000 

• Between $15,000 and $30,000 

• $30,000 or more 

 Mid-range income follow-up questions:

• Less than $10,000 

• Between $10,000 and $20,000 

• $20,000 or more 

If the respondent selects the lowest bracket (less than $10,000), the instrument 

presents the following ranges: 

• Less than $1,000 

• Between $1,000 and $5,000 

• $5,000 or more 

 Low-range income follow-up questions:

• Less than $1,000 

• Between $1,000 and $3,000 

• $3,000 or more 

If the respondent selects the lowest bracket (less than $1,000), the instrument presents 

the following ranges: 

• Less than $100 

• Between $100 and $500 

• $500 or more 

5 
Income Source Range Level

Earnings High

State or Federal Unemployment Compensation Mid

Social Security Mid

Supplemental Security Income (SSI) Mid

Disability Income Mid

Veteran's Payments Mid

Survivor Benefits Mid

Pensions/Retirement/Annuities Mid

Retirement Interest Mid

Property Income Mid

Child Support Mid

Nonretirement Interest Low

Education Assistance Low

Regular Financial Assistance Low

Interest earning accounts Low

Public Assistance/ TANF Low

Food Assistance/ SNAP Low

- Changes to Retirement Accounts and Asset Income 

To capture more relevant retirement income the redesigned ASEC specifically asks if anyone in 

the household has a pension, and then if anyone has a retirement account (such as a 401(k), 

403(b), IRA, or other account designed specifically for retirement savings), in contrast to 

production ASEC, which asks one broad question on pension and retirement income. If the 

respondent has a retirement account, the redesigned ASEC instrument asks the respondent to 

identify the specific type of account. The instrument will proceed to inquire if there was a 

withdrawal or distribution from that retirement account.5 For recipients over 70 years old the 

5 The 2011 Data Analysis Report by the Urban Institute identified that the ASEC misses over 90 percent of 

retirement account withdrawals.

6 

 
question text changes to add, “including distributions you may have been required to take.” To 

make sure the value of the withdrawal correctly counts as household income, there is a follow-up 

question on if the money was “rolled over” or reinvested to another account. 

To better capture asset income, interest and dividend income received on retirement accounts is 

asked separately from non-retirement accounts. Production ASEC makes no distinction 

between investment income received in a retirement account and investment income received 

outside of a retirement account.6 Asking about the specific types of accounts or assets could 

help cue respondents and decrease underreporting. 

Results 

Income source recipiency, reporting of income values and aggregate income totals from the 2013 

production ASEC is compared to data from the 2013 March content test to evaluate the 

performance of the redesigned ASEC.

7

 Table 1 shows the percentage point differences in 

recipiency of various income sources.8 The wording of earnings questions and the four 

unemployment questions, shown in Table 1, are unchanged in the redesigned ASEC. The 

percentage point differences between redesign and production ASEC for these unchanged 

questions show that the results from the content test are comparable. Continuing with Table 1

are sources of income that did change in the redesigned ASEC. Social security recipiency is 

about 14 percentage points higher in the redesign. Recipiency of retirement, pension or annuity 

income in the redesign is about 38 percentage points higher than the production ASEC and 

6 Czajka and Denmead (2008) estimate that the ASEC misses about 40 percent of aggregate interest income and 

about 75 percent of dividend income.

7 The response rate was 43.1 percent of eligible households for the redesign sample (Hornick, 2013). Given the low 

response rate, these data are released on request, despite concerns about their data quality. The Census Bureau 

recommends using these data only for research or evaluation purposes.

8 The recipiency counts are using unedited and unweighted data. 

7 

 
respondents with interest-earning accounts not including retirement accounts (such as checking, 

savings, or money market accounts) is about 18 percentage points higher in the redesign.9 

Table 2 shows the percent difference of recipients age 15 and older who reported amounts for 

each income source between redesigned ASEC and production ASEC.

10

 As stated above, 

questions on earnings and unemployment are unchanged in the redesigned ASEC, but income 

range questions are offered if the respondent refuses or does not know a discrete value. The 

percentages calculated for the redesigned ASEC include the number of people who provided 

discrete or range values. Collection of earnings and Social Security amounts are approximately

9 percentage points and 10 percentage points higher, respectively, in the redesign ASEC. 

Collection of interest amounts is about 32 percentage points higher in redesigned ASEC, which 

was one of the targeted income sources of the redesign. 

Aggregate income total percent differences by source of income are shown in Table 3.11 

Calculations of income aggregates from the redesigned ASEC are with and without income 

range values.

12 Without adding the range values, total aggregate income is 31 percent higher in 

the redesign ASEC as compared to production, and approximately 44 percent higher when using

the mean value of the range reported. Disability, retirement, interest and dividend aggregate 

income totals are all higher using the redesigned ASEC. 

9

Some of the higher totals for retirement and asset income might be due to the higher average age of the content test 

sample.

10 The counts of reported income values are using unedited and unweighted data. 

11 Calculations are using weighted, unedited data.

12 Mean range values calculated based on discrete amounts reported in the 2013 production ASEC. Range values 

will be used as an imputation variable in the future.

8 

 
Conclusion 

Based on this preliminary analysis, it appears that the redesigned 2013 ASEC content test 

captured more income recipiency and more reported amounts resulting in higher aggregate 

income than 2013 production ASEC. Going forward, the Census Bureau plans to do a second 

instrument test in 2014. The 2014 CPS ASEC will proceed with a split-sample test; 5/8ths of the 

national ASEC sample will receive the traditional production ASEC and 3/8ths of the sample 

will receive the redesigned ASEC.

13

 This split-sample will permit a better evaluation of the test 

instrument compared to the production instrument in a full production environment using fully 

edited and weighted data. This test will ultimately compare income and poverty estimates. 

 

13 Redesigned ASEC health insurance questions will be administered to the full sample.

9 

 
REFERENCES

Bee, Adam and Aaron Cantu. 2013. “Evaluating Respondent Burden of the CPS ASEC Content 

Test with Timer Data.” Paper presented at the 2013 meetings of the Federal Committee 

on Statistical Methodology.

Brault, Matthew W. 2013. “Non-response Bias in the 2013 CPS ASEC Content Test.” Paper

presented at the 2013 meetings of the Federal Committee on Statistical Methodology.

Czajka, John L. and Gabrielle Denmead. 2008. “Income data for Policy Analysis: A 

Comparative Assessment of Eight Surveys” Mathematica Reference No.: 6302-601

[http://mathematica-mpr.com/publications/PDFs/incomedata.pdf] 

DeNava-Walt, Carmen, Bernadette D. Proctor, and Jessica C. Smith. 2012. Income, Poverty, and 

Health Insurance Coverage in the United States: 2012. U.S. Bureau of the Census, 

Current Population Reports, P60-243. Washington, DC.: U.S. Government Printing 

Office.

Hicks, Wendy and Jeffrey Kerwin, 2011. “Cognitive Testing of Potential Changes to the Annual 

Social and Economic Supplement of the Current Population Survey.” Unpublished 

Westat report to the U.S. Census Bureau. July 25, 2011. 

Hornick, David. 2013. “The 2013 Annual Social Economic Supplement Health Insurance 

Questionnaire Test: The Sample Design.” Paper presented at the 2013 meetings of the 

Federal Committee on Statistical Methodology.

Medalia, Carla, Amy Steinweg, Brett O’Hara, David Lee, Jessica Smith, Joanne Pascale, and 

Jonathan Rodean. 2013. “Health Insurance in the Current Population Survey: Redesign 

and Production.” Paper presented at the 2013 meetings of the Federal Committee on 

Statistical Methodology. 

Meyer, Bruce, Wallace K. C. Mok, and James X. Sullivan. 2009. “The Under-Reporting of 

Transfers in Household Surveys: Its Nature and Consequences.” Harris School Working 

Paper #09.03. 

10 
 

Worked at job/business -2.9

Work seasonal 1.1

State or Federal unemployment 0.2

Supplemental unemployment benefits 0.0

Union unemployment/strike benefits 0.0

Worker's compensation 0.3

Disability Income (excluding SSDI) 2.2

Social security 13.8

SSI -0.4

Survivor benefits 1.0

Retirement/Pension income 37.7

Non-retirement accounts 17.5

Dividends 9.2

Public assistance -0.3

1

 The counts of reported income recipiency are using unedited and unweighted data.

Source: U.S. Census Bureau, Current Population Survey, 2013 Annual Social and 

Economic Supplement and the 2013 CPS ASEC Content Test

Social Security

Survivor Benefits

Retirement

Interest/Dividends

Public Assistance

Disability

Table 1. Comparison of Income Source Recipiency for Redesigned and Production ASEC1

Income Source

Percentage Point 

Difference 

(Redesign - Production)

Earnings

Unemployment/Worker's Compensation

11 
 

Earnings 52.8 43.7 9.1

Wages and Salary 8.0 4.6 3.5

Nonfarm Self-Employment 1.1 0.6 0.5

Farm Self-Employment 0.4 0.2 0.2

Unemployment Compensation 3.0 2.5 0.4

Workers Compensation 0.6 0.3 0.3

Social Security 20.8 10.9 9.9

SSI (Supplemental Security Income) 1.2 1.7 -0.5

Survivor Benefits 1.5 0.7 0.8

Public Assistance 0.3 0.6 -0.3

Veterans Benefits 1.5 0.9 0.7

Disability Benefits 3.5 0.4 3.1

Retirement Income 9.4 4.0 5.4

Interest 45.8 14.0 31.8

Dividends 13.7 4.1 9.7

Rents, Royalties, Estates or Trusts 5.0 2.8 2.2

Pell Grant 2.0 1.0 0.9

Other Education Assistance 1.5 1.5 0.0

Alimony 0.0 0.1 -0.1

Child Support 1.2 1.7 -0.5

Financial Assistance 0.7 0.6 0.1

Other Income 2.2 1.0 1.1

1

 The counts of reported income values are using unedited and unweighted data.

Source: U.S. Census Bureau, Current Population Survey, 2013 Annual Social and Economic Supplement and the 

2013 CPS ASEC Content Test

Table 2. Percent of Recipients Reporting an Amount for Redesigned and Production ASEC1

Income Source 

Redesign ASEC

Percent Provided 

Amount 

(range included)

Production ASEC

Percent Provided 

Amount

Percentage Point 

Difference 

(Redesign-Production)

12 
Income Source 

Percent Difference (Redesign 

without Range 

value/Production)

Percent Difference (Redesign 

with Range value/Production)

Total Income 31* 44*

Total Earnings 30* 43*

Earnings 30* 42*

Wages and Salary 46* 63*

Nonfarm Self-Employment -58* -21

Farm Self-Employment -0.2 7

Unemployment Compensation 12 15

Workers Compensation 48 61*

Social Security -4 -2

SSI (Supplemental Security Income) -39* -39*

Survivor Benefits -11 16

Public Assistance 9 10

Veterans Benefits -11 -3

Disability Benefits 488* 556*

Retirement Income 31* 41*

Interest 399* 598*

Dividends 50* 113*

Rents, Royalties, Estates or Trusts -8 0.7

Pell Grant 61 102*

Other Education Assistance 8 20

Child Support -7 -4

Financial Assistance 46 52

Other Income 160* 163*

Table 3. Comparison of Aggregate Income Redesigned and Production ASEC1

* Statistically significant at the 90 percent confidence level.

Source: U.S. Census Bureau, Current Population Survey, 2013 Annual Social and Economic Supplement 

and the 2013 CPS ASEC Content Test

1

 Aggregates calculated using unedited but weighted data.

13