# Presentation: 827418

## American Community SurveyTechnical Training

- Deborah Griffin, David Hubble, Susan Love, Larry McGinn, U.S. Census Bureau
- Presented at the State Data Center/Business and Industry Data Center
- Annual National Training Conference
- September 21 - 23, 2004

## Overview of Session

- Plan to cover broad set of topics, including both technical details and specific examples.
- After each topic is discussed there will be time allotted for questions and discussion.

## Overview of Session

- Goals
- Sample Design
- Data Collection
- Data Processing
- Weighting and Estimation
- Comparisons with Census 2000
- Data Products
- Current Status

## Goals

- Produce more timely social, economic, and housing data for small geographic areas
- A decennial census that is streamlined to collect only the most basic data

## Sample Design

## Sample Design

- About 3 million addresses selected each year
- About 250,000 sample addresses each month
- In every county
- Survey designed to include
- Population in both housing units and group quarters
- Stateside and Puerto Rico

## Sample Design

- Sample is accumulated over TIME to produce lowest levels of geographic detail to replace census sample
- Five years of data are required for areas with less than 20,000 population
- block groups
- census tracts

## Sample Design: Sampling Rates

| Governmental Unit Size: Number of Occupied Housing Units | Census 2000 Sampling Rates | ACS 5-year   Sampling Rates |
| --- | --- | --- |
| 0 - 200 | 50.0% | 50.0% |
| 201 - 800 | 50.0% | ~35.0% |
| 801 - 1200 | 25.0% | ~17.5% |
| 1201 - 2000 | 16.7% | ~12.0% |
| Over 2000 | 12.5% | ~8.5% |

**Notes:**
Looking at item nonresponse, this table summarizes what we call, summary allocation rates.  

Allocation rates are a measure of the proportion of responses for an item that had to be imputed using allocation methods. 

Summary allocation rates summarize allocations across all items.  So a 10% summary allocation rate means that for all of the data that should have been provided, 10% had to be allocated.  

Overall and across all modes of data collection – the ACS had a summary allocation rate of 6.6% while the Census 2000 sample had a summary allocation rate of 10.4%.  [The ACS uses 3 modes – mail, telephone and PV followup;  the census primarily uses mail and personal visit followup methods of data collection]

This table shows that the level of item nonresponse for mail returns was fairly similar in the ACS and census for population items but census levels were much higher for housing items on mail returns.

Also, data collected during followup in the census were markedly less complete than data collected by interviewers in the ACS.

## Sample Design: Frame

- Sample cases selected from an updated Census 2000 Master Address File (MAF)
- MAF updated through the use of...
- Delivery Sequence File from USPS
- Community Address Updating System

**Notes:**
Starting with DATA QUALITY

We have created a website that includes measures of quality at state and natl levels – it includes measures of coverage, unit and item nonresponse and sample sizes.  The site provides user-friendly explanations of each of these error sources and details on how the rates were computed.

Our plan is to release QMs along with the annual release of ACS data.

I encourage you to visit the website to review this information and to become familiar with how these rates vary across state and over time.

The ACS acknowledges having smaller sample sizes, and thus larger sampling variances on the estimates produced from the survey than could be produced from the once-a-decade long form sample.  With respect to nonsampling errors, I want to compare levels of unit and item nonresponse between the census 2000 sample and the ACS.

## Sample Design

- Sample Design Evaluation Projects Include
- Monitor the reliability of the estimates and costs to adjust sampling rates, if needed
- Monitor response to mail and CATI to adjust CAPI sampling rates, as appropriate

**Notes:**
Starting with DATA QUALITY

We have created a website that includes measures of quality at state and natl levels – it includes measures of coverage, unit and item nonresponse and sample sizes.  The site provides user-friendly explanations of each of these error sources and details on how the rates were computed.

Our plan is to release QMs along with the annual release of ACS data.

I encourage you to visit the website to review this information and to become familiar with how these rates vary across state and over time.

The ACS acknowledges having smaller sample sizes, and thus larger sampling variances on the estimates produced from the survey than could be produced from the once-a-decade long form sample.  With respect to nonsampling errors, I want to compare levels of unit and item nonresponse between the census 2000 sample and the ACS.

## Your Questions?

## Data Collection 

## Data Collection

- Methodology based on best practices from decennial census and demographic surveys
- Overlapping monthly samples using multi-mode data collection methods
- Mail
- Telephone
- Personal Visit

**Notes:**
Success of self-response in decennial - mail implementation strategy

computer assisted survey instruments and permanent staff from current surveys

## Data Collection

- Mail
- full implementation strategy
- telephone follow-up of incomplete mail returns
- Telephone
- CATI
- Personal Visit
- CAPI using laptops
- subsample selected for personal visit

**Notes:**
Success of self-response in decennial - mail implementation strategy

computer assisted survey instruments and permanent staff from current surveys

## Data Collection 2002 Interview Mode Distributions

## Data Collection - Response Rates

- Unit nonresponse to the ACS is very low - nationally and for all states
- ACS national-level response rates for 2000 - 2003 have all exceeded 95%
- Quality measures page provides
- annual response rates at national and state levels
- detailed reasons for noninterviews

**Notes:**
These three modes are successful - survey response is high

## Data Collection

- Data are collected using a current residence rule
- unlike decennial census usual residence rules
- Data are collected throughout the year

**Notes:**
Success of self-response in decennial - mail implementation strategy

computer assisted survey instruments and permanent staff from current surveys

## Data CollectionLanguage Support

- Mailout currently in English only but Spanish questionnaire available upon request
- Phone and personal visit instrument available in English and Spanish
- Telephone and personal visit operations include bilingual interviewers in most languages

**Notes:**
Rate of noninterviews due to language barriers remains very low - less than 1/2 of 1% of all sample cases

## Data Collection - Issues

- Language research ongoing to develop additional tools
- Maintaining high levels of mail response is a priority
- Looking into methods to improve effectiveness of telephone interviewing
- Evaluations of Puerto Rico data collection planned

## Your Questions?

## Data Processing

## Data Processing

- Annual boundary definition
- Accumulate data collected in the calendar year
- Data editing
- Identify the interviews and the noninterviews
- Identify inconsistent and missing answers
- Item imputation
- Same methods as those used in the census
- Rule-based assignments
- Nearest neighbor allocation

## Item Nonresponse Quality Measures page

- Item Nonresponse Quality Measures page

## Item Nonresponse  2003 results

## Item Nonresponse

- Lowest allocation rates
- sex
- citizenship
- telephone availability
- monthly condominium fee
- grandchildren living in home

- Highest allocation rates
- Yearly mobile home costs
- Yearly property insurance
- Other monthly mortgage payments
- Yearly real estate taxes
- Year built

## Your Questions?

- Weighting and Estimation

## 3 Major Components to Annual Weighting Process

- Initial weights to reflect the probability of selection
- Adjust weights of interviewed households to account for noninterviews
- Adjust weights to independent housing unit and population estimates (controls)

## Ratio Adjustments to HU and Population Controls

- Intercensal estimates are produced by updating the previous census results using various administrative records data
- In a multi-stage process, HU and population adjustment ratios are applied to the weights
- Applied at the county (or group of counties) level by race/ethnicity and age/sex groups

## Multi-year Estimation

- Most multi-year estimates are generated by computing an average based on combining each years’ estimates

## Example 3-year Estimate

- Percent foreign born population in year 1:

## Example 3-year Estimate

- Three-year estimate of percent foreign born in years 1-3:

## Multi-year Estimation

- Estimates are computed using the geography in place as of the most recent year of the period
- Dollar valued data items are inflation adjusted to most recent year of the period

## Multi-year Estimation: Medians

- Medians are produced using combined data records from all years, not by averaging each year’s median
- A 3-year median household income estimate is determined by combining the household records from the 3 years into one data set and determining the median from this combined distribution

## Weighting and Estimation

- Research and Evaluation Projects Include
- Develop Group Quarters and Puerto Rico Estimation Methods
- Refine estimation process as we move from the testing period to full implementation
- Small Area Estimates Reliability
- Current Residence Based HU and Population Controls

**Notes:**
Starting with DATA QUALITY

We have created a website that includes measures of quality at state and natl levels – it includes measures of coverage, unit and item nonresponse and sample sizes.  The site provides user-friendly explanations of each of these error sources and details on how the rates were computed.

Our plan is to release QMs along with the annual release of ACS data.

I encourage you to visit the website to review this information and to become familiar with how these rates vary across state and over time.

The ACS acknowledges having smaller sample sizes, and thus larger sampling variances on the estimates produced from the survey than could be produced from the once-a-decade long form sample.  With respect to nonsampling errors, I want to compare levels of unit and item nonresponse between the census 2000 sample and the ACS.

## Understanding Multi-year Estimates

## Multi-year Estimates

- Two examples
- Item increasing from year-to-year
- Item with year-to-year increases and decreases
- Overview only
- Focus today is on the timing and creation of 3-year and 5-year data products

## Multi-year Estimates

## Slide 39

## Slide 40

## Slide 41

## Multi-year Estimates

## Multi-year Estimates

## Slide 44

## Slide 45

## Slide 46

## Multi-year Estimates

## Multi-year Estimates - Issues

- What should users do when multiple products are available?
- Trend analysis
- How should users interpret each data item as a multi-year estimate?

## Your Questions?

## Comparisons with Census 2000

- Comparison studies recently released
- Four reports compared single-year data from the ACS with Census 2000
- Two reports compared three-year averages from the ACS with Census 2000
- To access reports
- _[www.census.gov/acs/www/advMeth/Reports](http://www.census.gov/acs/www/advMeth/Reports)_

## Comparisons with Census 2000

- ACS distributions were similar to those of Census 2000
- Only about 9% of the profile table estimates differed by more than 1 percentage point
- Self-response data from mail returns were responsible for the consistent results

## Comparisons with Census 2000

- Characteristics differing the most were
- Race and relationship
- Ancestry, people 25+ with only a high school diploma
- Employment rate, median household income, poverty rate of female householders with children <18
- Units built in 1939 or earlier, 4-room units, units heated with utility gas

## Comparisons with Census 2000

- Reasons for some differences
- Timing and reference periods
- Quality of nonresponse followup
- Training and staff experience
- Treatment of proxy information
- Data completeness
- Troublesome questions
- Weighting

## Your Questions?

## Data Products

## Illustration of an ACS Release schedule (assumes full implementation begins in 2005)

| Type of Data | Population
Size of Area | Data For The Previous Year Released In The Summer Of:
 2006        2007         2008        2009       2010       2011        2012 |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Annual Estimates | 65,000+ |  |  |  |  |  |  |  |  |
| 3-year averages | 20,000+ |  |  |  |  |  |  |  |  |
| 5-year averages | Down to Census Tract and Block Group  |  |  |  |  |  |  |  |  |

**Notes:**
So, let’s look at each of the years, 2006 through 2010.  
2006 – single-year estimates only (We will probably not be able to do multi-year data products including past years because of the changes in the sample, especially the inclusion of the GQ pop).  About 9000-10000 geo areas will get estimates
2007 – same as 2006
2008 – Single-year estimates plus a 3-year average data product covering the years 2005-2007 (about 20,000 geo areas will get estimates from the ACS for the first time)
2009 – Same situation as 2008: this time the 3-year average covers 2006-2008
2010 – single-year estimates 2007-2009 averages, and the first 5-year averages covering 2005-2009.  
2011.  In this year, it is worth noting that the 3-year averages product covers 2008 – 2010, no overlap with the first 3-year averages product covering 2005-2007.

## Example of a Base Table

**Notes:**
We are working with the American FactFinder support staff to find a way to allow data users to have more options for combining or hiding estimates from base tables.  Since ACS is the long form replacement, we understand that we must produce products very similar to the 2000 Decennial Summary File 3, the most commonly used base tables product based on the long form sample.  We are also investigating how we might create an ACS counterpart to the much more detailed Summary File 4 product.

## Example of a Ranking Table

## Example of a Multi-year Profile

**Notes:**
We want to show as many years side by side for each characteristic as is feasible in a display window.  Makes it easier to assess whether there is any potential change over time that one may wish to examine more closely.  This approach to profils allows us to merge together into a single data product what are now two separate data products: single-year profiles and change profiles.

## Here’s what the Educational Attainment subject table might look like:

## Other Products

**Special Tabulations**
- Done on a reimbursable basis similar to the Special Tabs based on the 2000 Decennial Census
- Disclosure Review Board has different rules for these tables than for the regular data products
**Public Use Microdata Sample (PUMS)**
- Created each year from the full ACS sample
- Size cannot exceed 1% of the total sampling universe
- Recoding and other techniques applied to data to protect confidentiality
- PUMS data will be released for the 2000 Decennial PUMAs areas once full   implementation begins

**Notes:**
You can find the details about the ACS Special Tabulations program on our website.

## Your Questions?

## Current Status

- Plan to begin full 3 million sample in 2005
- produce first products based on the full sample in the summer of 2006
- Plan to begin Puerto Rico Community Survey in 2005
- produce first Puerto Rico products in summer of 2006

**Notes:**
Overall sampling rate of about 2.5%
Sampling rate varies by size of govt unit (annual samples range from 
1-in-10 to 1-in-57, 
5year averages thus range from
 1-in-2 to 1-in-11.5)

## Current Status

- Delay “ramp up” from June 2004
- Increased hiring, training delayed until budget uncertainties resolved
- Delay start of group quarters until 2006
- First total population products delayed until summer of 2007

## Challenges

- Educating users
- on transitioning to ACS multi-year products
- on new options possible with annual data
- Collecting group quarters data
- sampling and estimation issues
- quality issues
- Maintaining data quality
- with expanded sample

## Your Questions?