**American Community Survey**

**Multi-Year Estimates**

**Prepared for the Spring 2007 meetings of the**

**Census Advisory Committee of Professional Associations**

Deborah Griffin and Cynthia Davis,

American Community Survey Office

Revised 3/8/07

This report is released to inform interested parties of ongoing research
and to encourage discussion of work in progress. The views expressed are
those of the authors and not necessarily those of the U.S. Census
Bureau.

**Abstract**

In preparation for production of the first set of multi-year estimates
(MYES) from the American Community Survey, the Census Bureau has created
a set of research data files for a sample of geographic areas. These
data will be used to dress rehearse the production steps required to
produce these estimates as well as to evaluate the properties of
multi-year estimates. In addition, these data will be released publicly
on the ACS website. This paper describes the project and its associated
evaluations.

**Questions**

> 1\. Several of our planned evaluations of quality involve thousands of
> measures of unit and
>
> item response, coverage rates, and reliability measures for a variety
> of geographic levels
>
> across these test sites. Do you have suggestions on how best to
> summarize these data to
>
> fully address issues of survey quality for small areas?

2\. In order to study the usability of MYES we have proposed developing
case studies as

examples for users. Do you have any other suggestions for assessing
usability?

3.  What additional evaluations or analyses do you suggest we consider
    to improve our understanding of multi-year estimates? For example,
    what type of analysis would you recommend we conduct to assess the
    trade off between currency and reliability in advising users about
    when to use single versus multi-year estimates?

4.  We acknowledge that even though single-year estimates from the ACS
    are period estimates covering 12 months, many users do not think of
    them that way. It is critical that we communicate to users the
    concept of a period estimate, especially as we move to 3-year and
    5-year time periods. Do you have any advice on how best to explain
    this concept? Are there conventions that we should consider in table
    titles that could make this more clear?

**1. Introduction**

> The American Community Survey (ACS) is designed to address the small
> area data needs of America's communities. The first set of full sample
> data were released in the summer of 2006 based on the 2005 ACS. Those
> data were the first of a continuous series of single-year estimates
> providing data for the largest geographic areas in the United States
> and Puerto Rico. Beginning in 2008 the Census Bureau will release
> multi-year estimates based on three years of ACS sample (2005, 2006,
> and 2007). In 2010 the first set of 5-year estimates will be released.
> These data will serve the needs of the smallest geographic areas.
>
> In preparation for the 2008 release of multi-year estimates and to
> help data users begin to understand the characteristics of multi-year
> estimates, the Census Bureau has undertaken this Multi-Year Estimates
> Study. This paper provides background on the ACS and an overview of
> the basic methods that are planned for production of multi-year
> estimates. The paper describes the Multi-Year Estimates Study, a
> project that will result in a public release of 1-year, 3-year, and
> 5-year estimates for a sample of geographic areas. Also included is a
> summary of the set of evaluations that are currently proposed.

**2. Background**

> The ACS collects the sample data that were collected on the Census
> 2000 long form. The main difference between the ACS and the census
> long form sample is that the ACS data are collected continuously
> throughout the decade instead of once every ten years. The trade-off
> of the ACS providing much more timely data each year is the much
> smaller annual sample size (about 3,000,000 addresses) relative to the
> long form (about 19,000,000 addresses).
>
> Combining sample from multiple years will reduce the degree of the
> differences in terms of reliability. The sample size of ACS 5-year
> estimates (about 15,000,000), while still less than that of the long
> form, will result in substantially more reliable estimates than ACS
> 1-year estimates.
>
> The Census Bureau will produce ACS 5-year estimates for the same set
> of legal, administrative and statistical geographic entities as the
> long form all the way down to the tract and block group level. So, in
> this sense, these ACS 5-year estimates will replace the estimates
> historically produced from the census long form.
>
> However, the ACS design provides the flexibility for the production of
> additional estimates in legal, administrative and statistical
> geographic entities with larger populations based on aggregating
> sample from less than 5 years. Specifically, the Census Bureau will
> produce tabulations for entities with populations of 20,000 or more
> based on 3 years of sample and for entities with populations of 65,000
> or more based on 1 year of sample. In many ways, these 3-year and
> 1-year estimates are highly useful and important auxiliary data
> products from the ACS Program. Plans call for the ACS to produce
> 1-year, 3-year and 5-year estimates using sets of sample cases as
> illustrated in Table 1.

Table 1. Sets of Sample Cases Used in Producing ACS Estimates

![](media/image1.png){width="6.286111111111111in"
height="2.1354166666666665in"}

> From this table, one can see the "rolling" nature of the annually
> updated multi-year estimates. The 5-year estimates released in 2010
> are based on sample from 2005 through 2009. In 2011, the updated
> 5-year estimates are based on sample from 2006-2010. In addition, the
> table shows that once steady state is reached, the Census Bureau will
> release all three sets of estimates each year for entities with
> populations of 65,000 or more.

In fall of 2005 David Hubble presented a paper on the proposed methods
for the production of multi-year estimates in the ACS. That paper
explained that similar to 1-year estimates, multi-year estimates will be
conceptualized as period estimates that are meant to reflect the
characteristics of the entity over the entire data collection period: 60
months for 5-year estimates, 36 months for 3-year estimates, and 12
months for 1-year estimates. He also noted that our proposed methodology
would:

1.  Present all addresses in sample during the multi-year period into
    one consistent

set of geography defined by the final year of the period,

1.  Pool all sample interviews across all years of the period and
    process them together as though they were a single year's worth of
    data,

```{=html}
<!-- -->
```
2.  Average the single-year controls across the period, and

3.  Make inflation adjustments to the final year of the period.

> Since then a model-assisted (generalized regression or GREG
> application) weighting step has been added to the methodology to
> reduce the variances of base demographics at the tract level in 5-year
> estimates (and at the place level for 3-year estimates). Details on
> this methodology are also being shared at these meetings.

**3. Multi-Year Estimates Study**

> The study includes a series of 1-year, 3-year, and 5-year estimates
> for 34 of the 36 ACS Test counties (all except Fort Bend and Harris
> counties in TX). A full list of these counties is shown in
> Attachment 1. A total of 14 data sets were created including:

1.  1-year estimates for 2000, 2001, 2002, 2003, 2004, and 2005;

2.  3-year estimates for 1999-2001, 2000-2002, 2001-2003, 2002-2004, and

2003-2005; and

3.  5-year estimates for 1999-2003, 2000-2004, and 2001-2005.

> Data products were produced in the form of data profiles (demographic,
> social, economic, and housing characteristics) for a broad set of
> geographic areas including counties, places, Minor Civil Divisions,
> school districts, American Indian Areas, Public Use Microdata Areas,
> Zip Code Tabulation Areas, tracts, and block groups. Estimates are
> released as both estimated counts and estimated percentages.
> Attachment 2 is an example of one of the profile tables.
>
> The production methods for standard products, including thresholds and
> data release rules were used to determine the final set of products.
> All disclosure avoidance methods were used, as planned for 2008. Only
> those areas with populations of 20,000 or greater qualify for 3-year
> estimates (and 65,000 or greater for 1-year estimates).

A special ACS web page was designed for this study. Users can access
these data from the ACS home page. They are not accessible via American
FactFinder. The web page includes background information about the
study, all data sets, methodology documentation, quality measures, and a
set of questions and answers. Attachment 3 is a screen shot of the main
page for the Multi-Year Estimates Study.

The data released as part of this study are considered research data.
The estimates were produced to test production methods and have not
undergone the subject matter and technical review required for standard
ACS data products.

**4. Evaluations**

> Evaluations are being conducted by staff within the Census Bureau and
> by researchers associated with these test sites external to the Census
> Bureau. The objectives of these evaluations are to answer questions
> about the reliability, quality, stability, and usability of multi-year
> estimates and to assess methodological issues involved in their
> production.
>
> **Reliability and Quality.** Staff will evaluate the quality of
> multi-year estimates for small areas with respect to both sampling and
> nonsampling error. Unit and item nonresponse rates, coverage rates and
> coefficients of variation will be calculated and compared to Census
> 2000. Research questions include,

1.  Will 5-year ACS estimates achieve acceptable levels of reliability
    for the small

> areas they are designed to describe?

4.  What are typical variance characteristics for small sub-county areas
    and small

population groups?

1.  Will the low levels of unit and item nonresponse found at higher
    levels of

> geography continue for small areas described by 3-year and 5-year
> estimates?
>
> **Stability.** Demographic analysis methods are planned as a method to
> assess the stability of ACS multi-year estimates over time. Research
> questions include:

1.  How stable are the 3-year and 5-year estimates in measuring change
    over time in the characteristics of the population and housing of a
    given area?

2.  How reliable are the ACS estimates for measuring the characteristic
    distributions

> of total population as well as population and housing subgroups?
>
> **Usability/interpretability.** These estimates will be used to
> develop guidance for users in the form of cases studies on when to use
> single versus multi-year estimates. This research will compare 1-year,
> 3-year, and 5-year estimates for specific areas to show the types of
> differences that should be expected. Research questions include:

1.  How do the distributions of 1-year, 3-year, and 5-year estimates
    compare for a

common area?

2.  How do variances compare across periods of different lengths and
    what guidance

should be given to users on when to use 1-year, 3-year, and 5-year
estimates when all sets are available?

> **Measuring change.** Time series methods are proposed to develop
> methods to compare consecutive 3-year and 5-year estimates.
> Specifically,

1.  How can short term change information be extracted from a series of
    overlapping

3-year or 5-year estimates?

> **Operational feasibility.** Assessments will be made during the
> production of these estimates of operational or methodological issues
> that must be addressed prior to the production of 3-year estimates in
> 2008 and 5-year estimates in 2010. Population thresholds and data
> release rules will be studied.
>
> **Data product design.** Alternative table layouts and formats will be
> developed using these data to assess the optimal presentation of
> multi-year and single-year estimates on AFF.
>
> Many of these evaluations have begun. They are expected to continue
> into the fall of this year.

**5. Key Dates**

> Data were released to external researchers in February 2007. Public
> release on the ACS website is expected in April 2007. Preliminary
> quality, stability, reliability, and usability assessments are
> scheduled in May through July 2007.

Attachment 1

Multi-Year Estimates Counties

![](media/image2.wmf)

Attachment 2

Example of a Data Profile table

**Online Data** -\> **2001-2005 ACS** -\> **Petersburg City, Virginia**

**TABLE 2. PROFILE OF SELECTED SOCIAL CHARACTERISTICS**

  ---------------------------------------------------------------------------------------------
  **Selected Social Characteristics**        **Estimate**   **Margin   **Percent**   **Margin
                                                            of Error**               of Error**
  ------------------------------------------ -------------- ---------- ------------- ----------
  **SCHOOL ENROLLMENT**                                                              

  **Population 3 years and over enrolled in  7,544          +/-316     (N/A)         (N/A)
  school**                                                                           

    Nursery school, preschool                475            +/-153     6.3%          +/-1.9

    Kindergarten                             490            +/-133     6..5%         +/-1.8

    Elementary school (grades 1-8)           3,664          +/-253     48.6%         +/-3.4

    High school (grades 9-12)                1,611          +/-169     21.4%         +/-2.2

    College or graduate school               1,304          +/-220     17.3%         +/-2.5

                                                                                     

  **EDUCATIONAL ATTAINMENT**                                                         

  **Population 25 years and over**           20,937         +/-57      (N/A)         (N/A)

    Less than 9th grade                      1,748          +/-216     8.3%          +/-1.0

    9th to 12th grade, no diploma            3,523          +/-365     16.8%         +/-1.7

    High school graduate (includes           7,754          +/-425     37.0%         +/-2.0
  equivalency)                                                                       

    Some college, no degree                  3,519          +/-384     16.8%         +/-1.8

    Associate\'s degree                      977            +/-166     4.7%          +/-0.8

    Bachelor\'s degree                       2,342          +/-296     11.2%         +/-1.4

    Graduate or professional degree          1,074          +/-175     5.1%          +/-0.8

                                                                                     

    Percent high school graduate or higher   74.8           +/-1.8     (N/A)         (N/A)

    Percent bachelor\'s degree or higher     16.3           +/-1.7     (N/A)         (N/A)

                                                                                     
  ---------------------------------------------------------------------------------------------

Attachment 3

Screen shot of web page

![](media/image3.png){width="6.495833333333334in"
height="4.871527777777778in"}