American Community Survey: Key Unresolved Issues
Published: 2004-10-07T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The Congress asked GAO to review operational and programmatic aspects of the Census Bureau's ACS that will affect the reliability of small geographic area data. The ACS will be a mail survey of about 3 million households annually, whose results will be cumulated over 5 years to produce estimates that will replace information previously provided by the Decennial Census long form. In addition, annual data will be published for geographic areas with 65,000+ populations and as 3-year averages for areas with populations of 20,000 to 65,000. Annual data will be published beginning in 2006 with data for 2005. The 5-year averages for 2008-12 will provide data for small geographic areas.
section_title: What GAO Found
paragraphs: The Census Bureau's development of the American Community Survey goes back several decades and has included intensive research and field testing programs, as well as substantial outreach efforts, in particular through the reports and workshops at the National Academy of Sciences (NAS). However, if the ACS is to be an adequate replacement for the Decennial Census long form as the major source of data on small geographic areas and if it is to provide similar annual data for larger areas, the Census Bureau will need to incorporate in a timely manner the resolution of issues it has already identified in the ACS testing and 2000 Decennial Census evaluation programs, such as the residence concept, group quarters, and questions on disability; complete the ACS testing plan as originally planned, such as the comparison and evaluation of long form-ACS supplementary survey data at the state level, to identify other unresolved issues and to provide information for users of 2000 Decennial Census long-form data that will be necessary for the transition to the full ACS; evaluate and consult with stakeholders and users on the resolution of issues identified in this report, such as the methodology for deriving population and housing controls, guidance for users on the impact of the characteristics of multiyear averages for small geographic areas, and the presentation of dollar-denominated values; coordinate the results of the testing program for the 2010 Decennial Census short form with the ACS implementation schedule; and resolve all issues so that the ACS estimates beginning with 2008 are consistent with the ACS estimates for 2009-12 and with the 2010 Census short form. Although the Census Bureau has solicited advice from external stakeholders and users and has supported research by its own staff on most of the issues identified in this report, there is no indication that the Census Bureau has yet followed this advice or implemented plans for consultation on resolving these issues. In addition, it has been more than a year since the Census Bureau announced that it was looking into establishing an ACS partnership program that would involve advisory groups and expert panels to improve the program, but no such program has been established. Another issue related to the proposed ACS is how the Census Bureau might provide more timely and reliable small geographic area data. This goal could be accomplished, but it would require additional funding. The most direct approach would be to increase the sample size for 2009-11. This increase would enable the Bureau to provide small geographic area data that would be the replacement for the 2010 Census long form 1 year earlier.

section_title: Letter
paragraphs: The Census Bureau has designed the 2010 Decennial Census around three  new operations. One will replace the Census long-form questionnaire with  the American Community Survey (ACS). Testing the ACS began in 1996  and full implementation will begin in 2005 and continue as long as the  program receives annual funding.
A separate long-form questionnaire has been mailed to a sample of  households once a decade to collect detailed information on demographic,  housing, social, and economic characteristics since the 1960 Decennial  Census. This information has been the main source of information for  small geographic areas, including tracts and block groups; it has been used  extensively by federal agencies for program implementation and by state  and local governments for programmatic and planning purposes. In the  2000 Decennial Census, the long form was mailed to a sample of about   19 million housing units.
The ACS will contain the same questions as the long form but will be  mailed monthly to an annual sample of 3 million housing units. With the  smaller sample, the ACS is designed to provide the same information at  the same level of geographic detail as the long form by means of a  continuous measurement methodology in which survey responses will be  accumulated over time. The Census Bureau has determined that in order  to produce reliable estimates at the same geographic level of detail as the  long form, ACS results will be cumulated over 5 years. It also has  determined that the ACS will provide reliable estimates for geographic  areas with populations of 20,000 to 65,000 by cumulating ACS responses  over 3 years and for geographic areas with populations of more than   65,000 by cumulating ACS results for 1 year but that these estimates will  be less reliable than the corresponding long-form estimates.
According to the plan the Congress approved, the first annual ACS data for  geographic areas with populations larger than 65,000 will be published  beginning in 2006 with data for 2005; 3-year averages for geographic areas  with populations between 20,000 and 65,000 will begin in 2008; and 5-year  averages for geographic areas with populations smaller than 20,000,  including tracts and block groups, will begin in 2010. The 5-year averages  for 2008–12 to be published in 2013 will replace the 2010 Decennial Census  long form for small geographic areas, as they will be centered on 2010 and  closely reflect the population and housing characteristics data from the  2010 Decennial Census short form. In replacing the long form, the ACS will  provide the same long-form data items at the same level of geographic area  detail but in a more timely way. Whereas the long form provided small  geographic detail once a decade, the ACS will provide annual estimates for  large geographic areas and estimates for smaller areas in terms of 3-year  or 5-year averages.
You asked us to examine issues about replacing the long form with the  ACS related to the reliability of data for small geographic areas. As agreed  with your offices, our objectives for this report were to (1) review the  Census Bureau’s testing program on operational and programmatic  aspects that will affect the reliability of small geographic area data and   (2) determine whether alternatives to the proposed ACS would provide  more frequent and more reliable data for small geographic areas.
To address these topics, we reviewed ACS-related Census Bureau  documents, congressional testimony, National Academy of Sciences (NAS)  reports, and consultants’ reports prepared for the Census Bureau, Bureau  of Labor Statistics (BLS), and Department of Housing and Urban  Development (HUD). We also interviewed small-area data experts on the  latest NAS report on the ACS and reviewed the Census Bureau’s responses  to recommendations on the ACS in our earlier reports.
We conducted our work between April 2003 and August 2004 in  accordance with generally accepted government auditing standards. We  describe our scope and methodology in more detail in appendix I.
subsections: section_title: Background
paragraphs: Now that the Census Bureau has congressional approval to begin the full  ACS, data collection will begin in November 2004. The ACS test survey of a  sample of 800,000 housing units, which has been conducted since 2000,  will end in December 2004. The Bureau has been using this survey, known  as the ACS Supplementary Survey, to test procedures and to produce  annual data for geographic areas with populations of 250,000 or more. As  one part of the test program, the supplementary survey data for 2000 have  been compared with corresponding data from the 2000 Census long form  to evaluate the quality of the ACS data and to provide users with  information to make the transition from the long-form data to the full ACS  data. According to the plan the Congress approved, the first annual ACS  data for geographic areas with populations larger than 65,000 will be  published beginning in 2006 with data for 2005; 3-year averages for  geographic areas with populations between 20,000 and 65,000 will begin in  2008; and 5-year averages for geographic areas with populations smaller  than 20,000, including Census tracts and block groups, will begin in   2010. The 5-year averages for 2008–12 will replace the 2010 Decennial  Census long form for small geographic areas; they will be published in  2013 and will incorporate population and housing characteristics data  from the 2010 Decennial Census short form.
In replacing the long form, the ACS will provide the same long-form data  items at the same level of geographic area detail but in a more timely way.  Whereas the long form provided small geographic detail once a decade,  the ACS will provide annual estimates for large geographic areas and  estimates for smaller areas in terms of 3-year or 5-year averages; the 5-year  averages will provide data at the same geographic area level as the long  form. According to the Census Bureau, these 5-year averages will be about  as accurate as the long-form data; the annual and 3-year averages will be  significantly less reliable than the long-form data but more reliable than  existing annual household surveys the Census Bureau conducts.
In the remainder of the Background section of this report, we briefly  describe the major differences between the ACS and the Decennial Census  long form. We also discuss the Census Bureau’s outreach program,  designed to involve stakeholders and users in shaping the ACS. Appendix  III provides additional background information on the evolution of the  ACS plan, appendix IV on the ACS testing and measurement program.  Appendix II describes recent NAS findings on Continuous Measurement  (CM) and the ACS.
subsections: section_title: Multiyear Averages
paragraphs: The 2000 Census long form used a decennial sample of about 19 million  housing units; the full ACS will use an annual sample of 3 million housing  units. In order to provide reliable estimates for geographic areas with  populations of 65,000 or less, monthly ACS responses will be cumulated  over several years—3 years for places with populations of 20,000 to   65,000 and 5 years for places with populations smaller than 20,000.  Because of the statistical properties of these averages and users’  unfamiliarity with them, the Census Bureau has long recognized the need  to provide guidance on such topics as the reliability of the averages for  areas with rapidly changing population and the use of multiple estimates  for states and other, larger geographic areas.
section_title: The Concept of Residence
paragraphs: For the 2000 Decennial Census, the ACS test programs, and federal  household surveys, including the Current Population Survey (CPS),  seasonal residents are recorded in a geographic area according to a  concept of usual residence. As we noted above, under this concept, people  who spend their winter in Florida and the rest of the year in New  Hampshire, for example, are recorded as residents of New Hampshire;  college students living away from home in dormitories are recorded as  residents of the college.
For the full ACS, the Census Bureau has announced its decision to change  the concept to current residence. According to the Census Bureau,  although each concept requires that a person have only one residence at  any point in time, current residence recognizes that the place of residence  does not have to be the same throughout a year, allowing ACS data to  more closely reflect the actual characteristics of each area. The Census  Bureau plans to use current residence because the ACS is conducted every  month and produces annual averages rather than point-in-time estimates,  unlike the Decennial Census. Current residence is uniquely suited to the  ACS, because it continuously collects information from independent  monthly samples throughout all months of all years. Because the ACS is  designed to produce a continuous measure of the characteristics of states,  counties, and other places every year, the new residence rules were  needed for seasonal and migratory individuals.
section_title: Reference Period
paragraphs: The underlying population and housing characteristics data for the   2000 Census long form were for April 1, 2000. For the ACS test program,  the underlying population and housing characteristics varied. For all years  except 2000, they were for July 1; for 2000, they were for April 1. For the  full ACS, because the data are collected monthly, the reference period will  be the average for the year, and the Census Bureau will assume this  average is equivalent to data for July 1.
section_title: Independent Controls for Population and Housing Characteristics
paragraphs: The ACS will use population characteristics (age, sex, race, and ethnicity)  and housing characteristics (occupied and vacant units) derived from an  independent source and not from the results collected in the survey. Using  independent controls for these characteristics is standard practice to  correct sample survey results for the effects of nonresponse and  undercoverage. Population and housing characteristics from the   2000 Census short form were used as independent controls for the   2000 Census long form, down to the tract level. For the ACS  supplementary surveys, independent controls were from ICPE, which uses  Decennial Census short-form data as benchmarks and administrative  record data to interpolate between and extrapolate from the census  benchmarks. ICPE develops and disseminates annual estimates of the total  population and the distribution by age, sex, race, and Hispanic origin for  the nation, state, counties, and functioning government units. ICPE  provides annual estimates of population and housing characteristics at the  county level, and for some subcounty levels, as of July 1, using the usual  residence concept for seasonal residents.
section_title: Dollar-Denominated Data Items
paragraphs: According to current Census Bureau plans, annual estimates of dollar- denominated data items, such as income, rent, and housing-related  expenses, will be presented after adjustment for inflation in order to  facilitate comparisons over time. As in the ACS test programs, only annual  estimates with this adjustment will be presented. The Census Bureau also  has decided to continue to adjust annual data collected each month in the  ACS to a calendar year basis. It will be using the Consumer Price Index  (CPI) for the annual and monthly adjustments for all geographic areas.
section_title: Operational Differences
paragraphs: The long form and ACS will also differ in how operations are conducted,  such as nonresponse follow-up and data capture. For the 2000 Census long  form, nonresponse follow-up was conducted for all nonrespondents. For  the ACS supplementary surveys and for the full ACS, nonresponse follow- up will be conducted for a sample of one-third of all nonrespondents. For  the 2000 Census long form, all data items were entered using automated  optical character recognition procedure; data from the ACS will be  manually keyed.
section_title: Group Quarters
paragraphs: The ACS supplementary surveys excluded persons living in group  quarters. Group quarters—which include nursing homes, prisons, college  dormitories, military barracks, institutions for juveniles, and emergency  and transitional shelters for the homeless—accounted for roughly   2.8 percent of the population in 2000. The Census Bureau decided not to  cover these persons in the supplementary surveys, to avoid duplication  with the 2000 Census, and because it lacked funding to cover them in  subsequent years. Procedures for including in the ACS persons living in  group quarters beginning with 2005 are discussed in the Census Bureau’s  ACS Operations Plan, issued in March 2003. In addition, it has announced  that it intends to continue testing procedures to improve the mailing list  for group quarters to be used for the 2010 Decennial Census.
section_title: Outreach
paragraphs: The Census Bureau has long recognized the need to seek input from  stakeholders and users in making decisions for all its programs. The  Census Bureau sponsors technical reports that NAS prepares. (In  appendix II, we summarize recent NAS reports on the ACS and related  decennial censuses.) The Census Bureau has also held conferences on the  ACS and has contracted with Westat Inc. to organize two conferences of  experts on specific aspects of the ACS. Additionally, the Census Advisory  Committees, which are Census Bureau–appointed advisory committees  whose members represent professional associations such as the American  Statistical Association (ASA) and the American Marketing Association,  meet twice a year. The Census Bureau and other federal statistical  agencies also participate in the quarterly meetings of the Council of  Professional Associations on Federal Statistics, whose members include  professional associations, businesses, research institutes, and others  interested in federal statistics.
To obtain input from other federal agencies, the Office of Management and  Budget (OMB) established an interagency advisory committee for the ACS  in 2000. The committee’s major purpose was to coordinate the review of  questions to be included in the ACS. Because of the committee’s limited  focus, the Census Bureau established the ACS Federal Agency Information  Program in 2003, responding to a recommendation we made. This  program is designed to assist each federal agency that has a current or  potential use for ACS data to achieve a smooth transition to using the ACS.
section_title: Outstanding Issues Jeopardize ACS’s Replacement of the Long Form
paragraphs: From its beginnings in the mid-1990s, the Census Bureau’s development  plan for the ACS was designed to ensure that the ACS would satisfactorily  replace the Decennial Census long form as the major source of small  geographic area data. In our review of the plan, we found that the Census  Bureau, as well as key ACS stakeholders, had for many years identified the  key issues that needed to be resolved if the ACS were to reach this goal.
We have identified the following unresolved issues from our research  (described in appendix I):  the methodology to be used for deriving independent controls for  population and housing characteristics with ACS definitions of place of  residence and reference date, improvements needed to operational procedures,   methods for valuation and presentation of dollar-denominated data items,    comprehensive analysis of the comparability between new ACS data and  corresponding data from the 2000 Census long-form and 2004  supplementary survey, and the provision of user guidance on multiyear averages.
Despite the Census Bureau’s early identification of issues critical to the  successful replacement of the 2010 Decennial Census long form as the  new source of small geographic area data, we found that its plans to  resolve these issues have been only partially completed. Furthermore, we  found that despite recent changes to the ACS implementation schedule, it  is not fully synchronized with the Census Bureau’s time schedule for  implementing the testing program for the 2010 Decennial Census.  Consequently, if these issues are not resolved in a timely manner, the  Census Bureau’s plan to replace the 2010 Decennial Census long form with  the 2008–12 ACS averages for detailed geographic areas will be  jeopardized.
subsections: section_title: A Methodology for Independent Controls for Population and Housing Characteristics Is Lacking
paragraphs: It is standard practice to use independent controls for population and  housing characteristics to correct the results of sample surveys for the  effects of nonresponse and undercoverage. For the 2000 Census long form,  characteristics from the 2000 Census short form were used as independent  controls down to the tract level. For the annual ACS supplementary  surveys, characteristics from ICPE were used as the independent  controls.
Independent controls for the full ACS will require a new methodology.  Short-form data are available only once every 10 years, and the annual  ICPE estimates do not provide data for the detailed geographic areas  needed to prepare long-form detail and do not use the ACS residence  concept or reference period. The new methodology is critical to the  reliability of the ACS estimates of small geographic areas that ICPE does  not provide and of areas that have large numbers of seasonal residents.
Census Bureau staff have long recognized the need for the new  methodology. For example, a 1995 paper by Love, Dalzell, and Alexander  expressed concern about population controls and residence rules as well  as the need for consultation with users on these topics. They reported that  the Census Bureau was planning to conduct research using data from the  1996 test sites to produce controls at the census tract and block group  levels. They also noted that the Census Bureau would need to conduct  research on the residence rule. A 2000 paper by Alexander and Wetrogan  also discussed the issue of population controls. They reviewed possible  methods for using ICPE to develop controls for the ACS and noted the  need to consult with users on how to present information on the  differences in ACS controls and ICPE in ACS publications.
Key stakeholders, including experts on the ACS we interviewed in August  2003 (listed in appendix I), expressed similar concerns about the  methodology.
It appears that no progress had been made on a new methodology until the  Census Bureau reported in October 2003 to its advisory committees on the  status of a new methodology to derive controls. It announced that when  full ACS collection starts in November 2004, (1) interim procedures would  be used and (2) a final methodology would not be determined until after  the necessary research was completed. The Census Bureau did not  provide a date when the methodology would be incorporated.
In our review of Census Bureau presentations about the new methodology  (described in detail in appendix V), we found that it had no plans to  maintain time-series consistency of the population and housing controls  by routinely incorporating the regular revisions to ICPE estimates into the  ACS. Without such revisions, there could be a significant lack of  comparability in the ACS data being averaged, and the reliability of  multiyear estimates would be reduced. For example, without such  revisions, the 2008–12 averages that are to replace the 2010 Decennial  Census long form would be based on controls extrapolated from the   2000 Census for 2008–09 and controls from the 2010 Census for   2010–12. In addition, time-series consistency in the annual ACS data would  be reduced, especially in the data for 2010 and previous years. Census  Bureau officials told us that they were not planning any such revisions,  unless the inconsistencies between 2010 ICPE and 2010 Census  characteristics were significant, even though there were significant  inconsistencies between the 2000 ICPE estimates and the 2000 Census  data, especially for small geographic areas.
We found that regularly incorporating all revisions to ICPE into the ACS  would improve ACS reliability and that planning would give users advance  notice on the Census Bureau’s revision practice. The need for such  planning is critical, as evidenced by the failure that occurred in January  2004, when a revised set of ICPE data was incorporated into the  calculation of monthly CPS data on employment. Initially, the revised  employment estimates were released without a revision of the   pre-2004 data, resulting in a significant discontinuity between December  2003 and January 2004. As a result of users’ dissatisfaction about the  discontinuity, a consistent set of employment estimates was released.
Finally, failure to adequately involve stakeholders in the decision process  may contribute to significant misunderstanding about the use of the ACS  estimates and corresponding estimates from the Decennial Census. In past  decennial censuses, except for the very smallest geographic areas, the  population and housing characteristics data published as part of the long- form detail were the same as the official data based on data collected on  the short form. Because of differences in the residence and reference  period concepts and the use of multiyear averages for small geographic  areas, there will be less consistency between the ACS averages for   2008-12 and the 2010 Census data.
section_title: Operational Issues Have Not Been Addressed
paragraphs: The Census Bureau has identified operational issues with the ACS test  programs, primarily from its evaluation studies on the 2000 Decennial  Census and Census Bureau staff research papers on comparisons between  data collected in the ACS 2000 Supplementary Survey and the 2000  Decennial Census long form. These issues (described in detail in appendix  V) include problems with questionnaire design, nonresponse followup, and  data capture, as well as coverage of persons living in group quarters.
For example, the Census Bureau conducted a study to evaluate the design  of the ACS questions that are needed to implement the residence concept  and reference period for the ACS. The study suggested that additional  testing was needed for the questions about multiple residences and noted  “that asking these questions on a person basis may produce different and  probably better data than asking them on a household basis.” Similarly,  the authors found potential problems with the identification of seasonal  residents. We were not able to identify in the Census Bureau’s plans  whether these issues would be addressed before implementation of the  full ACS.
We also found, for the implementation of the full ACS for 2005, that the  Census Bureau had addressed only the inclusion of group quarters and  that it may not resolve the issue of questionnaire design until 2010. In  addition, even for group quarters, it is planning for improvements that may  not be included until 2010. Furthermore, not all problems have been  identified because of the delays in the Census Bureau’s completing the  evaluation studies of comparisons of long-form and ACS data items.  Moreover, the Census Bureau’s plans do not provide for external  consultations on key decisions about resolving issues.
Although the Census Bureau has acknowledged the importance of the  timing of incorporating changes to resolve the various issues, any delay in  implementing solutions to 2010 would not meet the needs of the ACS  collection and production schedule. For example, in its March 2003 ACS  operations plan, the Census Bureau recognized the need for maintaining  questionnaire continuity to calculate consistent multiyear averages. It also  has reported that it needs to incorporate changes in the ACS questionnaire  no later than 2008 because changes introduced after 2008 and before   2013 would create inconsistencies in calculating the 5-year averages that  are to replace the 2010 Decennial Census long form. Nevertheless, we  found that the Census Bureau’s current time schedule does not call for  resolving issues such as questionnaire design before 2008.
Incorporating changes into the ACS beginning with 2008 will help maintain  the reliability of the 5-year averages for small geographic areas; failing to  incorporate them beginning with 2005 will reduce the reliability of the  annual changes in the ACS data.
With regard to external consultation, we found that the Census Bureau’s  plans do not include time for consulting with stakeholders and users,  despite NAS, BLS, and Census Advisory Committee suggestions and  recommendations. For example, in a February 15, 2001, report to the  Census Bureau, the NAS Panel on Research on Future Census Methods  recommended that it conduct evaluation studies on “the effectiveness of  operations used to designate special places and enumerate the group  quarters and homeless populations.” Members of the Census Advisory  Committee had raised similar concerns. In a 2003 report prepared for BLS,  their consultant had made a number of recommendations about the  questions on employment.
We found that the Census Bureau needs to develop a time schedule so that  changes can be introduced to minimize inconsistencies between the   2005 and subsequent ACS data and to ensure that all necessary changes  are made so that the ACS data for 2008–12 that will replace the 2010  Decennial Census long form will be collected consistently. In addition, the  prompt completion of the ACS—long-form comparison studies and related  evaluations will provide sufficient time for the Census Bureau to consult  with stakeholders and to provide users with the information they need to  understand the effect of making changes to the ACS questionnaires or  procedures between 2005 and 2008.
section_title: Plans for Valuation and Presentation of Dollar- Denominated Data Items Are Questionable
paragraphs: When the Census Bureau began releasing data from the ACS test  programs, all dollar-denominated items such as incomes, housing values,  rents, and housing-related expenditures were adjusted for inflation. As in  the ACS test programs, only annual estimates with this adjustment will be  presented, and when the Census Bureau releases ACS data for each new  year, it revises all dollar-denominated data for prior years. It makes a  similar inflation adjustment for the annual income data collected in the  CPS, but it releases the unadjusted estimates. The Census Bureau also  has decided to continue to adjust annual data collected each month in the  ACS to a calendar year basis. It will be using the CPI for the annual and  monthly adjustments for all geographic areas.
The treatment of dollar-denominated data items is critical to all users of  these data. It is particularly critical for federal agencies that will be using  the ACS instead of the long form for many government programs to  determine the allocation of funds or program eligibility. It is also critical to  users of dollar-denominated items for small geographic areas because the  inflation adjustments under the current procedure are based on a national  average index.
In our review of the development and implementation of the ACS, we  identified questions on the appropriateness of the methodology for the  adjustment and the suppression of the unadjusted annual values. A report  prepared for HUD found problems with the calculation of the adjustment  and the use of the adjustment for income measures used for HUD  programs. The report also noted that the lack of the unadjusted annual  data would severely limit HUD’s use of calculations appropriate to its  program needs. Research by Census Bureau staff questioned the  adjustment for incomes when they found that it was a probable source of  difference between income data from the supplementary survey and  corresponding data from the CPS and the 2000 Census long form. (We  discuss these findings in detail in appendix V.)
Our statisticians reviewed these findings and found a similar problem with  the calculation of the adjustment because of the lack of trending  adjustment. We found that the Census Bureau could estimate calendar  year values using a combination of past trends in related series,  information from other ACS respondents, or known information such as  changes in cost-of-living adjustments for various transfer payment  programs and changes in wage rates. We also found that converting ACS  data from monthly to calendar year data is similar to conversion issues  faced by other agencies that collect annual statistics compiled on a fiscal- year basis and that the procedures these agencies use could be adapted for  the ACS. With regard to the use of a national cost-of-living adjustment,  we have previously reported that for purposes such as allocating federal  funds to states using income and poverty data, the CPI, a national measure  of inflation, does not reflect variations in geographic areas. Census  Bureau staff have reported similar findings.
The HUD and Census Bureau findings and our review raise serious  questions about the inflation adjustments. We found no documentation  explaining the rationale for the adjustment for either the ACS or the CPS,  where its use is limited to income data. Bureau officials informed us that  alternative procedures had not been examined and that stakeholders or  users had not been consulted on the adjustment.
section_title: Evaluations of Comparisons Are Incomplete and Users Lack Information on ACS Time- Series Consistency
paragraphs: We noted above that one of the Census Bureau’s major justifications for  the ACS test programs has been its comparing data collected in these  programs, and corresponding data from the 2000 Decennial Census short  and long forms, to identify operational problems. Another major  justification for the ACS test programs has been the use of these  comparisons, and comparisons with corresponding data from the CPS, to  inform users in making the transition from the 2000 long form to the ACS. “These data will also contribute to a comparison with data from Census 2000 that is  necessary because there are differences in methods and definitions between the census  and the ACS. Moreover, decision makers will want to compare an area’s data to those from  Census 2000. Comparisons using data from the operational test and from the 31 sites are  essential to determine how much measured change between Census 2000 and future years  of the ACS is real and how much is due to operational differences between the ACS and the  census.”
Despite acknowledging the importance of these comparisons, the Census  Bureau’s publication of evaluations of the comparisons has been delayed,  and their scope has been reduced in terms of levels, data items, and time  period. The lack of information will create problems for ACS users who  will be comparing the annual ACS data for 2005 (to be released in mid- 2006) with 2000 Decennial Census data or comparing annual ACS  supplementary survey data beginning with 2000.
In addition to delaying the release of the evaluation studies, the Census  Bureau has reduced their scope. For the evaluations of ACS test site data,  local experts did not participate in the evaluation of the comparisons for  27 of the 31 test sites. For the 4 test sites that were studied by local  experts, they did not cover subcounty local government units. For  evaluations of ACS supplementary survey data, the Census Bureau has  eliminated the analyses of comparisons between (1) the 2000  supplementary survey and the 2000 long form for geographic areas with  populations of 250,000 or more and (2) the supplementary surveys for  2000–02 to corresponding data from the CPS. It has reduced the scope of  its evaluation studies by also eliminating comparisons of single-year  estimates for most subnational areas and comparisons of data items such  as financial characteristics of housing.
NAS found that the Census Bureau has not placed sufficient priority on  completing the necessary evaluation studies. Furthermore, we found that  the Census Bureau does not have a plan that includes the timely  completion of all the studies. Once the studies are complete, it will need to  incorporate the findings into ACS operations, consult with stakeholders,  and provide users with the information they need to make the transition  from the long form to the ACS. The plan will be needed to ensure that as  many changes as possible can be introduced before the first annual ACS  estimates are published in 2006 and that all necessary changes are  implemented before 2008.
We found that the delays in completing the evaluations and their reduction  in scope are likely to affect the use of the ACS in improving the small  geographic area estimates of unemployment and poverty. For example,  Labor uses the unemployment data extensively to administer a variety of  federal programs. Several other departments use the poverty rates for  similar purposes.
section_title: Users Are Not Informed on Key Properties of Multiyear Averages
paragraphs: One of the major differences between the ACS and the long form is that  the ACS will provide data for geographic areas with populations smaller  than 65,000 in terms of multiyear averages. Experts outside and inside the  Census Bureau have identified serious issues regarding the statistical  properties of multiyear averages and have recommended that the Census  Bureau provide guidance to federal agencies and others on their use. We  found that stakeholders have urged the Census Bureau for many years to  provide guidance on the strengths and weaknesses of these averages. The  most recent request for guidance on using multiyear averages came in the  July 2003 report by the NAS Panel on Research on Future Census  Methods: “The Census Bureau should issue a user’s guide that details the  statistical implications of the difference between point-in-time and moving  average estimates for various uses.” In the report’s executive summary,  the panel also stated that “The Census Bureau must do significant work in  informing data users and stakeholders of the features and the problems of  working with moving average-based estimates.” It also expressed  particular concern about the use of the multiyear (or moving) averages in  fund allocation formulas.
Stakeholders have requested guidance on topics such as (1) the reliability  of multiyear averages for areas with rapidly changing populations, (2) the  reliability of trends calculated from annual changes in multiyear averages,  and (3) the selection of ACS data for geographic areas with populations  larger than 20,000 for which there will be multiple estimates. The Census  Bureau has recognized the need for such guidance but has not announced  any information about its contents or when it might be available, even  though the guidance is needed well in advance of the release of the first  multiyear averages in 2008.
We also found that plans for research to evaluate the statistical properties  of multiyear averages are limited. The contracts to evaluate 3-year  averages for the ACS test sites cover only averages for 1999–2001, with no  comparisons with averages for 2000–02, 2001–03, or 1999–2003. In  addition, the evaluation studies discussed earlier lack any time-series  dimension, such as comparisons of the supplementary surveys with annual  data from the CPS. Thus, it appears that the Census Bureau has missed the  opportunity to test (1) distortion and stability in multiyear averages,   (2) differences between multiple estimates for the same geographic areas,  and (3) the use of annual ACS data for small geographic areas.
section_title: Meaningful External Consultation on Key Issues Is Needed
paragraphs: We found that in recent years, the Census Bureau has used its outreach  efforts with stakeholders and users primarily to gain support for the ACS.  Although it also has solicited advice from NAS panels, advisory committee  members, and experts at workshops and conferences on some of the  issues we have identified in this report, there is no indication that the  Census Bureau will be following this advice. (For additional information,  see appendix V.) Likewise, it has not yet followed similar advice from us,  other government agencies, or even its own staff.
It has been more than a year since the Census Bureau announced, in  March 2003, that it was looking into establishing an ACS partnership  program that would involve advisory groups and expert panels to help it  improve the program. We found that no such program has been  established yet. Given that many key issues remain unresolved and that  the Census Bureau has no plans to seek advice on resolving them, key  aspects of the ACS will receive little or no input unless the Census Bureau  revises its plans.
section_title: Improving Timeliness and Quality of Small Geographic Area Data Would Increase Costs
paragraphs: In 1994, the Congress began to fund testing of the survey to replace the  2000 Decennial Census long form, beginning with the 2000 Census. In  reviewing the development of the ACS, we found that the Census Bureau  was planning to replace the 2000 long form by starting the ACS program  with an annual sample of 4.8 million housing units for 1999, 2000, and   2001 and reducing the sample for subsequent years to 3 million. The  larger sample would have provided 3-year averages for all small  geographic areas for 2000 and would have provided data for the smallest  geographic areas of the same quality as the traditional long form. In fiscal  year 1998, plans to introduce the ACS to replace the census long form  were delayed until after the 2000 Census was completed.
When the Census Bureau submitted its plans in 1998 to replace the long  form for the 2010 Decennial Census, a similar increase in sample size for  2009–11 was not proposed. Thus, compared with the plans for 2000, data  for small geographic areas for 2010 would be delayed by a year and would  be based on 5-year averages. When we reviewed the previous plan and  other alternatives to the proposed ACS that would provide more timely  and reliable data for small geographic areas, we determined that the only  viable alternative to the current plans would be to expand the sample size  for 2009–11, as proposed earlier.
This expansion would allow the Census Bureau to publish data for  geographic areas with populations smaller than 20,000 a year earlier, and it  would provide more reliable small- area data than under the currently  planned 5-year averages. In addition, if the Congress were to provide the  additional funds for this alternative, an additional year would be available  for the Census Bureau to resolve issues we have identified in this report by  giving it until the collection of data for 2009 rather than for 2008.  According to Census Bureau estimates, increasing the sample size for the   3 years would add about $250 million to the estimated $500 million cost for  the 3 years, using the smaller sample.
section_title: Resolving Outstanding Issues Needs a Time Schedule
paragraphs: The most recent Census Bureau schedule for implementing the ACS over  the complete cycle of the 2010 Decennial Census was prepared in  December 2003. Except for the completion of the questionnaire for the  2008 ACS, the milestones do not cover the resolution of issues that it has  already identified and issues we identify in this report. (See table 1.)  Ideally, all these issues should be resolved before the first annual results  of the full ACS sample are released. However, the Census Bureau has  already announced that final plans for calculating independent population  and housing controls with ACS residence and reference concepts will not  be available for several years, the 2004 test plans for the 2010 Decennial  Census will cover group quarters and residence rules, reports from the  2004 tests will not be completed until 2005, and the 2006 test plans for  2010 also cover group quarters.
In addition, the Census Bureau has announced that comparisons of   2000 ACS and 2000 Census long-form data critical to the transition to the  full ACS will be limited. Nevertheless, users who need the evaluation of  these comparisons to compare data from the 2000 Decennial Census long  form with data from the new ACS data or from the ACS supplementary  surveys would benefit from the early resolution of other issues. For  example, resolving issues before the release of the first 3-year averages  (2005–07) would improve the consistency between these averages and the  subsequent ACS data. Resolving all issues for the 2008 ACS is critical if  these data are to be fully consistent with the ACS data for 2009–12 and the  2008–12 averages are to be fully consistent with the 2010 Decennial  Census short-form data. As we noted above, the Census Bureau’s schedule  does call for timely completion of the 2008 questionnaire. However, if  questions to be included in the 2010 Census short form are changed during  the congressional and OMB approval processes, currently scheduled for  2008 and 2009, data collected on the 2010 Census short form will be  inconsistent with the ACS data.
section_title: Conclusions
paragraphs: The Census Bureau’s development of the ACS goes back several decades  and has included intensive research and field testing programs, as well as  substantial outreach efforts, in particular through the reports and  workshops at NAS. However, its current plan to begin full implementation  of the ACS for 2005 has several critical deficiencies. The Census Bureau  has not completed its testing program, and it has not acted to resolve key  issues already identified by the ACS test program, by evaluation studies of  the 2000 Decennial Census, by Census Bureau research studies, and by  stakeholders and users, including us, NAS, and other federal agencies.  Furthermore, the ACS implementation plan and the 2010 Decennial  Census test programs are not synchronized, and there is no  comprehensive program for external consultation on the resolution of  these issues. Without prompt resolution of issues such as those relating to  the calculation of independent controls for small geographic areas and the  consistency of data used to calculate multiyear averages, the ACS will not  be an adequate replacement for the long form in the 2010 Decennial  Census. If the Census Bureau is not able to use the ACS to replace the long  form, the Congress and other stakeholders need to be advised in 2005 in  order to allow for the Census Bureau time to reinstate the long form for  the 2010 Census.
section_title: Recommendations for Executive Action
paragraphs: To ensure that the ACS is an adequate replacement for the Decennial  Census long form, we recommend that the Secretary of Commerce direct  the Census Bureau to (1) revise the ACS evaluation and testing plan and  focus on the issues we have identified in this report; (2) provide key  stakeholders, such as the National Academy of Sciences, with meaningful  and timely input on decisions relating to these issues; and (3) make public  the information underlying the Census Bureau’s decisions on these issues  when it makes the decisions. We also recommend that the Secretary direct  the Census Bureau to prepare a time schedule for the 2010 Decennial  Census that provides for resolving these issues by incorporating all  operational and programmatic changes into the 2008 ACS so that the   5-year averages for 2008–12 will adequately replace the 2010 Decennial  Census long-form data for small geographic areas. These revisions should  be reflected in the single, comprehensive project plan for the 2010 Census,  as we have previously recommended.
section_title: Agency Comments and Our Evaluation
paragraphs: In written comments on a draft of this report, the Secretary of Commerce  provided comments on our recommendations. (The Secretary’s comments  are reprinted in appendix VI.) He disagreed with our recommendation that  the ACS evaluation and testing plan needed to be revised to focus on  issues we have identified in this report, stating that the current ACS testing  and evaluation plan already included these issues. In following up on the  Secretary's response, we learned that there is not yet a written plan, but  only a rough outline of the types of work planned. Therefore, we believe  our recommendation remains valid. The Secretary did not accept our  recommendation to provide key stakeholders more direct and timely input  into decisions on these issues because he believes that the present  consultation process is adequate. We disagree, because as noted in  Appendix II of our report, the Census Bureau has not been responsive to  recommendations from several National Academy of Sciences reports  relating to the ACS. The Secretary agreed with the recommendation that  the Census Bureau provide public documentation for key decisions on  issues we have identified in this report. The Secretary did not respond  directly to our recommendation that he direct the Census Bureau to  prepare a schedule for the 2010 Census that ensures that all necessary  changes are made in time for the 2008 ACS so the 5-year ACS averages for  2008-2012 will be an adequate replacement for the 2010 long form for small  geographic areas.
The Secretary provided comments on the five major outstanding issues  that, in our view, jeopardize the ACS as a replacement of the long form:  lack of methodology for independent controls, operational issues not  addressed, questionable plans for dollar-denominated items, incomplete  evaluations and lack of information on ACS time-series consistency, and  lack of information about multiyear averages.
The Secretary disagreed with our findings about the lack of a methodology  for independent population and housing controls.  He stated that a  methodology for the ACS was already in place. On the issue that changes  to that methodology are needed to account for the difference in the ACS  residence concept, the Secretary agreed that a change was needed but  stated that it could be delayed for several more years. On the issue of  independent controls for subcounty areas, he stated that the Census  Bureau had no plans to develop such controls, which we found were used  for the 2000 Census long form, but that it might develop such controls  using data from the ACS or administrative records. However, he did not  respond to our findings about the use of existing subcounty area data from  the ICPE or from the 2010 Census short form. The Secretary stated that  the Census Bureau also had no plans to revise the ICPE. On the issue of  the ACS reference period, the Secretary reported that the Census Bureau  had recently decided to assume that July 1 would be used as the reference  period. The Secretary did not comment on GAO’s findings about the lack  of plans to incorporate into the ACS 2010 Census data and related  revisions to the ICPE estimates for previous years.
We disagree with the Secretary’s comments about the independent  subcounty population and housing controls and believe that their use in  the ACS is needed for the ACS to be an adequate replacement for the 2010  Census long form for small geographical areas. We found that independent  controls from the 2000 Census short form were used for detailed  geographic areas for the 2000 Census long form and that differences in  counts of population and housing (occupied and vacant) between the long  form and the short form were limited to the smallest geographic areas.   The similar use of 2010 Census short-form counts in the ACS also would  minimize differences in these counts from the ACS and the 2010 Census.  Consequently, we disagree with the Bureau’s plan not to commit to the  development of subcounty controls and its plans not to base these  controls on ICPE total population and housing estimates, which are  prepared annually for all general government units, and the more detailed  and reliable data from the 2010 Census short form.
We also disagree with the Secretary that the implementation of a new  methodology for independent controls with subcounty controls and the  new residence concept can wait until 2008. As we noted in our report, we  found that controls for subcounty areas with population of more than  65,000 will be needed before the 2005 ACS estimates are released for these  areas in 2006 and that controls for subcounty areas with populations of  more than 20,000 will be needed before the first multiyear averages are  released in 2008.  (For the 2000 Census long form, controls for most areas  of this size were from the 2000 Census short form.) With regard to the new  residence concept, a decision to delay introducing a new methodology  until 2008 would create time-series inconsistencies between 2000–2007  and 2008 and subsequent years. These inconsistencies could be very  significant for geographic areas with a large population of seasonal  residents.
The Secretary did not comment on our findings about the need for a  methodology to revisions relating to the ICPE into the ACS. We found that  this methodology, which is important to both the time-series consistency  of the annual ACS estimates and to the multiyear averages, is not covered  by the current ACS methodology, but that it will be needed when the 2010  Decennial Census short form data become available. We found that it has  been the Census Bureau’s practice for the ICPE, whose estimates are used  as the independent controls for the ACS, to be benchmarked to the  decennial census short-form data and that it uses similar practices for  many other Census Bureau programs. For the ICPE, the Bureau will  replace the 2010 ICPE estimates with the 2010 Census data, and use the  differences in these estimates to revise the ICPE estimates back to the  previous benchmark year, which for 2010 will be 2001. (Table 4 of our  report shows the impact of benchmarking on county population data for  2000.) It should be noted that we found that this practice is not followed in  all Census Bureau programs. For example, for the Annual Economic and  Social Supplement to the CPS, the Census Bureau introduced the  benchmark information from the 2000 Decennial Census into the 2001  estimates and presented the data on both the old and the revised basis.   This approach, to present estimates on an old and new basis for a single  year, may be appropriate for an annual survey. However, GAO found that  because of the use of multiyear averages in the ACS, it is imperative that  the ACS estimates for all years beginning with 2001 be revised. Without  such a revision program, ACS estimates for 2010, which we assume will  not be released until the 2010 Census short-form data have been  incorporated, will be inconsistent with the 2009 estimates. In addition, the  ACS estimates for 2008 and 2009 used to calculate the 5-year averages that  will replace the 2010 Census long form will be based on controls that are  inconsistent with those for 2010–12. Based on the revisions for 2000 shown  in our report, there could be many significant inconsistencies, especially  for small geographic areas.
Although the Secretary did not comment on the issue of revision, in its  technical comments on our draft report, the Census Bureau reported  (comment 22) that with regard to incorporating 2010 Census data, it has  decided “to make appropriate changes to the  population controls  when necessary, including the possibility of reweighting the data around  the 2010 time period and for all multiyear estimates.” We disagree with the  Census Bureau’s approach primarily because it is not consistent with the  practices used by the Census Bureau to incorporate census data into  surveys and programs such as the ICPE and monthly retail sales that are  controlled or benchmarked to a census or similar data set.  For these  surveys, it revises all previously published data on a predetermined  schedule using a transparent statistical procedure. Most important, these  procedures do not depend on the size of revisions, which can only be  determined after a benchmark is completed. Regardless of the  benchmarking procedures adopted for the ACS, we believe that the  Census Bureau needs to have extensive consultation with external  stakeholders to make its decision. In addition, because of the complexity  of most benchmarking procedures, the Census Bureau needs to begin this  consultation as soon as possible.
With regard to the recent Census Bureau decision about the reference  period for the ACS, we are pleased that a decision has been made because  any delay in this decision would have resulted in additional time-series  inconsistencies in the ACS. We have changed our report to reflect this  decision. Unfortunately, we have no documentation on the research  underlying the decision and, as has been the case in other key decisions,  we do not believe that there was any public discussion of this decision.
The second issue identified in our report related to the operational aspects  of the ACS, including questionnaire design and the collection of data for  persons living in group quarters. On these issues, the Secretary limited his  comments to the questionnaires and addressed our findings that  improvements identified as part of the 2000 Census cognitive testing  research and research based on comparisons of ACS and 2000 Census  long-form data would not be completed until 2008. The Secretary noted  that the Census Bureau has resolved the issue of finalizing the ACS  questions, including the questions to be asked on the 2010 Census short  form before 2008. Although this recent decision appears to have resolved  the scheduling issue, we believe that uncertainties remain as to whether  this schedule can be met. For example, the ACS milestones in the latest  available schedule call for final approval of the questions by the Congress  and by OMB in 2008 and 2009, respectively, so that any changes made as a  result of these steps would not be incorporated into the 2008 questions.
As the Census Bureau has recognized, failure to maintain consistency in  the questions for the 2008-2012 ACS will result in inconsistencies in the 5- year averages centered on 2010, which are the averages designed to  provide the small geographic area data that would have been collected on  the 2010 Census long form. In addition, the recently released ACS  evaluation reports identify issues on which new research is necessary,  including the issues with the questions on disability identified in our  report, but the Census Bureau has not indicated its plan to complete this  additional research or to consult with stakeholders about decisions related  to the research. Although the Secretary did not comment on our findings  with regard to group quarters, we remain concerned that the work on  group quarters being conducted as part of the 2004, 2006, and 2008 tests  for the 2010 Census will not be reflected in the ACS beginning with 2008.
Our report also identified as unresolved issues the two inflation  adjustments that the Census Bureau is applying on all dollar-denominated  ACS items. The first adjustment is used to convert annual data collected  each month in the ACS to a calendar year basis. This adjustment  recognizes that the annual data collected in the ACS are for different  periods because the data are collected monthly and cover the previous 12  months. The second adjustment is used to present dollar-denominated  items in dollars of the most recent calendar year. This adjustment  eliminates the impact of inflation when comparing data across years. The  index used for both adjustments is the national-level CPI.  The Secretary  correctly observed that the CPI is a generally accepted measure of  inflation and that most federal programs that allocate funds do not use  regional measures of inflation. However, these observations did not  directly address GAO’s findings about the adjustments or the concerns  raised by HUD in its report on future use of the ACS, which are discussed  in appendix V of our report. For example, the Secretary did not address  our finding about a lack of a rationale for adjusting items other than  incomes for changes in overall inflation rather than adjusting with indexes,  such as wage rates or rent, that are directly related to the item being  adjusted. He did indicate that the Census Bureau would reconsider its  present policy of showing only the inflation-adjusted annual estimates and  multiyear averages. We believe our findings about the need for the Census  Bureau to provide a comprehensive rationale for the two adjustments still  apply.
The Secretary disagreed with the issue we identified on completeness of  the Census Bureau’s comparison and evaluation reports. He noted that  after our draft report was completed, the Census Bureau released seven  additional comparison reports and that it planned to prepare additional  reports to evaluate issues we identified on the time-series consistency of  the annual ACS estimates. However, despite earlier statements by the  Census Bureau to compare and evaluate differences between state-level  estimates from the Census 2000 Supplementary Survey (C2SS) and the  2000 Census long form, these reports did not include any reference to the  preparation of such comparisons, and the Secretary did not indicate they  would be prepared. Because the focus of the long form and the ACS is on  data from small geographic areas, we believe that reports on states and on  other areas with population of 250,000 or more should be prepared.
The last issue we identified was the need to provide users with guidance  on the interpretation of key properties of multiyear averages. The  Secretary agreed about the need but noted that guidance is not needed in  2005. He reported on a newly created NAS panel that will be studying  many of the key issues identified in our report. However, we believe that  the Census Bureau should begin to release guidance on the averages  before the first multiyear averages are released in 2008. One area in which  such guidance will be needed is the interpretation and use of the multiple  ACS estimates. When the 2005–07 averages are released in 2008, users will  have annual estimates for some of these areas for 2006 as well as the 3- year averages, which will be centered on 2006. In 2010, when the first 5- year averages are released (2005–09), users will have three sets of ACS  estimates for places with populations larger than 20,000. For example, for  each state, there will be an annual estimate for 2007 as well as 3-year and  5-year averages centered on 2007.
The comments from the Secretary also include a list of detailed technical  comments from the Census Bureau. We reviewed each of these comments  and revised the report where appropriate.
As agreed with your offices, unless you release the report’s contents  earlier, we plan no further distribution of it until 30 days from its issue  date. We will then send copies to the Secretary of Commerce, the Director  of the U.S. Census Bureau, and others who are interested. Copies will be  made available to others on request. This report will also be available at no  charge on GAO’s Web site at http://www.gao.gov.
If you or your staff have any questions concerning this report, please  contact me at (202) 512-9750. Other staff who made major contributions to  this report are listed in appendix VII.
section_title: Appendix I: Scope and Methodology
paragraphs: We used a combination of approaches and methods to examine the Census  Bureau’s plans to develop, test, and implement the American Community  Survey (ACS). We reviewed published and unpublished ACS-related  Census Bureau reports, papers, presentations, budget documents, and  congressional testimony; National Academy of Sciences (NAS) reports;  congressional testimony delivered by outside experts; and consultants’  reports prepared for the Census Bureau, the Bureau of Labor Statistics  (BLS), and the Department of Housing and Urban Development (HUD).
We reviewed an extensive set of internal planning documents prepared  between 1992 and 1995 that the Census Bureau provided, relevant papers  Census Bureau staff presented at professional association meetings and  similar symposiums from 1995 on, and evaluation reports based on the  2000 Census. We also reviewed official Census Bureau presentations in  special reports, congressional testimony, and recent advisory committee  meetings. We reviewed similar materials NAS and consultants prepared for  the Census Bureau and other federal agencies, as well as materials we  prepared. The most important documents we reviewed are listed in the  bibliography, organized by document type, at the end of this report.
In addition, we conducted independent research and analysis. To assess  the evaluations the Census Bureau conducted to assist users in making the  transition from the 2000 Census long form to the ACS, we obtained data  from the 2000 Census and 2000 ACS (the Census 2000 Supplementary  Survey) and prepared comparisons of key detailed data items at the state  level. To determine the potential effect of replacing independent  population and housing characteristics controls from the 2000 Census with  corresponding data from the 2010 Census, we compared county-level  intercensal estimates for April 1, 2000, based on the 1990 Census, with  2000 Census counts. We also analyzed the Census Bureau’s use of  independent controls for estimates of population and housing  characteristics for previous decennial censuses and its plans for the ACS.
section_title: Appendix II: Recent NAS Findings on Continuous Measurement and the ACS
paragraphs: “Although we believe that the proposed continuous measurement system deserves serious  evaluation, we conclude that much work remains to develop credible estimates of its net  costs and to answer many other fundamental questions about data quality, the use of small- area estimates based on cumulated data, how continuous measurement could be integrated  with existing household surveys, and its advantages compared with other means of  providing more frequent small-area estimates. In our judgment, it will not be possible to  complete this work in time to consider the use of continuous measurement in place of the  long form for the 2000 census.”  “With regard to proposals to drop the long form in the next decennial census and substitute  a continuous monthly survey to obtain relevant data, substantial further research and  preparatory work are required to thoroughly evaluate the likely effect and costs of these  proposals. Continuous measurement deserves serious consideration as a means of  providing more frequent small-area data; however, the necessary research and evaluation  cannot be completed in time for the 2000 census.”
Although 1994 saw the first proposals to implement the continuous  measurement methodology as a replacement for the 2000 Census long  form, the Census Bureau changed its plans in 1998, shifting to  implementation to replace the long form in 2010.
Since 1995, NAS has produced several reports that relate totally or in part  to the ACS, including a summary of a September 13, 1998, Committee on  National Statistics workshop at NAS, two interim reports, a letter report,  and a final report, by the Panel on Research on Future Census Methods,  and a report released in early 2004 by the Panel to Review the 2000  Census. (In this appendix, we do not discuss NAS reports after 1995 in  which the ACS was discussed as a potential data source for federal  programs.)
With few exceptions, the members of these two NAS panels and the  workshop participants reported findings that cover most of what we have  identified as unresolved issues and summarize in this appendix. The NAS  reports and ours differ somewhat in emphasis. We have focused on the  production and use of ACS data, whereas NAS focused more on data  collection and processing methodologies. These differences may reflect  the fact that NAS panel members are very sophisticated users who are  more likely to use ACS microdata files and make their own adjustments  for methodological issues; they make little use of the regular ACS  publications.
subsections: section_title: 1998 NAS Workshop on the ACS
paragraphs: NAS sponsored a 1-day workshop in September 1998 to discuss  methodological issues related to the ACS. Experts prepared “thought  pieces” on issues NAS staff selected, with input from Census Bureau staff.  The workshop’s specific discussion topics were combinations of  information across areas and across time, funding formula, weighting and  imputation, sample and questionnaire design, and calibration of the output  from this survey with that from the long form. The thought pieces and  comments on them prepared Census Bureau staff for the discussions at  the workshop. “has been on refining data collection, leaving the final answers to the difficult analysis  questions for later. Thus, procedures for nonresponse and undercoverage adjustment were  modeled, to the extent possible, after current procedures used for the census long form.
Now that data collection has matured as the ACS demonstration phase is well under way,  the Census Bureau is developing a research plan and initiating research to address all  issues relating to ACS methodology. Fall 1998 therefore seemed an opportune moment for  a workshop to assist the Census Bureau in developing a research agenda to deal with many  of these challenging issues.”
The report contained no specific recommendations but identified areas  where additional research was needed, including issues we have  expressed concern about, such as the availability of multiple ACS  estimates for geographic areas with populations larger than 20,000 and the  likelihood of differences between ACS estimates and estimates from a  Decennial Census short form. From our perspective, the most relevant of  the workshop’s specific issues were (1) combining information across  time, (2) weighting and imputation, and (3) calibrating the output from this  survey with that from the long form.
Technical papers in the workshop’s agenda book contained considerable  discussion of time-series issues. The discussion in this section of the  workshop focused on replacing moving averages with time-series  modeling and using current household survey data to develop models.  Speaking for the Census Bureau, Alexander stated that “Our current plan  is to release annual data for even very small areas and let users perform  their own time series analyses. We welcome ideas about what the Bureau’s  role should be . . .” “to make comparisons between the long form and ACS for all states, large metropolitan  areas, large substate areas, and population groups.  “The objective of the 1999–2001 comparison is to understand the factors associated with  the differences between the 1999–2001 ACS and the 2000 long form in the 31 areas, using  the second comparison study to develop a calibration model to adjust the 2000 long-form  estimates to roughly represent what the full ACS would have yielded in 2000.”
Chapter 7 of the report was devoted to a discussion of calibration. The  report stated that the model would “determine the effects that would be  expected when switching from the long-form estimates to those from the  ACS on various applications of long-form data.” Once adjusted, the  calibrated long-form data for 2000 can be compared with ACS data that are  collected following full field implementation in 2003, “in order to  understand the dynamics over time of such characteristics as poverty and  employment.” “We very much like the idea of viewing information from an ongoing comparison of ACS to  CPS and other surveys as a way to help understand how the ACS ‘error profile’ might be  changing over time and using this to help interpret ACS data in the context of the long-term  time series of census estimates.”
The use of independent controls for population and housing  characteristics was also discussed at the workshop, but very generally,  because the Census Bureau had not yet developed proposals for the  controls. For example, the report’s chapter 5 discussed improving the  existing population controls. The Census Bureau reported discomfort with  the quality of the existing county-level controls (from ICPE) and agreed  that the ACS could be used to improve these estimates. The Census  Bureau also acknowledged that differences in residence rules and  reference period would complicate the calculation of population weights.  However, no discussion was reported of how the population counts from  the 2010 Census would be used. “the development of estimates that (a) sum to estimates at higher levels of geographic  aggregation and (b) more closely approximate direct estimates at higher levels of  aggregation . . . in the event that aggregate estimates are not constrained to  (approximately) equal direct estimates (and also the release of direct estimates at lower  levels of aggregation for analysis purposes) . . . .”
section_title: 2000 Interim Report
paragraphs: The Panel on Research on Future Census Methods, sponsored by the  Census Bureau, was formed to examine alternative designs for the   2010 Census and to assist the Census Bureau in planning tests and  analyses to help assess and compare their advantages and disadvantages.  In addition to the first interim report, Designing the 2010 Census,  released in 2000, a letter report was issued in 2001, and a second interim  report was issued in 2003 (both discussed below). The panel issued a final  report in 2004. “The Census Bureau should develop a detailed plan for each evaluation study on how to  analyze the data collected and how to use the results in decision making concerning 2010  census design. The Census Bureau should then use these plans to identify the benefits and  resources required for each evaluation study, set priorities among them, and allocate  sufficient resources for the careful completion of all or, at least, the highest priority  evaluations.” “The American Community Survey is a proposed national, continuous, mailout-mailback  survey of 250,000 households per month, with field follow-up that makes use of techniques  closely related to those used in the census. Therefore, rather than rely exclusively on the  two or three large-scale census tests, which are always at least slightly limited in their  generalizability by the specific locations selected, the Census Bureau could use the ACS as  a platform for testing possible changes in the census. This work could serve as preliminary  testing to the larger mid-decade tests for the census design.” “The decennial census makes use of one residence rule definition, the ACS uses a second,  and a third approach is being tested in the alternative questionnaire study. As the Census  Bureau is well aware (based on the allocation of an experiment to this issue), confusion  over residence rules is a source of possibly substantial error in the census. . . . The Census  Bureau needs to find the residence rule (within the set of rules satisfying legal and other  restrictions) that results in the most accurate estimates. To learn more about this issue, the  panel proposes an ACS-short-form match study in 2000 to examine this and other short- form measurement error issues.” “The development of the ACS raises a number of issues related to the quality of and  planning for the 2010 census. There are also many other important technical issues raised  by the introduction of the ACS into the federal statistical system. Formation of a technical  working group could help to address many of these issues.”
section_title: 2001 Letter Report
paragraphs: The 2001 letter report—addressed from Benjamin King, Chair of the Panel  on Research on Future Census Methods, to William Barron Jr., Acting  Director of the Census Bureau—was prepared in response to a December  7, 2000, presentation by Census Bureau staff on the major elements of the  Census Bureau’s strategy for the 2010 Census. The panel recommended  that the Census Bureau produce a “business plan” for the 2010 Census that  would provide an overall framework for development. It recommended  that this plan include (1) a statement of objectives, (2) a timeline for  completing tasks, (3) a cost-benefit analysis, and (4) more complete  information on coordinating tasks within the Census Bureau. The panel  also recommended the preparation of specific types of evaluation studies. “The Bureau is currently conducting a wide array of evaluation studies and experiments  designed to assess the quality of the 2000 census and inform approaches to the 2010  census. As noted above, the panel applauds the scope of these evaluation studies. However,  the panel is concerned that the Bureau has not sufficiently focused its evaluation program  and has instead labeled most of its evaluation categories as high priority.” “comparison of estimates from the ACS and 2000 census long-form data, in sites where  both are available; coverage of the population, disaggregated by demographic and  geographic subgroups; the effectiveness of major automated systems for data collection,  capture, and processing; the quality and completeness of long-form data collection; and the  effectiveness of operations used to designate special places and enumerate the group  quarters and homeless populations.” “to broaden its justification for the ACS, detailing the need for and use of long-form data  and how those data needs will be addressed through the ACS, perhaps in conjunction with  the CPS and other demographic surveys. Accordingly, the Bureau should expedite ongoing  evaluations that assess the quality of ACS data relative to the quality associated with the  traditional census long form.”
section_title: 2003 Interim Report
paragraphs: “The most basic question the panel faces regarding the ACS is whether it is a satisfactory  replacement for the census long form. We recognize that significant estimation and  weighting challenges must be addressed and that more research is needed on the relative  quality of ACS and long-form estimates.” “The Census Bureau should carry out more research to understand the differences between  and relative quality of ACS estimates and long-form estimates, with particular attention to  measurement error and error from nonresponse and imputation. The Census Bureau must  work on ways to effectively communicate and articulate those findings to interested  stakeholders, particularly potential end users of the data.
The Census Bureau should make ACS data available (protecting confidentiality) to analysts  in the 31 ACS test sites to facilitate the comparison of ACS and census long-form estimates  as a means of assessing the quality of ACS data as a replacement for census long-form data.
Again, with appropriate safeguards, the Census Bureau should release ACS data to the  broader research community for evaluation purposes.
The Census Bureau should issue a user’s guide that details the statistical implications of  the difference between point-in-time and moving average estimates for various uses.
The Census Bureau should identify the costs and benefits of various approaches to  collecting characteristics information should support for the full ACS not be forthcoming.
These costs and benefits should be presented for review so that decisions on the ACS and  its alternatives can be fully informed.” “The fact that the Census Bureau has not done more in comparing the data collected from  the 31 test sites, the C2SS, and the 2001 and 2002 Supplementary Surveys with the data  collected by the 2000 census long form is disappointing. Such analyses would help assess  the quality of ACS data and would be helpful in making the argument for transition from  the long form to the ACS. This deficiency is probably due to limited analytic resources at  the Census Bureau and creates an argument for ‘farming out’ this analysis to outside  researchers.” “The ramifications of this basic concept emerge when moving average estimates are  entered into sensitive allocation formulas or compared against strict eligibility cutoffs. A  smoothed estimate may mask or smooth over an individual year drop in level of need, thus  keeping the locality eligible for benefits; conversely, it may also mask individual-year  spikes in activity and thus disqualify an area from benefits. It is clear that the use of  smoothed estimates is neither uniformly advantageous nor disadvantageous to a locality;  what is not clear is how often major discrepancies may occur in practice.” “It is incorrect to use annual estimates based on moving averages over several years when  assessing change since some of the data are from overlapping time periods and hence  identical. At the least, the results will yield incorrect estimates of the variance of the  estimates of change. Therefore, users should be cautioned about this aspect of the use of  moving averages.”
In both recommendations on evaluations and moving averages, the panel  called for the Census Bureau to engage in a greatly expanded effort to  inform users and stakeholders. It also suggested that the Census Bureau  farm out some of the research efforts. “Eight years later, faced with the task of offering advice on making the vision of continuous  measurement a reality in the 2010 census, the similarity between the arguments then and  now is uncanny. Similar, too, are the points of concern; the current panel is hard-pressed to  improve upon the basic summary of concerns outlined by our predecessors. We are,  however, much more sanguine that a compelling case can be made for the ACS and that it  is a viable long-form replacement in the 2010 census.”
However, while the panel was identifying its concerns, it also supported  full funding of the ACS, believing that existing “flaws” in the plan could be  resolved.
section_title: The 2000 Census: Counting under Adversity
paragraphs: “the Census Bureau, the administration, and Congress agree on the basic design for the  2010 census no later than 2006 in order to permit an appropriate, well-planned dress  rehearsal in 2008. In particular, this agreement should specify the role of the new American  Community Survey (ACS). Further delay will undercut the ability of the ACS to provide, by  2010, small-area data of the type traditionally collected on the census long-form sample and  will jeopardize 2010 planning, which currently assumes a short-form-only census.” “the Bureau should also study the effects of imputation on the distributions of  characteristics and the relationships among them and conduct research on improved  imputation methods for use in the American Community Survey (or the 2010 census if it  includes a long-form sample).” “publish distributions of characteristics and item imputation rates, for the 2010 census and  the American Community Survey (when it includes group quarters residents), that  distinguish household residents from the group quarters population (at least the  institutionalized component). Such separation would make it easier for data users to  compare census and ACS estimates with household surveys and would facilitate  comparative assessments of data quality for these two populations by the Census Bureau  and others.”
The panel’s findings were similar to our findings, with one major  difference. The panel’s findings imply that some research on the ACS can  be conducted after the results of the 2010 Census short form become  available. In contrast, we see that such research is needed in order to  improve the ACS by 2008, the first year in which ACS data will enter into  the calculation of the 5-year average estimates (2008–12) that will replace  the long form.
section_title: Appendix III: The Decennial Census Long Form and the Evolution of the ACS Plan
subsections: section_title: Decennial Census Long Form
paragraphs: In the decennial census for 1940 and for 1950, the Census Bureau used a  single form to collect, from all households, population and key  characteristics such as age and gender and, from a sample of households,  detailed demographic, economic, and housing items. In the 1940 Census,  the Census Bureau used a sample of 5 percent of the population to collect  data on questions on income, internal migration, and Social Security  status, as well as on more refined questions on unemployment. In addition,  the Congress authorized a new set of questions about the types of  plumbing, heating, and appliances in dwellings.
Beginning with the 1960 Census, the first conducted by mail, it became  necessary to use separate forms—a short form to collect population data  from all households and a long form to collect the detailed items from a  sample of households. In the 2000 Census, for example, the Census Bureau  conducted a sample of 17 percent of the population and asked   45 questions on the long form. Since 1960, the long form has evolved into a  cost-efficient way to collect data federal agencies need that minimizes  respondent burden. For 2000, for example, the long form consisted of   45 questions that the Census Bureau developed working through OMB and  with the consent of the Congress. Each question provided information  required by statute. Thus, the 2000 long form provided all federal  departments and agencies with critical data, and it was estimated that  these data were used to allocate more than $200 billion in federal funds.
section_title: Evolution of ACS Plan
paragraphs: In the 1950s, Census Bureau officials and users of Decennial Census data  had begun to develop a program to provide intercensal data on population  characteristics. The first major proposal to provide intercensal data called  for a mid-decade census that would provide information every 5 years. In  1976, the Congress enacted legislation to require a mid-decade census  beginning with 1985, but did not fully fund the program.
In the late 1980s, the Census Bureau shifted efforts to provide intercensal  estimates to a program based on CM methodology, or Continuous  Measurement. This approach would provide for more timely population  data as well as the detailed demographic, economic, and housing data  collected every 10 years by the Decennial Census long form. The program  would integrate a new sample survey, existing surveys, administrative  records, and statistical modeling. After a thorough analysis of alternatives  based on this methodology, the Census Bureau developed a plan similar to  the current ACS to replace the 2000 Census long form.
Initial $2.6 million funding for the CM program was included in the   2000 Decennial Census budget for fiscal year 1995. These funds were to  develop, test, and evaluate a CM program to replace the Decennial Census  long form and to provide more timely long-form type data. In the program  description in the budget documents, the Census Bureau reported that it  planned to develop the new program that would integrate a new sample  survey, existing surveys, administrative records, and statistical modeling.  Table 2 shows that about $330 million has been provided to fund the CM  program since 1995, with funding provided separately until 2003 and  additional funding from both the 2000 and 2010 Decennial Census  programs. Beginning with 2003, all funding has been provided as part of  the 2010 Census program. The Census Bureau requested   $165 million for fiscal year 2005.
In 1996 and 1997, funding was provided to field-test what became the ACS,  to replace the 2000 Census long form. The ACS was to begin in 1999 with  an annual sample of 4.8 million housing units for 1999, 2000, and 2001 and   3 million housing units for subsequent years. Under this plan, a 3-year  average of ACS data for 1999–2001 was to replace the 2000 Census long  form. It would provide the same detailed items and same level of  geographic detail as the traditional long form with about the same quality.  Annual ACS data would subsequently be provided for geographic areas  with populations of 65,000 or more, 3-year averages would provide ACS  data for geographic areas with populations larger than 20,000, and 5-year  averages would provide ACS data for small geographic areas, such as  census tracts, small towns, and rural areas. The 5-year average for   2010, 2020, and beyond would replace future Decennial Census long forms.
In the 1998 budget request, the Census Bureau shifted the timing for  replacing the long form from the 2000 Census to the 2010 Census. As a  result, it was funded to conduct annual supplementary surveys of   750,000 households beginning with 2000, in addition to the ACS testing at  four test sites (or counties). The Census 2000 Supplementary Survey,  known as C2SS, and the surveys for subsequent years were to be used to  test the feasibility of collecting long-form data at the same time as, but in a  separate process from, the Decennial Census. Data from C2SS and the  supplementary surveys were also to be used to test ACS data usability and  reliability and to evaluate operational and programmatic issues associated  with implementing the ACS. Also, the number of test sites was increased  to 31 by 1999. Funding to compare and evaluate differences between data  collected from the 2000 Census long form and the ACS testing programs  began in 1999, to develop data to expand coverage to group quarters and  Puerto Rico in 2001. Plans to integrate existing surveys, administrative  records, and statistical modeling into the new program were dropped in  2001.
The 1998 budget request also reported that the Census Bureau would  proceed with plans to replace the 2010 Census long form with an ACS  based on an annual sample of 3 million housing units, as with the previous  plan. Unlike that plan, the sample size for 2009–11 would not increase to  provide 3-year averages for 2010. This revised plan called for full  implementation of the ACS in 2003. Full ACS data for 2003 to 2007 would  have made 5-year averages available in 2008, 4 years before the long-form  sample statistics from the 2010 Census would become available. However,  budget decisions by the Congress delayed full implementation until the  fourth quarter of fiscal year 2004.
section_title: Appendix IV: Continuous Measurement ACS Testing and Development Program
paragraphs: The Congress initially provided funds for testing the CM methodology in  1994. As we have noted, the Census Bureau had begun formal testing of  the CM program in 1996 with an operational test of the ACS in four  counties; this test was expanded to 31 test sites by 1999. A second testing  program, the Supplementary Survey program, began in 2000 as a part of  the 2000 Decennial Census. The Census Bureau designed C2SS to test the  feasibility of collecting long-form data at the same time as, but in a  separate process from, the 2000 Decennial Census. Data from C2SS and  the same supplementary surveys, beginning with 2001, were also to be  used to test ACS data usability and reliability. According to the Census  Bureau, these surveys were to be used to examine technical, statistical,  and operational issues associated with implementing the ACS and to  document the key results in a series of reports.
Before field testing began, the Census Bureau had conducted an extensive  research program to identify the issues related to using the CM  methodology and to replacing the long form. The research program  resulted in a series of 20 reports, known as the Continuous Measurement  Series, between 1992 and 1995. These reports, most of which were  prepared by Charles Alexander, addressed a wide range of topics such as  replacing the 2000 Census long form, collecting intercensal population  data, and integrating the ACS with existing household surveys. The reports  on replacing the long form identified the key issues that needed testing,  and they served as the primary input to the Census Bureau’s ACS test  program. These issues included those subsequently tested by the Census  Bureau as well as the unresolved issues we identify in this report.
Following the CM reports, Census Bureau staff presented papers from  1995 through 2001 on ACS testing at various professional association and  similar meetings, as well as at a 1998 symposium on the ACS sponsored by  the Census Bureau. For example, the 1995 paper by Love, Dalzell, and  Alexander discussed issues related to the evaluation of the 1996 test site  results, expressing concern about population controls and residence rules  as well as the need for consultation with users. They reported that the  Census Bureau was planning to conduct research using data from the   1996 test sites to produce controls at the census tract and block group  level. They also noted that the Census Bureau would need to conduct  research on the residence rule.
Alexander and Wetrogan also discussed the issue of population controls in  their 2000 paper. They reviewed possible methods for using ICPE to  develop controls for the ACS and discussed using ACS estimates on the  foreign-born U.S. population to improve the Census Bureau’s foreign- migration component of the intercensal estimates. (They reported that this  effort would be part of what the Census Bureau had previously referred to  as the Program of Integrated Estimates.) They also noted the need to  consult with users on how to present information on the differences in  ACS controls and ICPE in ACS publications.
Several papers have focused on the key role of evaluating differences  among the ACS test data, census long-form data, and CPS data. Alexander,  Dahl, and Weidman reported in 1997 that during the demonstration period,  they would be working closely with experts familiar with specific test sites  to learn about the quality of the ACS estimates. For example, they  reported that the Census Bureau would be looking into sources of  differences between the 1999–2001 ACS test-site average estimates and the  2000 Census long-form results and using the results of differences between  the 2000–02 national sample and the 2000 long form to generate model- based estimates for small geographic areas. The authors noted that these  model-based estimates, based largely on information from test sites, would  be used to interpret changes between 2000 and future ACS estimates.
In another 1997 paper, Davis and Alexander reported the Census Bureau’s  action plan for evaluation studies. They called for evaluating the results of  all test sites and releasing the expert review of the analyses of the  differences between the 1999–2001 ACS and the 2000 Census long form.  The schedule called for releasing this information before beginning the  implementation of the full ACS. Alexander’s 1998 paper on completed  research, research in progress, and planned research included among the  four items for planned research a “close study of differences between  1999–2001 ACS and 2000 long form in comparison areas.”
The quality of the ACS measures of income was the subject of the paper  Posey and Welniak presented at the Census Bureau’s 1998 symposium on  the ACS. They compared income reported in the 1996 ACS and   1990 Decennial Census in an effort to evaluate the quality of the ACS  income data. One of the adjustments they made to compare the two series  was for the effect of inflation between 1990 and 1996. They noted that the  results of the comparisons indicated a potential problem that may relate to  the ACS inflation adjustment. (They described the calculation of the  adjustment, which is based on the CPI, but did not provide a rationale for  using the adjustment in the ongoing ACS data.)
Alexander and two BLS staff reported in 1999 on the potential for using  the ACS to improve labor force data from the CPS for state and smaller  geographic levels. They stressed that to develop procedures for making  these improvements, much research would be needed to evaluate  differences between the ACS and CPS.
The last research paper in this period was Alexander’s 2001 paper focusing  on the origins of the CM methodology and its developers. He discussed  the ACS in the context of the methodology, noting several important  differences related to the nature of the ACS. He included a review of the  Census Bureau’s testing and evaluation program, noting that the ACS test- site program had been expanded and that national sample supplementary  surveys had been added. He said that these test data would be compared  with the 2000 Census long-form data and that in 2001 and 2002, the  Supplementary Survey would be used as part of the transition to the ACS.  He also pointed to unresolved issues relating to the residence rule and the  multiyear averages, because they would provide users with multiple  estimates for geographic areas with populations larger than 20,000.
Between 2001 and 2003, the Census Bureau has issued three official  reports and one internal report on the status of the ACS testing and  development program. In Demonstrating Operational Feasibility,  published in July 2001, the Census Bureau gave a brief history of the ACS  development program, which by 2001 was focused on preparing for full  implementation in 2003 (although the Census Bureau later revised this to  2004) but on its operational feasibility, using data from C2SS. On the  basis of the Census Bureau’s analysis of the results of its tests of operation  feasibility, it reported the tests a success. However, it recognized that  more evaluation on measures of data quality was necessary, as well as on  differences between ACS and 2000 Census long- form data. The Census  Bureau announced that over the next 2 years it would issue reports  comparing data from the 2000 Census long form at the national, state, and  smaller geographic areas with data from the C2SS and the ACS  development program.
Demonstrating Survey Quality, published in May 2002, focused on  measures of C2SS survey quality, summarizing sampling and nonsampling  error levels in both C2SS and the 31 ACS test sites. The Census Bureau  used available, generally accepted measures of quality. On the basis of its  analysis of the results of these quality tests, the Census Bureau reported  the tests a success. This conclusion rested on test results that showed the  C2SS program capable of providing reliable long-form data.
As in the July 2001 report, the Census Bureau recognized that more  evaluation was necessary on measures of data quality as well as on  differences between ACS and 2000 Census long-form data and the detailed  estimates produced from C2SS. The Census Bureau repeated its  commitment that over the next year and a half, it would release other  reports to (1) analyze in detail basic demographic characteristics  (relationship, race, tenure) produced from the C2SS at the national and  state levels, including comparisons between C2SS and Census 2000;   (2) describe the data release plan and products for the ACS and the  usability and accessibility of estimates resulting from ACS methods; and  (3) give several detailed analyses of selected social, economic, and  housing characteristics (education, income, commuting patterns),  including comparisons between C2SS and Census 2000 at the national and  some subnational levels.
In June 2002, shortly after Demonstrating Survey Quality was released, a  team of Census Bureau specialists who had been working on the ACS for  several years prepared an internal report on testing. They presented a  revised program development plan and identified key questions to be  answered in testing the adequacy of the ACS in replacing the Decennial  Census long form. Their plan included the preparation of a series of nine  evaluation reports over 2 years. The reports that evaluated differences  between the 2000 Census short-form data (100 percent reported) and  corresponding C2SS items were included in Demonstrating Survey  Quality. Three reports to be completed between October 2002 and  January 2003 would evaluate differences between the detailed housing,  social, and economic characteristics between C2SS and the 2000 Census  long form, as described in Demonstrating Survey Quality. (Although this  schedule was later extended to the end of 2003, these three reports still  had not been released when we prepared our final draft of this report.)
Finally, the team’s plan included a report that would focus on the  comparisons of 3-year averages for the basic demographic, housing, social,  and economic characteristics from the C2SS and ACS test sites and  comparable estimates in the 2000 Census long form. The last report in the  plan would compare data for 2001 and 2002 with measures shown in  Demonstrating Operational Feasibility. The plan did not provide  completion dates for these reports.
American Community Survey Operations Plan, Release 1, published in  March 2003, identified research projects to be completed in preparation  for full implementation of the ACS. Two projects were on “weighting and  estimation,” which covered the methodology for using independent  population and housing controls, and on “program of integrated  estimates,” which covered the calculation of these controls from the  Census Bureau’s intercensal population estimates program. The  operations plan also reported on the schedule for completing several  comparison and evaluation projects with ACS and 2000 Census long-form  data discussed in Demonstrating Survey Quality. It discussed the need to  evaluate multiyear estimates from the supplementary surveys to  demonstrate the usability, reliability, and stability of ACS estimates over  time, and it stated that a report comparing 3-year ACS data with data   from the 2000 Census long form would be released in mid-2003.
The Census Bureau reported that the results of these research projects  would not be available in 2004. Instead, it said, it would use interim  procedures, taking “extensive long-term investigation and  experimentation” to develop final procedures. For the ACS weighting and  estimation project, the Census Bureau reported that it would be using an  interim adjustment to adjust the intercensal population and housing  characteristics estimates to the ACS residence concept. The Census  Bureau reported that ACS estimates of occupied housing units,  households, and householders should agree at all geographic levels.
For the program of integrated estimates project, the operations plan  discussed the need for more research to introduce improvements to the  estimates from ICPE. (The ACS estimates are weighted to a population  benchmark, either the most recent Decennial Census results or the most  recent ICPE estimates.) The Census Bureau reported that because the  accuracy of the intercensal estimates is important to overall ACS  accuracy, it is important to use ACS data wherever appropriate to improve  the intercensal estimates. The plan for the program on integrated  estimates will use information from the 2000 Census, more current ACS  distributions of population characteristics, and administrative records to  produce improved population and housing unit estimates for all areas,  including small areas. The plan also discussed improving housing  characteristics by incorporating ACS distributions of local area vacancy  rates and household characteristics into statistical models to better  estimate subcounty populations. No time schedule for completing the  research was provided.
Finally, the March 2003 American Community Survey Operations Plan,  Release 1 discussed a plan in the ACS to cover group quarters. Persons  living in group quarters live in places that the Census Bureau does not  classify as housing units—for example, nursing homes, prisons, college  dormitories, military barracks, institutions for juveniles, and emergency  and transitional shelters for the homeless. Such residences accounted for  roughly 2.8 percent of the population in 2000. Although data on group  quarters were collected at the test sites beginning with 1999, data were not  collected in C2SS or subsequent supplementary surveys. The operations  plan discussed the use of an updated Census 2000 Special Places file for  the sampling frame for the full ACS. In this case, the plan noted, training  field representatives on collecting data from this population is to begin in  October 2004, so that full data collection production can begin in January  2005.
Census Bureau staff made a presentation on comparison and evaluation  reports at the April 2003 meetings of the Census Advisory Committee. The  paper’s author reported that work was under way on the comparison  reports noted in the March 2003 operations plan, and she described the  methodology to be used to evaluate differences between the 2000 long  form and C2SS. She also reported that the results of the comparisons  would be used to identify how the ACS should be improved but that  additional research would be needed to address consistency over time  between the 2000 Census and the full ACS. She stressed the importance of  evaluating consistency “in educating users on the transition from the  decennial census sample estimates to the ACS estimates.”
With regard to the comparison report of selected demographic, housing,  social, and economic characteristics of 3-year estimates from the ACS test  sites to the 2000 Census, the Census Bureau let four contracts with local  experts to conduct comparisons of 3-year averages of ACS data for   1999–2001 for selected test sites with selected 2000 Census long-form data  as well as 2000 Census population and housing unit characteristics. The  comparisons, prepared at the county and census tract levels, would be  made for measures of data quality (self-response rates, sample unit  nonresponse rates, item nonresponse rates, and sample completeness  ratios), as well as for data levels (counts, percentages, means, and  medians) for demographic, social, economic, and housing characteristics.
In summer 2003, Census Bureau staff presented a number of research  papers on the ACS at the annual Joint Statistical Meetings. Papers  evaluated differences between long-form and C2SS data items, such as  persons with disabilities, educational attainments, and income. Most of the  papers that provided comparisons with long-form data indicated whether  differences were statistically significant for every comparison.  Comparisons were presented at a variety of geographic levels (national,  state, and test site levels). Some papers cited operational differences as  possible explanatory factors, but information was not presented using a  standard set of factors.
The Census Bureau published ACS-2010 Census Consistency Review  Plan, an internal document, at the beginning of October 2003. Its purpose  was to identify methods for major operations used in the ACS and for the  2010 Census that were likely to lead to inconsistent results and to  recommend ways to address these inconsistencies. Papers prepared on  these operations were to discuss how an issue might result in  inconsistencies between the ACS and 2010 Census results and to set forth  options for dealing with consistency issues, including a research process.  The plan identified residence rules and group quarters as two topics. It did  not discuss completing the work in time to incorporate it into the full ACS  in the next several years.
Also in October 2003, the Census Bureau made two public announcements  related to the ACS development plan at the Census Advisory Committee  meetings. Two papers related directly to projects described in American  Community Survey Operations Plan, Release 1. In “Enhancing the  Intercensal Population Estimates Program with ACS Data: Summary of  Research Projects,” Weidman and Wetrogan reported on research to  improve the intercensal estimates by using ACS data for two “high  priority” areas—international migration and internal migration. This work  was being conducted within the Program of Integrated Estimates. The  second paper described options for determining population control  weights for ACS implementation in fall 2004 but did not indicate that  research was under way to determine the effect of the options.
Another source of information related to ACS development was the  various reports prepared as part of the Census 2000 Testing,  Experimentation, and Evaluation Program. Schneider’s January   2004 report compared employment, income, and poverty estimates from  the 2000 Census long form and the CPS. From this comparison, the  author concluded that this work should be continued in an effort to use  the results of the comparisons to improve consistency between data  collected in the CPS and data in the ACS; the ACS uses the same questions  as the 2000 long form. The author also identified for additional research  long-form questions that performed badly, based on a reinterview survey.
From May to July 2004, the Census Bureau released seven ACS evaluation  reports. Four reports compared data from the 2000 Census long form and  the C2SS at the national level. Two reports compared these long-form data  with 1999-2001 data from the ACS test sites for selected counties and one  of these compared these data at the tract level. The other report reviewed  operational data from the 2001 and 2002 supplementary surveys. In most  of the reports comparing long-form and ACS data, the Census Bureau  identified additional work that was needed to improve the quality of the  ACS estimates or to help explain differences between the two sets of data  for 2000. As noted earlier, these comparisons were limited to the national  level. (The seven new reports are listed in the bibliography.)
section_title: Appendix V: Current Status of Unresolved Issues
subsections: section_title: Independent Controls for Population and Housing Characteristics
paragraphs: According to the Census Bureau’s plans, the calculation of independent  controls for population characteristics (age, sex, race, and ethnicity) and  housing characteristics for the full ACS will require a significantly  different methodology from that used for the ACS supplementary surveys.  Controls will be needed at the same level of geographic area detail as  those that were used for the 2000 Census long form and will need to reflect  the new concepts of residence and reference period underlying the ACS.
For the annual ACS supplementary surveys, these characteristics were  used from ICPE as the independent controls. ICPE uses Decennial Census  short-form data as benchmarks and administrative record data to  interpolate between and extrapolate from the census benchmarks. The  program provides “official” annual estimates of population and housing  characteristics at the county level, and for some subcounty levels, as of  July 1 of each year, using the usual residence concept for seasonal  residents. The program also provides annual estimates of total population  and housing units for all areas of general-purpose government, such as  cities, villages, towns, and townships. Table 3 shows information on the  calculation of the independent controls used for the 2000 Census long  form, the ACS supplementary series, and the fully implemented ACS  through 2012.
Using ICPE for the ACS supplementary surveys, the Census Bureau  prepared controls for counties, or combinations of counties. As shown in  table 3, for the residence concept, controls from the 2000 Census and  ICPE, which were based on the usual residence concept, were used. The  reference period for the ACS test program for all years except 2000 was  for July 1; for 2000, it was for April 1. (Controls for the 2000 Census long  form also were for April 1.)
For the full ACS, the Census Bureau will use controls based on the current  residence concept. According to the Census Bureau, the current residence  concept recognizes that the place of residence does not have to be the  same throughout a year, so that the current residence concept allows the  ACS data to more closely reflect the actual characteristics of each area.  The Census Bureau will use the current residence concept because the  ACS is conducted every month and produces annual averages rather than  point-in-time estimates, as the Decennial Census does. Also, because the  ACS data are collected monthly, it will be necessary to use independent  controls that define the reference period as the average for the year using  a July 1 reference period.
To produce ACS estimates for the full sample, the Census Bureau will  need new methodologies for calculating independent controls. For the  first annual estimates, for 2005, a methodology will be needed to provide  ACS-defined controls for all places with population of 65,000 or more,  including those for which intercensal population estimates are not  available. For the 2005–07 estimates, which will be used to calculate the  first multiyear averages, a methodology for controls for geographic areas  with populations between 20,000 and 65,000 will be needed. For the   2008–12 estimates, a methodology for controls down to the geographic  levels used for the 2000 Census long form will be needed. Finally, when  the population and housing characteristics data from the 2010 Census  short form become available and are incorporated into the ICPE estimates,  another new methodology will be needed to revise the ACS controls for  2010.
The Census Bureau also has reported that it is not planning to revise  earlier years’ ACS data for consistency with revised 2010 estimates unless  the inconsistencies between the 2010 ICPE and 2010 Census  characteristics were significant. Table 4 shows the differences between  population estimates at the county level for 2000 using ICPE based on the  1990 Census and the corresponding data from the 2000 Census. In   2000, the population estimates for almost 20 percent of the counties  differed by more than 5 percent. For counties whose population was  smaller than 20,000, almost 25 percent had similar differences.
Census Bureau staff had long recognized the need for new methodologies  to develop independent controls for the ACS. For example, a 1995 paper  by Love, Dalzell, and Alexander, discussing issues related to evaluating the  1996 test site results, expressed concern about independent controls and  residence rules, as well as the need for consultation with users. In   1998, the Census Bureau sponsored a conference on the quality of ACS  data for rural data users. In the final report on this conference, the Westat  authors concluded that the Census Bureau needed to continue and expand  its contacts with stakeholders and to conduct additional research on  several issues, including independent controls. Alexander and Wetrogan  also discussed this issue at the 2000 Joint Statistical Meetings when they  reviewed possible methods for using ICPE estimates. They also noted the  need to consult with users on how to present information on the  differences in ACS controls and ICPE in ACS publications.
Census Bureau staff also recognized that the new ACS would create  differences between (1) ACS population and housing characteristics data  and the corresponding “official” data from the Decennial Census and   (2) ACS population and housing characteristics data and the “official”  ICPE population estimates, which are benchmarked to Decennial Census  data. They also recognized that the creation of new controls for the ACS  would result in inconsistencies between ACS data and data from federal  household surveys, such as the CPS, whose population and housing  characteristics are also based on the Decennial Census and ICPE  estimates. Such differences might hinder the use of ACS data to expand  and improve small geographic area estimates based on the other surveys.  (CPS provides official national estimates of labor force information, such  as the unemployment rate and income estimates used to calculate the  number of persons in poverty.)
In March 2003, the Census Bureau announced that it did not have a final  methodology and that such methodologies would not be established for  several years. In March 2003 in American Community Survey Operations  Plan, Release 1, the Census Bureau identified research projects to be  completed in preparation for full implementation of the ACS. One of these  projects, “weighting and estimation,” covered the methodology for  calculating the independent controls for the ACS; a second, “program of  integrated estimates,” covers the calculation of these controls from the  ICPE. This plan also reported that the results of these research projects  would not be available in 2004 to begin implementing them with the start  of the full ACS. Instead, the Census Bureau said it would use interim  procedures and that it would take “extensive long-term investigation and  experimentation” to develop final procedures.
For the weighting and estimation project, the Census Bureau reported that  it would be using an interim adjustment to adjust the intercensal  population and housing characteristics estimates to the ACS residence and  reference period concepts. This project would include research to  examine the need to achieve agreement between the estimates of occupied  housing units, households, and householders at all geographic levels. The  Census Bureau reported that work on the project to revise and simplify  the weighting methodology began in early 2003, that preliminary papers  documenting the revisions might be available by summer 2004, and that  research would continue for several years.
For the program of integrated estimates project, the operations plan  discussed the need for more research to introduce improvements to the  ICPE estimates using information from the 2000 Census, more current  ACS distributions of population characteristics, and administrative  records to produce improved population and housing unit estimates for all  areas, including small geographic areas. The plan also discussed improving  the housing characteristics. ACS distributions of local area vacancy rates  and household characteristics can be incorporated into statistical models  that use distributions of housing unit characteristics to better estimate  subcounty populations. No time schedule was provided for completing the  research.
In October 2003, Census Bureau staff presented a paper at the Census  Advisory Committee meetings that described the options being considered  to convert the ICPE estimate to the current residence concept. The paper  described options for determining controls for ACS implementation in fall  2004 but did not indicate that research was under way to determine the  options’ effects. A second paper at the same meetings reported on  research to improve the intercensal estimates by using ACS data for two  “high priority” areas—international migration and internal migration. This  work was being conducted as part of the Program of Integrated  Estimates. “Given the newly benchmarked intercensal estimates, the following question arises  regarding the use of the 2010 Census data in the ACS: Should ACS estimates continue to be  controlled to 2010 Census data at the county or county group level and differences between  the ACS and census population counts and characteristics allocated proportionately to the  tract or block group levels? Or should ACS estimates be controlled to 2010 Census data at  the tract and block group level, as would have been the case with a long form?”
All the experts agreed that the ACS should be controlled to the decennial  census, but several noted that they had not thought about the issue and  had not heard anything from the Census Bureau on the issue. (The experts  are listed in app. I.)
section_title: Operational Issues
paragraphs: The Census Bureau has identified operational issues with the ACS test  programs, primarily from information from evaluation studies on the   2000 Decennial Census and Census Bureau staff research papers on  comparisons between data collected in the ACS 2000 supplementary  survey and the 2000 Decennial Census long form. These issues include  problems with questionnaire design, nonresponse followup, and data  capture, as well as coverage of persons living in group quarters.
In January 2004, the Census Bureau released the results of a key  evaluation study of 2000 Decennial Census long-form data, using a  reinterview survey. The study identified problems with long-form  questions, which are the same as those used for the ACS, and proposed  several research efforts based on a statistical evaluation of the quality of  the responses to each question. For questions identified as having  significant quality problems, the study recommended research on the  design of the form and placement of the questions and suggested using  cognitive experts in testing revised questions. The study also  recommended that the Census Bureau and BLS work on the ACS  employment and unemployment questions to ensure that they would  complement the BLS local area unemployment statistics program.
The Census Bureau also conducted a study to evaluate the design of the  ACS questions that are needed to implement the residence concept and  reference period for the ACS. The study suggested that additional testing  was needed for the questions about multiple residences (currently, the last  set of questions in the housing section). It noted “that asking these  questions on a person basis may produce different and probably better  data than asking them on a household basis.” The study was limited in  scope and did not assess how accurately ACS respondents assign persons  associated with the household to a current residence.
In the ACS, the Bureau uses “In the past 12 months . . .” whereas the  Census Bureau used “In 1999 . . .” for the long form. Because the reference  date is not fixed, it is important for a respondent to supply the date that  the ACS questionnaire filled out. Otherwise, it cannot be determined  whether there is an inconsistency in an ACS questionnaire received in late  April 2004 that lists a resident aged 10 with a birthdate of April 15, 1993.
Census Bureau staff also discussed operational issues in research papers,  based on evaluations of comparisons between 2000 Decennial Census  long-form and ACS 2000 supplementary survey data for selected items  presented at the 2003 Joint Statistical Meetings. A paper on income data  identified the new question on the reference period as a potential source  of problems, even though an additional instruction had been added to the  ACS questionnaire in 1999. The authors expressed concern that some  ACS respondents may misinterpret the question on “income in the past   12 months” as a request for monthly income instead of income during the  previous year. The paper also included recommendations for additional  research on the effect of the data capture methods. For the 2000 long  form, all data items were entered with an automated optical character  recognition procedure; data from the ACS will be manually keyed.
Another paper, presented at the same 2003 meetings, that evaluated  differences in the data on disabled persons found large and significant  differences at the national level and also recommended that new questions  be tested. Additional areas were identified for further research, based on  evaluations of questions such as educational enrollment, ancestry, and  grandparents caring for grandchildren. These areas included specific  facets of the mailout-mailback system and nonresponse followup. For  example, nonresponse follow-up for the 2000 long form was conducted for  all nonrespondents, but for the ACS test program and for the full ACS,  nonresponse follow-up will be conducted for a sample of one-third on all  nonrespondents.
The Census Bureau also has discussed issues with the expansion of ACS  coverage to include persons living in group quarters—for example, nursing  homes, prisons, college dormitories, military barracks, institutions for  juveniles, homeless shelters. In October 2002, it informed its advisory  committee members of the formation of a special planning team to  address issues on the definition of group quarters and duplication in the  address file. From the minutes of this meeting, it appears that this team  will focus on group quarters in the context of the 2010 Census short form.  In the ACS March 2003 operations plan, the Census Bureau reported on a  new project to cover group quarters in the full ACS. The Census Bureau  reported that the special project was needed because of the special  challenges of developing an updated address list; in the past, such a list  had been updated only once a decade. According to the Census Bureau,  tests on the new list were to be completed in time for use in the full ACS in  January 2005. In addition, an internal planning document issued in  October 2003 identified group quarters (and residence rules) as special  problems and instructed staff to provide recommendations on the  collection of data on them in January 2004. Usually, the Census Bureau  tests new questions. According to recent Census Bureau decisions, those  tests would have to be completed so that new questions could be  incorporated into the 2008 ACS questionnaire.
section_title: Valuation and Presentation of Dollar-Denominated Data Items
paragraphs: The Census Bureau has adjusted all dollar-denominated items from the  ACS testing programs, such as incomes, housing values, rents, and  housing-related expenditures, for inflation. For example, ACS data for  2001 and 2002 released in September 2003 for median household income  are expressed in 2002 dollars. This practice means that when each added  year of ACS data is released, all dollar-denominated items for prior years  will be revised. The Census Bureau makes a similar adjustment for the  annual income data collected in the CPS. Unlike the ACS, the Census  Bureau releases annual CPS data without the adjustment. In addition, the  annual values collected in the ACS were adjusted to the calendar year. It  will be using the CPI for the annual and monthly adjustments for all  geographic areas.
A report prepared for HUD found problems with the adjustment, including  (1) the lack of a “trending” adjustment in the calculation of annual  averages, (2) the use of the adjustment for multiyear averages, (3) the  adjustment for cost of living for data items other than income, and (4) the  lack of the unadjusted annual data that would enable HUD to use  alternative methodologies. In addition, research by Census Bureau staff  questioned the adjustment for incomes when they found that it was a  probable source of difference between income data from the  supplementary survey and corresponding data from the CPS and the   2000 Census long form. “Making an inflation adjustment is not the same as trending. The cost of living adjustment  assumes that the purchasing power measured at any point in the data collection period  remains constant throughout the period. For example, assume that the cost of living rises  by 3 percent a year. If a household reports an annual income of $50,000 in January, a cost  of living adjustment to the end of the year would increase this income to $51,500, the  amount needed in December to equal the purchasing power of $50,000 in January. A  trending adjustment makes no assumption about purchasing power. It attempts to track  movements in dollar income. Assume that dollar income is growing at 5 percent a year.
Then a trending adjustment to the end of the year would increase the $50,000 reported in  January to $52,500 in December.” “The Census Bureau plans to report income in constant dollars. Income information  collected in the various months will be adjusted for inflation so that all collected income  will be expressed in dollars with the same purchasing power, presumably the purchasing  power of dollars in December of the survey year. For moving average tabulations, all  income information will be adjusted for changes in purchasing power over the period used  to calculate the moving average. In other words, income reported by a respondent in the  first month of a five-year moving average will be adjusted for almost five years of  inflation.” “The standard Census Bureau tables for areas over 65,000 will tabulate the rents reported  by respondents over the twelve months during which data were collected. A unit reporting  a contract rent of $800 in January might actually be paying $850 in December. The standard  table would record this unit as having a rent of $800. The standard Census Bureau tables  for areas under 20,000 will tabulate rents reported by respondents over a sixty-month  period. A unit reporting a contract rent of $800 in the January of the first year might  actually be paying $1,070 in December of the fifth year. The standard table would record  this unit as having a rent of $800.”
Such changes would not be captured with an adjustment based on the all- items CPI. “The ACS will generate income distributions comparable to those from the decennial  census, but the distributions will have a feature that will complicate the use of income data  from the ACS in APP  measures. Whereas the decennial  long form measures money income, the ACS reports average purchasing power.”
The report thus recommended that HUD use the unadjusted data—data  that the Census Bureau had not planned to publish—in order to make the  changes needed for HUD. “If no CPI adjustment had been made to the dollars reported on either Census 2000 or  C2SS/ACS, the difference between medians at the U.S. level would have been smaller than  the 4.6 percent shown in Table 3 . Instead, the difference would have been 2.5  percent. Since adjustment clearly played a role in determining the size of the difference  between Census 2000 and C2SS/ACS estimates, it would be worthwhile to examine the  costs and benefits of adjusting C2SS/ACS incomes as well as the choice of factors used to  adjust them.”
The authors summarized their findings by concluding that “it is clear that  we are just at the beginning stages of understanding why Census 2000 and  C2SS income figures differ.” They noted that the income comparisons are  most critical because these Census Bureau data are used in the calculation  of the number of people in poverty.
In a December 2003 research paper, Census Bureau staff examined  concerns about the absence from the official poverty measures of  adjustment for geographic differences in cost of living. Like the ACS, for  which the Census Bureau is assuming that the cost of living is the same  throughout all geographic areas, the poverty measures are based on the  same assumption. The authors concluded that the use of a poverty  measure that takes into account geographic differences in housing costs,  would significantly change the poverty measures in many states.
section_title: Evaluations of ACS, Long-Form, and CPS Data Comparisons
paragraphs: One of the Census Bureau’s major justifications for the ACS test programs  has been its comparing data collected in these programs, and  corresponding data from the 2000 Decennial Census short and long forms,  to identify operational problems. Another major justification for the ACS  test programs has been the use of these comparisons, and comparisons  with corresponding data from the CPS, to inform users in making the  transition from the 2000 long form to the ACS. “to make a transition from the Census 2000 long form to collecting long-form data  throughout the decade, we will begin ACS data collection in 1,203 counties. This data  collection will allow for comparison of estimates from Census 2000 with estimates from the  ACS for all states, large cities, and population subgroups, and will help data users and the  Census Bureau understand the differences between estimates from the ACS and the  Census 2000 long form.” “These data will also contribute to a comparison with data from Census 2000 that is  necessary because there are differences in methods and definitions between the census  and the ACS. Moreover, decision makers will want to compare an area’s data to those from  Census 2000. Comparisons using data from the operational test and from the 31 sites are  essential to determine how much measured change between Census 2000 and future years  of the ACS is real and how much is due to operational differences between the ACS and the  census.”
When the Census Bureau began in 2001 to report on full implementation of  the ACS, its first report focused on the operational feasibility of  conducting the ACS. Its second report in 2002 focused on differences in  operational characteristics of the ACS and the census long form, such as  response rates and the extent of imputations.
The 2002 report stated that three reports evaluating differences between  the ACS and census long form would be published at the end of 2003. The  Census Bureau repeated this schedule in March 2003 when it released  another official report on ACS plans. In September, we were told by one  of the ACS experts that consultants had been hired to conduct evaluations  for 4 of the 31 test sites. The reports on comparisons with long-form items  and for the test sites were published in May, June, and July 2004.
The results of these comparisons are similar to comparisons and  evaluations of long-form data items previously prepared by Census Bureau  staff, BLS, and GAO. “These comparisons showed large national differences for key items that did not appear to  be accounted for by coverage differences between the two surveys. For example, at the  national level, the largest differences were for these items: (1) for the number of housing  units lacking complete plumbing facilities, with the long-form estimate 27 percent higher  than the estimate from the supplementary survey, and (2) for the number of unpaid family  workers, with the long-form estimate 59 percent lower. . . . We also found a great degree of  variation in the state differences between the long form and the supplementary survey.” “We found that at the national and state levels, there were small differences for the  unemployment rate and for the poverty rate for all individuals. In contrast, comparisons of  these rates for the CPS with these two surveys showed larger differences. The national  unemployment rate, according to the CPS, was 4.0 percent, compared with 5.8 percent for  the long form and 5.4 percent for the supplementary survey. The national rate for  individuals in poverty for the CPS was 11.3 percent, compared with 12.4 percent for the  long form and 12.5 percent for the supplementary survey.”
Given these results, we recommended that the Census Bureau expand the  scope of evaluation studies to develop supplementary survey estimates for  states and large places consistent with the 2000 long form and that it  include in its evaluations comparisons of year-to-year changes for   2001 and 2002, using data from the supplementary surveys and the CPS at  the national and state levels for key economic and housing items. “Relative to the CPS, the ACS consistently generates lower estimates of the labor force and  employment but higher estimates of unemployment. These patterns are present in each of  the years 2000, 2001, and 2002. They are repeated in nearly all state-level data as well.”
He made a series of recommendations for additional research, some  requiring additional information from the Census Bureau. “provided a summary of the major differences between the two income data sources, in  terms of data collection, capture, and processing, and provided very preliminary  assessments of the possible role these differences may have played.”
The authors reported that additional work was needed to understand the  differences and offered recommendations for further research.
Another paper presented at the same meetings examined differences  between the national estimates for people aged 5 or older with a  disability—48.9 million was the 2000 Census long-form estimate,   39.7 million the C2SS estimate. The author did not determine which  estimate was more reliable but did find that the wording of some questions  might explain the overall difference. In addition, the author reported that  more work, such as additional analysis of currently available data and  testing of new questions, was needed to clearly identify the reasons for the  difference. The differences in disability data were also the subject of a  National Council on Disability position paper, which recommended  changes to the questions on disability. “The Census Bureau should carry out more research to understand the differences between  and relative quality of ACS estimates and long-form estimates, with particular attention to  measurement error and error from nonresponse and imputation. The Census Bureau must  work on ways to effectively communicate and articulate those findings to interested  stakeholders, particularly potential end users of the data.” “The Census Bureau should make ACS data available (protecting confidentiality) to  analysts in the 31 ACS test sites to facilitate the comparison of ACS and census long-form  estimates as a means of assessing the quality of ACS data as a replacement for census long- form data. Again, with appropriate safeguards, the Census Bureau should release ACS data  to the broader research community for evaluation purposes.”
section_title: Information on Key Properties of Multiyear Averages
paragraphs: One of the major differences between the ACS and the long form it will  replace is that the ACS will provide data for geographic areas with  populations smaller than 65,000 in terms of multiyear averages. Because of  the statistical properties of these averages and users’ unfamiliarity with  them, we and many other stakeholders have identified these averages as a  major challenge for users, including federal agencies. The Census Bureau  has recognized the need for such guidance on the averages but has not  made public plans for the topics to be discussed or when the guidance will  be published. “In discussing this issue, a number of the participants thought that averages were  particularly problematic for those areas in which change is irregular. For example, the  question was raised as to the meaning of ‘average poverty’ over a 5-year period in which  poverty rose and fell from one year to the next and, thus, the average would have no  obvious meaning.”
The report made similar comments with regard to such characteristics as  unemployment and income. Although the conference participants had  generally agreed with these concerns, the report pointed out that annually  updating the 5-year averages “will provide some insight into trends,  although turning points will be difficult to discern precisely, as will short- term trends.” “Although a 5-year moving average will generally provide reasonably reliable cross-section  statistics for all areas, including very small communities, some care will have to be  exercised in choosing time periods for which changes in population or their characteristics  are measured. With 5-year averages, four-fifths of the data in a pair of neighboring years  will be identical. The change being measured will then be one-fifth of the difference  between the most recent year and the first year of the earlier time period. The sampling  errors of the differences will thus be based on annual sample sizes, not 5-year averages,  and will generally be too large to make useful inferences for small areas. The two 5-year  averages that are being compared should generally be discrete and non-overlapping  periods, e.g., 2003–2007 and 2008–2012. These comparisons will have about the same  reliability as changes between two censuses using data collected in the Census long  form.” “encourage analysts to use the same length of cumulation when comparing areas of  different sizes . . . . For example, we would use one year for comparing states, but would  recommend 5 years for all the counties in a table comparing large and small counties.”
Alexander noted that this approach differed from that of Kish, the  developer of the concept of a “rolling sample,” who would “let us use  tables of counties with one-year estimates for large counties, 3-year  averages for medium-sized ones, and 5-year averages for small ones.” He  concluded this section of the paper by saying, “It will be interesting to see  what practices data users will adopt in this regard.” “If there is little change in the population over the time covered by the average, the  interpretation is about the same as that of a point-in-time estimate with the advantage that  the ACS estimate is more current than the historical decennial census long-form  estimate.”
The paper provided examples with “naive” assumptions about how users  extrapolate between censuses to show that multiyear averages “work.” By  implication, under other conditions, users will need guidance on when  multiyear averages can be used. The paper also did not discuss the  interpretation of changes in the multiyear averages, as in the 1999 Westat  conference report or multiple estimates, which Alexander had discussed  in his paper for the 2001 Statistics Canada conference.
In September 2002, two reports focused on issues related to the statistical  properties of multiyear averages. We published a report on several aspects  of the ACS, including the selection of questions and the feasibility of  conducting the ACS as a voluntary survey, and HUD released a report  prepared for its staff on the use of the ACS for HUD programs. “Analyze and report on differences between year-to-year changes for 2001 and 2002, using  the data—from ACS special supplements and the CPS at the national and state levels—for  key economic and housing characteristics, such as the unemployment and poverty rates, to  determine the stability of the annual ACS data.”
We also discussed the need for additional information on the  characteristics of the multiyear averages to help federal agencies make the  transition to the ACS. We specifically noted the need for information on  the selection of ACS data for geographic areas with populations larger  than 20,000 for which there will be multiple estimates. On this issue, we  stated that, “In addition, we found that the ACS development program did  not cover information about different ways to integrate the annual data for  states and large counties and the 3- and 5-year averages for smaller  counties.”
For example, federal agencies that need state data can choose to use the  annual data, multiyear averages of the annual data, or 3-year or 5-year ACS  averages. Federal agencies that also need county data can choose to use  the most recent annual data for large counties and adjust the averages of  the smaller counties to agree with annual data. Alternatively, they can  choose to use various combinations of multiyear averages. As many  federal agencies, as well as state and local governments, will be using the  ACS data for allocating funds, Census Bureau guidance would reduce the  inconsistent use of the multiple estimates.
HUD is a major user of Decennial Census long-form data for various  program applications. Its contract with ORC Macro to review how the ACS  will affect HUD programs that previously relied on the Decennial Census  long form for geographic area data resulted in a report that made two  points about the multiyear averages, in addition to raising the previously  discussed issues on the inflation adjustment to income. One of these  issues related to interpretations of changes in the multiyear averages and  their stability; the other related to the availability of multiple estimates for  the same area.
The ORC Macro report noted that year-to-year stability is important and  needs to be addressed. It warned that the “differences in the precision of  estimates or year-to-year changes in estimates can create problems for  HUD applications.” The report used eligibility and level of benefits as an  example of what could vary because of the effect of sampling variability  on these changes. ORC Macro also stated: “The ACS will report data using  different reporting periods for different sized areas. Inconsistent or  multiple reporting periods can create problems for HUD applications.”  ACS data for many geographic areas will be available in terms of annual  estimates and 3- and 5-year averages, and the annual and 3-year averages  (for larger areas) will be available before estimates for smaller areas. As a  result, HUD will have to choose from multiple measures for some  geographic areas. The study noted that HUD might decide to (1) continue  to use 2000 long-form data until 2008, when the first 5-year average data  will be available for all levels of geography, or (2) use the most recently  available data in all cases.
ORC Macro’s report also expressed concern about the amount of annual  ACS data that the Census Bureau will release for areas with populations  smaller than 65,000, whose accuracy the Census Bureau has found does  not meet publication standards. According to the study, the Census  Bureau informed HUD that beginning in 2008, it would provide researchers  and planners a “research file” containing annual ACS data for areas of all  sizes, including census tracts. ORC Macro recommended that if the Census  Bureau does release these data, HUD consider using these “unofficial”  research file results in some of its applications. The study noted, however,  that if HUD decided to use these unofficial data but other agencies decided  not to use them, there would be no standardization across government  programs in funding allocation where the same ACS items were used. “is a smoothed estimate; by averaging a particular time period’s data observation with  those within a particular time window, the resulting estimate is meant to follow the general  trend of the series but not be as extreme as any of the individual points. The ramifications  of this basic concept emerge when moving average estimates are entered into sensitive  allocation formulas or compared against strict eligibility cutoffs. A smoothed estimate may  mask or smooth over an individual year drop in level of need, thus keeping the locality  eligible for benefits; conversely, it may also mask individual-year spikes in activity and thus  disqualify an area from benefits. It is clear that the use of smoothed estimates is neither  uniformly advantageous nor disadvantageous to a locality; what is not clear is how often  major discrepancies may occur in practice.” “It is incorrect to use annual estimates based on moving averages over several years when  assessing change since some of the data are from overlapping time periods and hence  identical. At the least, the results will yield incorrect estimates of the variance of the  estimates of change. Therefore, users should be cautioned about this aspect of the use of  moving averages.”
section_title: External Consultation
paragraphs: During the past decade’s development of the ACS, the Census Bureau has  had many opportunities to consult with and take account of input from  stakeholders and users in making key decision on the programs. It has   (1) sponsored NAS panels, (2) held user conferences, (3) hired consultants  to organize two conferences, (4) met regularly with its advisory  committees and other user groups, and (5) encouraged its staff to present  reports at ASA meetings and meetings of similar professional  organizations. In the past several years, we and other federal agencies  have reported on the ACS and provided recommendations to the Census  Bureau. It established the ACS Federal Agency Information Program in  2003, responding to a recommendation we had made. It also announced  last year that it was looking into establishing a partnership with the  Congress and its oversight entities. “Eight years later, faced with the task of offering advice on making the vision of continuous  measurement a reality in the 2010 census, the similarity between the arguments then and  now is uncanny. Similar, too, are the points of concern; the current panel is hard-pressed to  improve upon the basic summary of concerns outlined by our predecessors. We are,  however, much more sanguine that a compelling case can be made for the ACS and that it  is a viable long-form replacement in the 2010 census.”
The Census Bureau has neither responded to the panel’s first interim  report in 2000 nor indicated that the recommendations were being  adopted. The Census Bureau also has not responded to recommendations  and issues raised by HUD and BLS. For example, it has not responded to  HUD’s recommendations on the ACS adjustments to dollar-denominated  items or to BLS’s recommendations on the ACS labor force questions. (On  the issue of dollar-denominated items, we found no indication that the  Census Bureau had ever consulted outside experts about either the  methodology or the implementation.)
Census Bureau summaries of discussion about the ACS at its Advisory  Committee meetings from October 2000 to April 2003 also indicate a lack  of responsiveness. During this period, committee members raised  concerns about the ACS. In particular, they made recommendations about  many of the issues we have discussed in this report, including the  evaluations of ACS and long-form comparisons, the new residence rules,  independent controls, ICPE, group quarters, and Spanish-language  questionnaires. At the April 2003 meeting, ASA committee members also  requested a change in the structure of the Advisory Committee meetings,  asking the Census Bureau to spend less time on update sessions and more  time on sessions devoted to gathering more detailed input, commentary,  and recommendations on topics the Census Bureau needs and wants  advice on. Although the Census Bureau has addressed issues related to  ICPE and Spanish-language questionnaires, the meeting summaries do not  report that it followed recommendations in other areas.
section_title: Appendix VI: Comments from the Department of Commerce
section_title: Appendix VII: GAO Contacts and Staff Acknowledgments
subsections: section_title: GAO Contacts
section_title: Staff Acknowledgments
paragraphs: Additional staff who made major contributions to this report were  Heather Von Behren, Penny Pickett, Mitchell Karpman, Michael Volpe,  Andrea Levine, Patricia Dalton, and Robert Goldenkoff.
section_title: Bibliography
paragraphs: The first section in this bibliography lists documents on the history of the  long form and mid-decade census. The remaining works are divided  between numerous types of Census Bureau reports and papers,  Association of Public Data Users papers, congressional hearings and  testimony, and other reports and papers. Recent reports from the National  Academy of Sciences are discussed in appendix II. Related GAO Products  are listed in a separate section at the end of this report.
subsections: section_title: History of the Long Form and Mid-Decade Census
paragraphs: Alexander, Charles H. “Still Rolling: Leslie Kish’s ‘Rolling Samples’ and the  American Community Survey.” In Proceedings of Statistics Canada  Symposium 2001: October 16–19. Ottawa: Statistics Canada, 2002.
Anderson, Margo J., ed. Encyclopedia of the U.S. Census. Washington,  D.C.: CQ Press, 2000.
House of Representatives, Committee on Post Office and Civil Service,  Subcommittee on Census and Population. Review of Major Alternatives  for the Census in the Year 2000. Serial 102-25. Washington, D.C.: August  1, 1991.
House of Representatives, Committee on Post Office and Civil Service.  Census Confidentiality/Mid-Decade Sample Survey Bill. Report 93-246.  Washington, D.C.: June 4, 1973.
House of Representatives, Committee on Post Office and Civil Service.  Mid-Decade Censuses of Population, Unemployment, and Housing.  Report 780. Washington, D.C.: August 12, 1965.
Salvo, Joseph, and Arun Peter Lobo. The American Community Survey:  Quality of Response by Mode of Data Collection in the Bronx Test Site.  Presented at 2002 Joint Statistical Meetings, New York City, August 14,  2002.
section_title: Census Bureau ACS Reports
paragraphs: American Community Survey Operations Plan, Release 1. Washington,  D.C.: March 2003.
Meeting 21st Century Demographic Data Needs: Implementing the  American Community Survey. Report 1. Demonstrating Operational  Feasibility. Washington, D.C.: July 2001.
Meeting 21st Century Demographic Data Needs: Implementing the  American Community Survey. Report 2. Demonstrating Survey Quality.  Washington, D.C.: May 2002.
Meeting 21st Century Demographic Data Needs: Implementing the  American Community Survey. Report 3. Testing the Use of Voluntary  Methods. Washington, D.C.: December 2003.
Meeting 21st Century Demographic Data Needs: Implementing the  American Community Survey. Report 4. Comparing General  Demographic and Housing Characteristics With Census 2000. Washington,  D.C.: May 2004.
Meeting 21st Century Demographic Data Needs: Implementing the  American Community Survey. Report 5. Comparing Economic  Characteristics With Census 2000. Washington, D.C.: May 2004.
Meeting 21st Century Demographic Data Needs: Implementing the  American Community Survey. Report 6. The 2001-2002 Operational  Feasibility Report of the American Community Survey. Washington, D.C.:  May 2004.
Meeting 21st Century Demographic Data Needs: Implementing the  American Community Survey. Report 7. Comparing Quality Measures:  The American Community Survey's Three-Year Averages and Census  2000's Long Form Sample Estimates. Washington, D.C.: June 2004.
Meeting 21st Century Demographic Data Needs: Implementing the  American Community Survey. Report 8. Comparison of the American  Community Survey Three-Year Averages and the Census Sample for a  Sample of Counties and Tracts. Washington, D.C.: June 2004.
Meeting 21st Century Demographic Data Needs: Implementing the  American Community Survey. Report 9. Comparing Social  Characteristics With Census 2000. Washington, D.C.: June 2004.
Meeting 21st Century Demographic Data Needs: Implementing the  American Community Survey. Report 10. Comparing Housing  Characteristics With Census 2000. Washington, D.C.: July 2004.
section_title: Census Bureau Advisory Committee Presentations
paragraphs: The presentations in this section were made by the Census Bureau’s  Decennial Census Advisory Committee, Professional Association Advisory  committees, and Race and Ethnic Advisory Committees.
The ACS: Data Products to Meet User Needs. Race and Ethnic Advisory  Committee meeting, Washington, D.C., March 14, 2001.
Alexander, Charles, Alfredo Navarro, and Deborah Griffin. Update on ACS  Evaluations. Decennial Census Advisory Committee meeting, Washington,  D.C., November 5, 2001.
Gordon, Nancy. The American Community Survey. Joint Meeting of the  Census Bureau Advisory Committees, Washington, D.C., July 28, 2000.
Gordon, Nancy. The American Community Survey. Decennial Census  Advisory Committee meeting, Washington, D.C., September 21–22, 2000.
Gordon, Nancy. The American Community Survey. Race and Ethnic  Advisory Committees meeting, Washington, D.C., November 2, 2000.
Gordon, Nancy. American Community Survey Update. Decennial Census  Advisory Committee meeting, Washington, D.C., May 2, 2002.
Griffin, Deborah. An Overview of the Research and Evaluation Program  for the American Community Survey. Decennial Census Advisory  Committee meeting, Alexandria, Virginia, October 2–4, 2002.
Griffin, Deborah H. Comparing Characteristics from the American  Community Survey and Census 2000: Methodology. Census Advisory  Committee of Professional Associations meeting, Washington, D.C., April  10–11, 2003.
Navarro, Alfredo. American Community Survey: Use of Population  Estimates as Controls in the ACS Weighting. Census Advisory Committee  of Professional Associations meeting, Washington, D.C., October 23, 2003.
Navarro, Alfredo. A Discussion of the Quality of Estimates from the  American Community Survey for Small Population Groups. Census  Advisory Committee of Professional Associations meeting, Washington,  D.C., October 2–3, 2002.
Weidman, Lynn, and Signe Wetrogan. Enhancing the Intercensal  Population Estimates Program with ACS Data: Summary of Research  Projects. Census Advisory Committee of Professional Associations  meeting, Washington, D.C., October 23, 2003.
section_title: Census Bureau Continuous Measurement Series
paragraphs: The memorandums listed here from the 20 in the Continuous  Measurement series are those most directly related to topics we review in  this report.
Alexander, Charles H. A Continuous Measurement Alternative for the  U.S. Census. CM-10, October 28, 1993. CM-11 summarized this paper at the  1993 annual meeting of the American Statistical Association, San  Francisco, California, August 10, 1993.
Alexander, Charles H. Further Exploration of Issues Raised at the  CNSTAT Requirements Panel Meeting. CM-13. Internal Census Bureau  memorandum, Washington, D.C., January 31, 1994.
Alexander, Charles H. A Prototype Continuous Measurement System for  the U.S. Census of Population and Housing. CM-17. Presented at the  annual meeting of the Population Association of America, Miami, Florida,  May 5, 1994.
Alexander, Charles H. Some Ideas for Integrating the Continuous  Measurement System into the Nation’s System of Household Surveys.  CM-19A. Internal Census Bureau memorandum, Washington, D.C., January  6, 1995.
section_title: Census Bureau Internal Reports
paragraphs: 2004 Census Test Operational Plan. Washington, D.C.: September 29,  2003.  2010 Census Decision Memorandum Series No. 5, Finalizing Content for  the 100 Percent Items in the 2010 Census and the American Community  Survey. Washington, D.C.: June 3, 2004.  2010 Census Planning Memorandum Series No. 24, Action Plan: 2010  Research and Development Planning Group on Race and Ethnic Data  Collection, Tabulation, and Editing. Washington, D.C.: June 9, 2004.  2010 Census Planning Memorandum Series No. 26, Action Plan: 2010  Research and Development Planning Group on Special Places/Group  Quarters Development and Testing. Washington, D.C.: March 8, 2004.
ACS-2010 Consistency Review Plan. Washington, D.C.: October 1, 2003.
American Community Survey Development Report Series Program Plan.  Washington, D.C.: rev. June 12, 2002.
section_title: Census 2000 Evaluation Reports
paragraphs: Abramson, Florence. Special Place/Group Quarters Enumeration. Census  2000 Testing, Experimentation, and Evaluation Program, Topic Report  No. 5. U.S. Census Bureau, Washington, D.C., February 2004.
Adlakha, Arjun, J. Gregory Robinson, Kirsten West, and Antonio Bruce.  Assessment of Consistency of Census Data with Demographic  Benchmarks at the Subnational Level. Census 2000 Evaluation O.20. U.S.  Census Bureau, Washington, D.C., August 18, 2003.
Clarke, Sandra, John Iceland, Thomas Palumbo, Kirby Posey, and Mai  Weismantle. Comparing Employment, Income, and Poverty: Census  2000 and the Current Population Survey. Census 2000 Auxiliary  Evaluation. U.S. Census Bureau, Washington, D.C., September 2003.
Palumbo, Thomas and Paul Siegel. Accuracy of Data for Employment  Status as Measured by the CPS-Census 2000 Match. Census 2000  Evaluation B.7. U.S. Census Bureau, Washington, D.C., May 4, 2004.
Schneider, Paula. Content and Data Quality in Census 2000. Census  2000 Testing, Experimentation, and Evaluation Program, Topic Report No.  12. U.S. Census Bureau, Washington, D.C., January 22, 2004.
section_title: Census Bureau 2003 JSM Staff Papers
paragraphs: Bureau staff presented many ACS-related papers at the August 2003 Joint  Statistical Meetings in San Francisco, California. We reviewed the papers  in this section in detail because they were related to comparisons between  ACS estimates and 2000 Census results.
Boggess, Scott, and Nikki L. Graf. Measuring Education: A Comparison  of the Decennial Census and the American Community Survey.  Presented at Joint Statistical Meetings, San Francisco, California, August  7, 2003.
Dye, Jane Lawler. Grandparents Living with and Providing Care for  Grandchildren: A Comparison of Data from Census 2000 and 2000  American Community Survey. Presented at Joint Statistical Meetings,  San Francisco, California, August 7, 2003.
Love, Susan, and Deborah Griffin. A Closer Look at the Quality of Small  Area Estimates from the American Community Survey. Presented at  Joint Statistical Meetings, San Francisco, California, August 4, 2003.
Posey, Kirby G., Edward Welniak, and Charles Nelson. Income in the  American Community Survey: Comparisons to Census 2000. Presented  at Joint Statistical Meetings, San Francisco, California, August 7, 2003.
Raglin, David A., Theresa F. Leslie, and Deborah H. Griffin. Comparing  Social Characteristics between Census 2000 and the American  Community Survey. Presented at Joint Statistical Meetings, San  Francisco, California, August 3, 2003.
Stern, Sharon M. Counting People with Disabilities: How Survey  Methodology Influences Estimates in Census 2000 and the Census 2000  Supplementary Survey. Presented at Joint Statistical Meetings, San  Francisco, California, August 7, 2003.
section_title: Other Census Bureau Staff Research Papers
paragraphs: Alexander, Charles H. American Community Survey Data for Economic  Analysis (October 2001). Presented at the Census Advisory Committee of  the American Economic Association meeting, Washington, D.C., October  18–19, 2001.
Alexander, Charles H. Recent Developments in the American Community  Survey. Presented at the 1998 Joint Statistical Meetings, Dallas, Texas,  August 12, 1998.
Alexander, Charles H., Sharon Brown, and Hugh Knox. American  Community Survey Data for Economic Analysis (December 2001).  Presented at the Federal Economic Statistics Advisory Committee  meeting, Washington, D.C., December 14, 2001.
Alexander, Charles H., Scot Dahl, and Lynn Weidman. Making Estimates  from the American Community Survey. Presented at the 1997 Joint  Statistical Meetings, Anaheim, California, August 13, 1997.
Alexander, Charles H., and Signe Wetrogan. Integrating the American  Community Survey and the Intercensal Demographic Estimates  Program. Presented at the 2000 Joint Statistical Meetings, Indianapolis,  Indiana, August 14, 2000.
Butani, Shail, Charles Alexander, and James Esposito. Using the  American Community Survey to Enhance the Current Population  Survey: Opportunities and Issues. Presented at the 1999 Federal  Committee on Statistical Methodology Research Conference, Arlington,  Virginia, November 15-17, 1999.
Davis, Mary Ellen, and Charles H. Alexander, Jr. The American  Community Survey: The Census Bureau's Plan to Provide Timely 21st  Century Data. Missouri Library World, Spring 1997.
DeMaio, Theresa J., and Kristen A. Hughes. Report of Cognitive Research  on the Residence Rules and Seasonality Questions on the American  Community Survey. U.S. Bureau of the Census, Statistical Research  Division, Washington, D.C., July 2003.
Love, Susan, Donald Dalzell, and Charles Alexander. Constructing a  Major Survey: Operational Plans and Issues For Continuous  Measurement. Presented at the 1995 Joint Statistical Meetings, Orlando,  Florida, August 16, 1995.
Nelson, Charles, and Kathleen Short. The Distributional Implications of  Geographic Adjustment of Poverty Thresholds. U.S. Bureau of the Census,  Housing and Household Economics Statistics Division, Washington, D.C.,  December 2003.
Posey, Kirby G., and Edward Welniak. Income in the ACS: Comparisons  to the 1990 Census. Presented at the American Community Survey  Symposium, Suitland, Maryland, March 1998.
Salvo, Joseph, and Arun Peter Lobo. The American Community Survey:  Quality of Response by Mode of Data Collection in the Bronx Test Site.  Presented at the 2002 Joint Statistical Meetings, New York, August 14,  2002.
Smith, Amy Symens. The American Community Survey and Intercensal  Population Estimates: Where Are the Crossroads? Technical Working  Paper 31, U.S. Census Bureau, Population Division, Washington, D.C.,  December 1998.
section_title: Association of Public Data Users Papers
paragraphs: Davis, Mary Ellen. The American Community Survey Data Products,  Alexandria, Virginia, October 20, 2003.
Gage, Linda, State of California, Department of Finance. American  Community Survey: Research by the Data User Community. Alexandria,  Va.: October 20, 2003.
Petroni, Rita. How Do 3-Year Averages from the ACS Compare to Census  2000 Data? (Preliminary Results). Alexandria, Va.: October 20, 2003.
Salvo, Joseph, City of New York, Planning Department. American  Community Survey: Research by the Data User Community. Alexandria,  Va.: October 20, 2003.
Scarr, Harry A. Deputy Director, Census Bureau. Continuous  Measurement. Association of Public Data Users, Washington, D.C.;  October 16, 1994.
section_title: Congressional Hearings and Testimony
paragraphs: Barron, William Jr., Acting Director, U.S. Bureau of the Census, before the  U.S. House of Representatives, Committee on Government Reform,  Subcommittee on the Census. The Census Bureau’s Proposed American  Community Survey (ACS), Serial 107-9. Washington, D.C.: June 13, 2001.
Kincannon, Charles Louis, Director, U.S. Bureau of the Census, before the  U.S. House of Representatives, Subcommittee on Technology, Information  Policy, Intergovernmental Relations, and the Census. The American  Community Survey: The Challenges of Eliminating the Long Form from  the 2010 Census, Serial 108-97. Washington, D.C.: May 13, 2003.
Prewitt, Kenneth, Director, U.S. Bureau of the Census, before the U.S.  House of Representatives, Committee on Government Reform,  Subcommittee on the Census. House Hearing on ACS July 20, 2000. The  American Community Survey: A Replacement for the Census Long  Form? Serial 106-246. Washington, D.C.: July 20, 2000.
section_title: Other Reports and Papers
paragraphs: Kalton, Graham, and others. The American Community Survey: The  Quality of Rural Data, Report of a Conference. Rockville, Md.: Westat,  June 29, 1998.
Nardone, Thomas, and others. Examining the Discrepancy in  Employment Growth between the CPS and the CES. Washington, D.C.:  FESAC, October 17, 2003.
National Council on Disability. Improving Federal Disability Data.  Washington, D.C.: January 9, 2004.
ORC Macro. The American Community Survey: Challenges and  Opportunities for HUD. Calverton, Md.: September 27, 2002.
Vroman, Wayne. Comparing Labor Market Indicators from the CPS and  ACS. Washington, D.C.: Urban Institute, September 2003.
Westat Inc. The American Community Survey: A Report on the Use of  Multi-Year Averages. Rockville, Md.: April 30, 1999.
section_title: Related GAO Products
paragraphs: 2010 Census: Cost and Design Issues Need to Be Addressed Soon.   GAO-04-37. Washington, D.C.: January 15, 2004.
Medicaid Formula: Differences in Funding Ability among States Often  Are Widened. GAO-03-620. Washington, D.C.: July 10, 2003.
Formula Grants: 2000 Census Redistributes Federal Funding Among  States. GAO-03-178. Washington, D.C.: February 24, 2003.
Major Management Challenges and Program Risks: Department of  Commerce. GAO-03-97. Washington, D.C.: January 1, 2003.
The American Community Survey: Accuracy and Timeliness Issues.  GAO-02-956R. Washington, D.C.: September 30, 2002.
Legal Authority for American Community Survey. B-289852.  Washington, D.C.: April  4, 2002.
Medicaid Formula: Effects of Proposed Formula on Federal Shares of  State Spending. GAO/HEHS-99-29R. Washington, D.C.: February 19, 1999.
Decennial Census: Overview of Historical Census Issues.   GAO/GGD-98-103. Washington, D.C.: May 1, 1998.
Poverty Measurement: Adjusting for Geographic Cost-of-Living  Difference. GAO/GGD-95-64. Washington, D.C.: March 9, 1995.
Status of the Statistical Community after Sustaining Budget  Reductions. GAO/IMTEC-84-17. Washington, D.C.: July 18, 1984.