2020 Census: Local Administrative Records and Their Use in the Challenge Program and Decennial
Published: 2013-02-21T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The Bureau's Population Estimates Challenge Program gives local governments the opportunity to challenge the Bureau's annual estimates of their population counts during the years between decennial censuses. Challenges rely on local administrative records, such as building and demolition permits. In addition to their role in the challenge program, these and national administrative records, such as tax data and Medicare records, could save the Bureau money if they are used to help build the Bureau's master address list, and reduce the need for certain costly and labor-intensive door-to-door visits among other things. GAO was asked to review changes to the challenge program and the Bureau's use of administrative records.
This report describes the changes to: (1) how local administrative records will be used in the challenge program; (2) how the Bureau will assure the quality of population estimates updated by the challenge program; and describes (3) what plans, if any, the Bureau has to use the types of local administrative records used for the challenge program to improve the cost or quality of the 2020 Decennial Census.
GAO reviewed documentary and testimonial evidence from Bureau officials and state and local data experts. Additionally, GAO interviewed Bureau officials to identify changes to the challenge program and reviewed documentation on the challenge program's quality assurance processes. GAO provided a draft of this report to the Department of Commerce. In response the Bureau provided technical comments, which were incorporated as appropriate.
section_title: What GAO Found
paragraphs: The Census Bureau (Bureau) issued significant changes to rules governing the records that communities use to challenge the Bureau's population estimates. Previously, the Bureau routinely accepted all challenges, largely without regard to the data sources cited or provided so long as they supported the calculations and covered the appropriate reporting periods. According to Bureau officials, these changes are based on research that shows that estimates based on some methods and records (e.g., births, deaths, and migration) are substantially more accurate than estimates based on others. Among other changes, the Bureau modified procedures so that challenges by subcounty governments to the Bureau's estimates of people living in housing units will no longer affect countylevel population estimates. Moving forward, any such challenge resulting in an increase in the estimate of a subcounty population will be offset by a downward revision to the population estimate of all other communities in the same county. Also, the Bureau plans to routinely review population challenges in light of each community's population growth trend.Corroborating data will be required for challenges inconsistent with the trend.
Challenge program officials told GAO that in the past the program focused quality assurance on (1) reviewing the calculations in the documentation submitted by local governments as part of challenge submissions and (2) checking documents and calculations for internal consistency. Moving forward, the Bureau is preparing a quality assurance plan for Bureau staff who review challenges to better ensure proper handling and processing of challenges, as well as the review of calculations.
The Bureau's 2020 research and testing program is exploring the use of local administrative records for the 2020 Census, such as those used in the challenge program, but this effort is a lower priority than research on the use of national records, in part because national administrative records show greater promise than local records for controlling costs. Bureau officials said local records show the most promise for supporting the development of the 2020 address list. Specifically, the Bureau is exploring how it can use local records to more seamlessly and continually update address lists and maps, rather than waiting to receive such information as part of a one-time decennial update. Bureau officials stated that it is important to continue research on local records because they may be helpful in targeting decennial operations to hard-to-count groups or those in certain geographic areas. However, the results of 2010 Census research and testing on national records have led Bureau officials to conclude that continuing research on national records should be a higher priority.

section_title: Letter
paragraphs: With a life cycle cost of around $13 billion, the 2010 Census was the most  expensive population count in U.S. history, costing over 50 percent more  than the $8.1 billion 2000 Census. The U.S. Census Bureau (Bureau)  estimates that if it used the same approach to count people in 2020 as it  did in 2010, it could potentially cost around $25 billion, or $151 to count  each housing unit compared to the $97 it cost in 2010. The Bureau is  considering a number of ways to control the costs of the 2020 Census  including the expanded use of administrative records, that is, data  collected and maintained by other organizations for the purpose of  administering programs and providing services, such as Medicare  enrollment data or Selective Service System registration address  information. The Bureau made limited use of administrative records for  the 2010 Census. For example, the Bureau used United States Postal  Service files to update its address list, and federal agency records, such  as those from the Department of Defense, were used to count military  and federal civilian employees stationed outside of the United States.
Administrative records could save the Bureau money if the Bureau uses  them to reduce the need for certain costly and labor-intensive door-to- door visits by Bureau employees, such as collecting data in-person from  nonrespondents, supporting quality control, or helping to evaluate the  quality of the census. The Bureau has initially estimated that by  expanding the use of administrative records it could save up to $2 billion  by reducing the workload from these and other operations. Administrative  records are already used by the Bureau’s Population Estimates  Challenge Program, which gives local governments the opportunity to  challenge the Bureau’s annual estimates of their population counts during  the years between decennial censuses. Under the challenge program,  local governments use their own local administrative records to question  the Bureau’s official population statistics. While not affecting  congressional apportionment or redistricting, the challenges can affect  other uses of census data, such as various state and federal assistance  programs where funding and eligibility are based in whole or in part on  population size. Over the past several months, the Bureau considered  various changes to the challenge program that are designed to, among  other improvements, enhance the accuracy of the intercensal estimates.  The Bureau issued the final rule in the January 3, 2013, Federal  Register, and the changes became effective as of February 4, 2013.
Given the importance of accurate population estimates, you asked us to  describe these changes. This report describes (1) how local  administrative records will be used in the challenge program and (2) how  the Bureau will assure the quality of population estimates updated by the  challenge program. Moreover, you asked us to describe the role  administrative records could play in controlling census costs. Accordingly,  this report describes what plans, if any, the Bureau has to use the types  of local administrative records currently used for the challenge program to  improve the cost or quality of the 2020 Decennial Census.
For all three objectives, we reviewed documentation from the Bureau on  2020 research and testing of administrative records, including local  records. We reviewed documentary and testimonial evidence from  Bureau officials and state and local data experts to identify specific  processes and local records used under the challenge program.  Additionally, we interviewed Bureau officials to identify changes to the  challenge program and reviewed documentation on the challenge  program’s quality assurance processes from the Bureau Population  Division officials responsible for implementing the program. More  information on our scope and methodology can be found in appendix I.
We conducted our work from September 2012 to February 2013 in  accordance with all sections of GAO’s Quality Assurance Framework that  are relevant to our objectives. The framework requires that we plan and  perform the engagement to obtain sufficient and appropriate evidence to  meet our stated objectives and to discuss any limitations in our work. We  believe that the information and data obtained, and the analysis  conducted, provide a reasonable basis for any findings and conclusions in  this product.
subsections: section_title: Background
paragraphs: Administrative records are a growing source of information about  individuals and households. Administrative records include records from  government agencies, such as tax data and Medicare records, as well as  commercial sources from major national data vendors. National  administrative records refer to data compiled and maintained nationwide,  including files compiled for the purpose of administering federal  programs. In comparison, data compiled and maintained by municipalities  are referred to as local administrative records, such as building permits  and local tax records.
According to Bureau officials, for the 2020 Census, the Bureau is  researching how to determine the quality and usefulness of administrative  records for obtaining addresses or information about individuals.  Administrative records could reduce the cost of the census if they can  help the Bureau reduce the workload for several operations, including  address list building; quality assurance; and nonresponse follow-up,  which, at $1.6 billion in 2010 and lasting several weeks, was the largest  and most costly census field operation.testing program has nine research and testing projects that are exploring  the expanded use of administrative records for these purposes.
The Bureau’s 2020 research and  The Bureau already uses administrative records to produce annual  population estimates for the nation, states, counties, cities, towns, and  townships as part of its program to estimate changes in population size  and distribution since the previous census. The Bureau produces  estimates at the state and county-level based on births, deaths, migration,  and changes in the number of people who live in group quarters, such as  college dormitories and nursing homes. Estimates of the population of  subcounty communities—which consist of both incorporated places, such  as cities, boroughs, and villages, and minor civil divisions, such as towns  and townships—are primarily based on data on housing units,  occupancy rates, and persons per household plus an estimate of the  population in group quarters.
Local governments may challenge these population estimates through a  process established by the Bureau’s Population Estimates Challenge  Program. Local data sources for challenges have included building permits, non-permitted construction, demolition permits, non-permitted demolitions, certificates of occupancy, utility connection data, and real and personal property tax information on residential units.
The Bureau permitted localities to submit challenges for population  estimates from 2001 to 2008. Counties and subcounties could challenge  the Bureau’s estimate based on evidence that the number of housing  units in their locality differed from the Bureau’s estimate of housing units.  From 2001 to 2008, the Bureau reports revising population estimates  from 287 challenges. These challenges were from 211 governments in 36  states and the District of Columbia. There were as few as 3 challenges in  2001 and as many as 61 each in 2006 and 2007. In 2010, the Bureau  temporarily halted the challenge program beginning for estimates from  2009 to accommodate the 2010 decennial census. The Bureau is  resuming the challenge program in 2013.
According to the Bureau, communities that challenged their population  estimate saw their estimate revised upward by an average of about 9.4  percent over the 8-year period. These challenges ranged from an  estimated increase of over 186 percent for Bluffton, South Carolina, in  2008, to a decrease of almost 18 percent for Winthrop, Massachusetts,  during the same year. In total, four communities submitted challenges  containing evidence that resulted in a decrease in population.
The effect of the program on smaller communities that participated was  much larger, on average in percentage terms, than it was for larger  communities that participated. According to Bureau data summarizing the  program, communities with populations of less than 100,000 averaged an  almost 13 percent upward revision in their population estimate—including  the results of the challenge—compared to an average revision of less  than 2 percent for communities with a population of more than 1 million  (see table 1).
section_title: The Bureau Revised Rules Governing Data Sources and Procedures for the Challenge Program to Improve Accuracy of Population Estimates
subsections: section_title: The Bureau Changed the Challenge Program to Standardize Evidentiary Requirements and Limit the Scope
paragraphs: Previously, the Bureau provided communities that chose to challenge  their population estimate with examples of the types of records they could  use to support their calculations, including permits for new residential  construction, public utility connection data, real and personal property tax  information on residential units, and records of annexations and other  types of legal boundary changes. In practice, according to agency  officials, the Bureau generally accepted all challenges, largely without  regard to the data sources provided so long as they supported  calculations of population change and covered the reporting periods  required by the challenge program. The rule changes are intended to  improve the quality of data that communities use to challenge the  Bureau’s population estimates and will affect the scope of what county  and state governments can challenge. According to Bureau officials,  these changes are based on research that shows that estimates based  on some methods and records (e.g., births, deaths, and migration) are  substantially more accurate than estimates based on others. The rules  vary for different levels of government—subcounty, county or equivalent,  and state.
Subcounty. Under the new rules, subcounty governments should use  building, demolition, and mobile home permits, and group quarter counts  to challenge population estimates. The Bureau plans to reject challenges  that rely solely on other types of data, except in cases where the data  provide overwhelming evidence that the Bureau’s estimate of population  growth is in error.research shows that public utility records can vary widely in their reliability  to indicate population growth, as well as in their availability to the public.  Therefore, utility records will be treated as corroborating, or secondary,  evidence to support other preferred data expected to be provided in  support of challenges. Utility records were ordinarily accepted as a basis  for successful challenges in the past.
For example, according to a senior Bureau official,  County and equivalent. Under the changes, county governments and  their equivalents should use birth and death records, immigration data,  and group quarters counts to challenge population estimates—a  significant reduction in the types of records from what the Bureau has  historically accepted. According to Bureau officials, research conducted  over the past several years has demonstrated that county-level estimation  methods based on other data sources do not produce, on average, as  accurate estimates of population. Consequently, the Bureau plans to  reject county-level challenges relying on other data sources. Under the  changes, the Bureau will only accept county-level challenges to either the  accuracy of the data the Bureau itself used when producing estimates or  to whether the Bureau carried out its estimation procedures properly,  such as in handling data files properly or carrying out calculations. Bureau  officials acknowledged that local governments might see this reduction in  the scope of permissible county-level challenges as being too restrictive,  as evidenced by some public comments submitted in response to the  proposed changes. Bureau officials stated that they will continue their  dialogue with representatives of the Federal-State Cooperative for  Population Estimates—a partnership between the Bureau and state  governments—and other researchers about improving methods of  estimating population, and may revisit the structure of the challenge  program if future research demonstrates that an alternative method of  estimating county population consistently outperforms that used by the  Bureau. Figure 1 shows changes in the types of administrative records  accepted by the challenge program.
State. The Bureau will generally no longer permit state governments to  submit their own challenges. Bureau officials stated that counties and  cities are the most appropriate entities to submit a population challenge  for their community because they have greater knowledge of their  population. Bureau officials also said that they want to avoid situations  where a state may challenge estimates for communities where respective  local governments disagree with such challenges. The Bureau will allow  states to submit challenges for counties or equivalents where there is no  seat of government, such as in certain New England states and in parts of  Alaska, but will otherwise require all communities affected by a challenge  to have their government communicate directly with the Bureau.
section_title: Other Changes to Procedures are Intended to Improve Accuracy of Population Estimates
paragraphs: The changes to procedures are intended to improve the accuracy of  revisions to population estimates stemming from challenges. The Bureau  modified procedures so that challenges by subcounty governments to the  Bureau’s estimates of people living in traditional housing—not living in  group quarters arrangements—will no longer affect county-level  population estimates. Previously, successful subcounty challenges were  added to respective county-level populations. The Bureau justifies the  change with its research demonstrating that its method for estimating the  county population that lives in housing is generally better than the method  typically used to challenge subcounty estimates. According to Bureau  officials, the method used to challenge subcounty estimates introduced  an upward bias for communities that were experiencing population  decline or a slowing of population growth. According to the change, any  challenge that results in an increase to the estimate of a subcounty  community’s population living in traditional housing will be offset by a  downward revision to the estimate for all other communities in the same  county, in effect reallocating the estimated population within the county.  Bureau officials emphasized that under the new procedures, as before, if  a government successfully challenges the Bureau’s estimate of the  population living in group quarters arrangements within its community, the  revision will be added to the county population as well, and will not be  offset by changes to populations in other communities in the county.
The Bureau also changed procedures so that it routinely reviews  subcounty population challenges in light of each community’s population  growth trend, requiring corroborating information when a government  claims population growth that is inconsistent with the trend. For example,  one local government in the Midwest successfully challenged the  Bureau’s 2006 population estimate and received an increased population  estimate of about 5.5 percent, even though evidence it submitted showed  the number of housing units in the community was about 3.5 percent  lower than the Bureau’s original calculation for that year. Another local  government successfully challenged the Bureau’s 2007 population  estimate and received an increased population estimate of about 7.4  percent, even though its evidence showed an increase in housing units of  only about 0.8 percent. According to 2010 Census counts in both of these  cases, the average actual annual population growth over the decade was  far below the challenge result, and the trend in population was declining.  A senior Bureau official with whom we spoke said that, in addition, this  and other changes in procedures will reduce the incentive that some  communities have had to file a challenge in order to provide a community  only a temporary reprieve from an otherwise declining trend in their  population.
section_title: The Bureau Reviews Quality of Calculations and Requires Local Governments to Certify that Data Are Accurate
paragraphs: Challenge program officials told us that the program focuses its quality  assurance on (1) reviewing the calculations presented in the  documentation submitted by local governments as part of challenge  submissions and (2) checking documents and calculations for internal  consistency. Specifically, different worksheets local governments could  choose to complete under the program have different data and  calculations required to support a challenge, which Bureau staff review for  accuracy and consistency. For example, Bureau staff would identify when  local governments provided data on local building permits with a  significant lag time, and then revise the submitted calculations  accordingly since some of the buildings would thus have been  constructed outside the time frame for which the population change was  being estimated.
Bureau officials told us that the Bureau assures quality of locally- submitted records by requiring a community’s highest elected official— such as a mayor or county commission chair—to certify the validity of  data used in any challenge to a population estimate that its government  submits. The Bureau takes such certification at “face value” and does not  examine the quality of these records, which Bureau officials said would be  prohibitively time consuming to investigate or verify. Bureau officials said  that no change to that approach is planned for the future of the program.  However, the Bureau is considering how to describe a quality threshold  that local records should satisfy, or steps that local governments can take  to check the quality of their records.
Additionally, the Bureau distributes a review guide containing  standardized procedures to local governments that are interested in  participating in the challenge program. The guide includes instructions on  filling out standardized worksheets and descriptions of the types of  administrative records that can be relied on as sources of data for the  challenge. According to Bureau officials, this helps to ensure consistency  across challenges and potential revisions to population estimates.
Moving forward, according to officials, the Bureau is preparing a quality  assurance plan for Bureau staff who review challenges to better ensure  proper handling and processing of challenges, as well as the review of  calculations. The Bureau intends to develop the plan over the coming  months, after the program has resumed. The Bureau is also undertaking  an agency-wide effort to update its record keeping policies, which would  include revising rules for retaining documentation submitted as part of the  challenge program. Changes to record keeping policies would help the  Bureau maintain challenge program documentation—including  calculations of revised estimates—in the event the Bureau needs to  perform any follow-up reviews.
In our review of the 287 challenges submitted to the Bureau resulting in  revised population estimates from 2001 to 2008, we identified a number  of cases where documentation of challenges appeared either incomplete  or inconsistent. For example, in some cases, documentation of  corrections to local government’s calculations was missing, as was  certification from a community’s highest elected official that data  submitted with a challenge were valid. Also missing were notices to local  governments from the Bureau on whether challenges were accepted. A  lack of documentation could make it difficult for an independent  verification of the integrity of the program. We discussed these  documentation issues with Bureau officials. In response, they explained  the steps they were taking to address them, such as documenting a  record keeping policy for the program that includes descriptions of what  documentation to create for each challenge and checklists of specific  items to be retained in files. Bureau officials agreed to provide us with  copies of the revised record keeping policy when the challenge program  is resumed later in 2013. Because of these planned actions, we are not  making a recommendation to the Bureau at this time.
section_title: The Bureau has Limited Plans for Using Local Records to Supplement National Records for 2020
paragraphs: The Bureau is exploring the use of local records for the 2020 Census, but  this effort will be of a lower priority than research on the use of national  administrative records, in part because national records show greater  promise than local records for controlling costs. The Bureau’s research on  the use of national administrative records for the 2020 Census is focused  on their possible use for such purposes as (1) building the address list,  (2) counting people, and (3) quality assurance and evaluation processes.  In contrast, Bureau officials believe that the use of local administrative  records is most likely to support the development of the 2020 address list.  The Bureau has used local census records in this regard in prior  decennials.
In previous enumerations, the Bureau developed its address list in part by  going door-to-door and canvassing every block in the country to verify  street addresses. However, the process of going door-to-door is labor  intensive. As a result, for 2020, the Bureau is exploring how to reduce  much of the costly, national canvassing done in the past by combining  both local and national administrative data with the United States Postal  Service data it already uses, allowing the Bureau to continuously update  its master address file throughout the decade. In particular, the Bureau is  considering how it can more seamlessly integrate regular input from local  government address lists and related geographic information systems into  its existing address list and map development processes. The Bureau  would like to update this information continuously rather than wait to  receive the input as part of a one-time decennial update. In addition, the  Bureau will research the quality of these updates to determine the extent  to which it can rely on them without necessarily having to verify them  door-to-door.
Bureau officials stated that if time and resources permit, the Bureau will  consider other ways that local records can be used to supplement the use  of national records. The officials stated that while it is important to  continue research on local records, because they may be helpful in  targeting decennial operations to hard-to-count groups or those in certain  geographic areas, the results of 2010 Census research and testing on  national records has led Bureau officials to conclude that continuing  research on national records, such as those listed in figure 2, should be a  higher priority. Bureau officials believe that research on the more broadly  available national records could yield cost savings more quickly than  research on locally available records, and that given resource constraints,  they should attempt to “lock in” at least some of the more likely cost  savings before pursuing more uncertain ones. Moreover, legislation may  be needed to allow the Bureau to expand its use of national records for  decennial purposes. Figure 2 shows the national and local records the  Bureau is considering and the operations these records could support.
Beyond helping to develop the address list, local administrative records  that have been used in the challenge program, such as school enrollment  data, could supplement national records to either improve or evaluate the  counting of targeted populations. Some Bureau stakeholders have  suggested to the Bureau that these records, in some locations, may be  more comprehensive and accurate than those records on which the  Bureau already relies. For example, the Bureau is researching how  statewide data, such as Supplemental Nutrition Assistance Program data  from Illinois, Maryland, New York, and Texas, could enhance person and  housing unit coverage—access to similar records is under negotiation  with several other states. By matching records in these files with the 2010  Census and other administrative records, the Bureau is determining their  suitability for use to identify addresses or people and their characteristics,  as well as their accuracy. Additionally, the Bureau is considering research  on other state-level files, such as those maintained by Bureau partners in  Florida and Montana, which maintain records on utility use and residential  construction, respectively.
However, the Bureau would face some challenges using these records for  decennial purposes. Bureau officials said local records are generally  more difficult to systematically access or to apply broadly to census  operations. In some cases, laws restrict the use of these records to  certain purposes, so the Bureau may need to negotiate their use with  various parties or work towards legislative change. For example, going  beyond school system enrollment data and obtaining access to student  and school lunch data that might help target hard-to-reach populations is  restricted to use for educational purposes by the Family Educational   Additionally, some of these records are not  Rights and Privacy Act.aggregated nationally, which could make it difficult for the Bureau to  obtain them. According to the Bureau, in other cases such as the  Supplemental Nutrition Assistance Program, not all states maintain  records equally well. If the Bureau ultimately has the time and resources  to supplement national data with local data, officials stated that they will  work to address these issues.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this report to the Department of Commerce. In  response, we received written comments from the department, which are  reprinted in appendix II. In its comments, the department stated that it  appreciates the time and effort that we put into the draft report and  thanked us for responding to technical comments provided earlier by  Bureau staff.
As arranged with your office, unless you publicly announce the contents  of this report earlier, we plan no further distribution until 30 days after the  date of this report. At that time, we will send copies of this report to the  Acting Secretary of Commerce, the Under Secretary of Economic Affairs,  the Acting Director of the U.S. Census Bureau, and interested  congressional committees. The report also is available at no charge on  GAO’s website at http://www.gao.gov.
If you have any questions about this report please contact me at (202)  512-2757 or goldenkoffr@gao.gov. Contact points for our Offices of  Congressional Relations and Public Affairs may be found on the last page  of this report. The GAO staff that made major contributions to this report  are listed in appendix III.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: To describe the U.S. Census Bureau’s (Bureau) changes to how local  administrative records will be used in the challenge program, we reviewed  the August 10, 2012, Federal Register outlining proposed changes; the  final rule, which was issued in the January 3, 2013, Federal Register;  Bureau reports and presentations, which served as the basis for the  changes; and comments received from state and local data experts  solicited by the Federal Register on the proposed changes prior to the  Bureau’s issuance of the final rule. We also interviewed current and  retired officials from the Bureau’s Population Division responsible for  implementing the challenge program.
To describe the changes to how the Bureau will assure the quality of  population estimates updated by the challenge program, we reviewed the  Federal Register notice outlined earlier, and we interviewed Bureau  officials to identify additional quality assurance steps the Bureau intends  to implement. To better understand the current procedures and  implications of these changes, we reviewed documentation the Bureau  provided to local governments on submitting accurate documentation to  challenge population estimates, including worksheets used to calculate  revised population estimates. We conducted a case file review to identify  specific quality assurance steps the Bureau has previously taken to  review the quality of submissions from local governments. During our  case file review, we observed a number of instances where case files and  other documentation of challenges appeared to be incomplete or  inconsistent. We reviewed each of the documentation weaknesses, and  we shared our observations with ranking Bureau executive and program  managers.
To examine the Bureau’s plans to use the types of local administrative  records currently used by the challenge program to improve the cost or  quality of the 2020 Decennial Census, we reviewed Bureau  documentation on research and testing of administrative records for the  2020 Census, as well as our prior reports on the Bureau’s research and  testing efforts. Additionally, we interviewed the Bureau officials  responsible for the research and testing efforts related to administrative  records to understand which local records and processes from the  challenge program the Bureau is considering for the 2020 Census.
section_title: Appendix II: Comments from the Department of Commerce
section_title: Appendix III: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: Other key contributors to this report include Ty Mitchell, Assistant  Director; David Bobruff; Benjamin Crawford; Sara Daleski; Robert  Gebhart; Will Holloway; Andrea Levine; Mark Ryan; and Timothy Wexler.