2010 Census: Preliminary Lessons Learned Highlight the Need for Fundamental Reforms
Published: 2011-04-06T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: GAO added the 2010 Census to its list of high-risk programs in 2008 in part because of (1) long-standing weaknesses in the Census Bureau's (Bureau) information technology (IT) acquisition and contract management function, (2) difficulties in developing reliable life-cycle cost estimates, and (3) key operations that were not tested under operational conditions. These issues jeopardized the Bureau's readiness for the count. Moreover, societal trends, such as concerns over privacy, have made a cost-effective census an increasingly difficult challenge. At about $13 billion, 2010 was the costliest U.S. Census in history. As requested, this testimony focuses on lessons learned from the 2010 Census, and initiatives that show promise for producing a more cost-effective population count in 2020. This testimony is based on completed and ongoing work, including an analysis of Bureau documents, interviews with Bureau officials, and field observations of census operations in urban and rural locations across the country.
section_title: What GAO Found
paragraphs: In February 2011, GAO removed the 2010 Census from its High-Risk List because the Bureau generally completed its peak enumeration activities and released congressional apportionment and redistricting data consistent with its operational plans. The Bureau improved its readiness for the census by strengthening its risk management activities, enhancing systems testing, and meeting regularly with executives from its parent agency, the Department of Commerce. Strong congressional oversight was also critical. Still, the 2010 Census required an unprecedented commitment of resources, and the cost of enumerating each housing unit has escalated from around $16 in 1970, to around $98 in 2010, in constant 2010 dollars. Based on the results of the 2010 and prior censuses, the following four early lessons learned could help secure a more cost-effective enumeration in 2020: 1. Reexamine the Nation's Approach to Taking the Census: The Bureau has used a similar approach to count most of the population since 1970. However, the approach has not kept pace with changes to society. Moving forward, it will be important for the Bureau to rethink its approach to planning, testing, implementing, and monitoring the census to address long-standing challenges. 2. Assess and Refine Existing Operations Focusing on Tailoring Them to Specific Locations and Population Groups: The Bureau plans to complete over 70 studies of the 2010 Census covering such topics as the Bureau's publicity efforts and field operations. As this research is completed, it will be important for it to assess the value-added of a particular operation in order for it to determine how best to allocate its resources for 2020. 3. Institutionalize Efforts to Address High-Risk Areas: Focus areas include incorporating best practices for IT acquisition management; developing reliable cost estimates; and ensuring key operations are fully tested, in part by developing clearly stated research objectives, a thoroughly documented data collection strategy, and methods for determining the extent to which specific activities contributed to controlling costs and enhancing quality. 4. Ensure that the Bureau's Management, Culture, and Business Practices Align with a Cost-Effective Enumeration: The Bureau will need to ensure that its organizational culture and structure, as well as its approach to strategic planning, human capital management, collaboration, and other internal functions are focused on delivering more cost-effective outcomes. The Bureau has launched an ambitious planning program for 2020. As these actions gain momentum, it will be important that they enhance the Bureau's capacity to control costs, ensure quality, and adapt to future technological and societal changes.
section_title: What GAO Recommends
paragraphs: GAO is not making new recommendations in this testimony, but past reports recommended that the Bureau strengthen its testing of key IT systems, better document and update its cost estimates, and develop an operational plan that integrates performance, budget, and other information. The Bureau generally agreed with GAO's findings and recommendations and is taking steps to implement them.

section_title: Letter
paragraphs: I am pleased to be here today to discuss planning efforts for the 2020  Census. The next enumeration might seem far over the horizon, but our  reviews of the 1990, 2000, and now 2010 Censuses have shown that early  planning and strong congressional oversight can help reduce the costs and  risks of the national headcount. As you know, because of societal trends,  including an increasingly diverse population and concerns over personal  privacy, a cost-effective census has become inherently difficult. Going  forward, the singular challenge facing the U.S. Census Bureau (Bureau) is  how best to control the cost of future enumerations while maintaining  their accuracy.
This afternoon’s hearing is especially timely. Just one year after Census  Day 2010, and with Census Day 2020 still 9 years down the road, today’s  session provides an early and important opportunity to look back on the  lessons learned from the recent enumeration while simultaneously kicking  off the congressional oversight necessary to help ensure the reforms  needed for the next enumeration proceed on track. Indeed, past  experience has shown that strong and continuing congressional  involvement—especially while there is still time to make decisions and  influence the direction of the census—is essential to the decennial’s  ultimate success.
Moreover, the fundamental design of the enumeration—mail out and mail  back of the census form with in-person follow-up for nonrespondents— has been in use since 1970, and suffers from declining response rates. In  short, this approach is no longer capable of cost-effectively counting a  population that is growing steadily larger, more diverse, increasingly  difficult to find, and reluctant to participate in the census. Much like going  up a down escalator, over the past 40 years, the Bureau has been investing  substantially more resources each decade in an effort to keep pace with  key results from prior enumerations. The 2010 Census, at around $13  billion, was the most expensive headcount in our nation’s history.  Meanwhile, the cost of conducting the census has, on average, nearly  doubled each decade since 1970 in constant 2010 dollars. If that growth  rate continues unchecked, we could be looking at spending more than $25  billion on the 2020 Census.
Early and focused attention is also needed because the census—a  constitutionally mandated effort—is critical to our nation, as the results  are used to apportion seats in Congress, redraw congressional districts,  help allocate more than $400 billion in federal aid to state and local  governments each year, and remake local political boundaries. Census  data are also used for investment decisions by the public and private  sectors. Unfortunately, the Bureau’s planning efforts for the last 3  decennials fell short, which resulted in operational challenges that  jeopardized a successful census. The magnitude of these problems led us  to add the 2000 and 2010 Censuses to our list of high-risk federal programs  in 1997 and 2008.
In 2008, for example, we designated the 2010 Census a GAO high-risk area  because (1) long-standing weaknesses in the Bureau’s information  technology (IT) acquisition and contract management function, (2)  difficulties in developing reliable life-cycle cost estimates, and (3) key  operations that were not tested under operational conditions, all  jeopardized a cost-effective enumeration.
In February 2011, we removed the 2010 Census from the High-Risk List  because the Bureau generally completed its peak census data collection  activities consistent with its operational plans; released the state  population counts used to apportion Congress on December 21, 2010,  several days ahead of the legally mandated end-of-year deadline; and  remaining activities appeared to be on track. More recently, on March 24,  2011, the Bureau announced it had completed the release of data that  states use to redraw federal, state, and local legislative districts.
In removing the 2010 Census from our High-Risk List we noted that the  Bureau improved its readiness for the census by taking such steps as  strengthening its risk management activities, enhancing systems testing,  bringing in experienced personnel to key positions, implementing our  prior recommendations, and meeting regularly with executives from its  parent agency, the Department of Commerce. Importantly, we removed  the high-risk designation because of the Bureau’s strong commitment to  and top leadership support for addressing problems; boosting its capacity  to address shortcomings; and developing a corrective action plan, among  other actions, and not simply because the census was coming to a close.  Moreover, active congressional oversight—including 12 hearings convened  by the House and Senate since we first named the 2010 Census a high-risk  area—helped ensure the Bureau effectively designed and managed  operations and kept the enumeration on schedule. The operational  success of the 2010 Census is also a tremendous credit to the hundreds of  thousands of career and temporary Bureau employees who diligently  implemented a vast array of census-taking activities, often under difficult  circumstances; and to the public, private, tribal, and nonprofit  organizations of all sizes that voluntarily partnered with the Bureau and  raised awareness of the census.
Going forward, while 2020 might seem distant, several issues suggest that  it is not too early for stakeholders to start considering the improvements  necessary to make the next national headcount as cost-effective as  possible. These issues include: (1) the scope of the reforms needed, (2) the  amount of taxpayer dollars at stake, (3) the criticality of a successful  count, and (4) the Bureau’s past planning difficulties. As requested, my  remarks today will focus on lessons learned from the 2010 Census, and  initiatives that show promise for producing an accurate and cost-effective  population count in 2020.
In summary, while still assessing its conduct of the 2010 Census, as the  Bureau continues its planning efforts for 2020, it will be essential for it to  address the following four lessons learned:  reexamine and perhaps fundamentally transform the way the Bureau  plans, tests, implements, monitors, and evaluates future enumerations;   assess and refine existing census-taking operations with an eye toward  tailoring them to specific geographic areas and population groups;   continue to address those shortcomings that led us to designate the  2010 Census a high-risk area including following key practices  important for managing IT and strengthening its ability to develop  reliable life-cycle cost estimates; and  ensure that the Bureau’s management, culture, business practices, and  automated systems are all aligned with a cost-effective enumeration.
While the Bureau needs to do all of this and more, an additional lesson  learned is that the Bureau cannot achieve a successful census on its own.  Rather, the enumeration is a shared national undertaking where federal  agencies, state, local, and tribal governments, nonprofit and private  organizations, and ultimately the public at large, all have vital roles to play.  Congress too, has an essential responsibility. As was underscored by the  2010 experience, House and Senate involvement was essential for  obtaining regular updates on the Bureau’s progress in addressing the  operational challenges it was facing, helping to hold the agency  accountable for results, and providing the Bureau with resources needed  to conduct a successful population count. Today’s hearing, Mr. Chairman,  builds on these past efforts, and will help ensure that lessons learned are  documented and acted upon in the years ahead, raises Congress’s  confidence that the Bureau has learned from 2010, and that the journey  toward 2020 continues in the right direction.
My testimony today is based on our completed work related to key 2010  operations and the Bureau’s planning efforts for 2020, as well as, ongoing  work that is focused on 2010 Census cost drivers and the 2020 life-cycle  cost estimate. For both completed and ongoing work we analyzed key  documents such as budgets, plans, procedures, and guidance for the  selected activities; and interviewed cognizant Bureau officials at  headquarters and local census offices. In addition, for our completed  work, we made on-site observations of key census-taking activities across  the country including such urban locations as Los Angeles, California;  Atlanta, Georgia; Philadelphia, Pennsylvania; Brooklyn, New York; New  Orleans, Louisiana; and Washington, D.C., as well as such less populated  areas as Meridian, Mississippi, and New Castle, Delaware. We selected  these locations because of their geographic and demographic diversity,  among other factors. To obtain information on various management and  organizational reforms that could help the Bureau become more  accountable and results oriented, we reviewed our prior work on  governmentwide reexamination, as well as leading practices and attributes  in the areas of IT management, organizational performance, collaboration,   stewardship, and human capital. More detail on our scope and  methodology is provided in each issued product.
On March 18, 2011, we provided the Bureau with a statement of facts  related to the information included in this statement, and Bureau officials  provided technical comments, which we included as appropriate. We  conducted our work in accordance with generally accepted government  auditing standards. Those standards require that we plan and perform the  audits to obtain sufficient, appropriate evidence to provide a reasonable  basis for our findings and conclusions based on our audit objectives. We  believe that the evidence obtained provides a reasonable basis for our  findings and conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: In conducting the 2010 Census, the Bureau encountered two sets of  challenges: internal management challenges that affected the Bureau’s  overall readiness and led us to designate the 2010 Census as a high-risk  area, as well as external sociodemographic challenges such as more non- English speakers and people residing in makeshift and other  nontraditional living arrangements. As shown in figure 1, the cost of  enumerating each housing unit has escalated from around $16 in 1970, to  around $98 in 2010, in constant 2010 dollars (an increase of over 500  percent). At the same time, the mail response rate—a key indicator of a  cost-effective census—has declined from 78 percent in 1970 to 63 percent  in 2010. In many ways, the Bureau has been investing substantially more  resources each decade just to try and match the results of prior  enumerations.
Average cost per housing unit (in constant 2010 dollars)
Beginning in 1990, we reported that rising costs, difficulties in securing  public participation, and other long-standing challenges required a revised  census methodology—a view that was shared by other stakeholders.  Achieving acceptable results using these conventional methods has  required an increasingly larger investment of fiscal resources, which in the  coming years will likely become scarcer.
Indeed, the 2010 Census required an unprecedented commitment of  resources, including recruiting more than 3.8 million total applicants— roughly equivalent to the entire population of Oklahoma—for its  temporary workforce; and rose in cost from an initial estimate of $11.3  billion in 2001 to around $13 billion. According to the Bureau, several  factors were largely behind the escalating costs of the 2010 Census  including (1) a flawed acquisition strategy, (2) the need to hire a large  number of field staff to enumerate people who did not mail back their  census forms, and (3) substantial investments in updating the Bureau’s  address list just prior to the start of the enumeration.
section_title: Lesson Learned #1: Reexamine the Nation’s Approach to Taking the Census
paragraphs: The results of prior enumerations underscore the fact that simply refining  current methods—some of which have been in place for decades—will not  bring about the reforms needed to control costs while maintaining  accuracy given ongoing and newly emerging societal trends. Since 1970,  the Bureau has used a similar approach to count the vast majority of the  population. For example, the Bureau develops an address list of the  nation’s housing units and mails census forms to each one for occupants  to complete and send back. Over time, because of demographic and  attitudinal trends, securing an acceptable response rate has become an  increasing challenge. Our concerns about the rising cost and diminishing  returns of the census are not new. In the mid-1990s, for example, we and  others concluded that the established approach for taking the census had  exhausted its potential for counting the population cost-effectively and  that fundamental design changes were needed to reduce census costs and  improve the quality of data collected.
A fundamental reexamination of the nation’s approach to the census will  require the Bureau to rethink its approach to planning, testing,  implementing, monitoring, and evaluating the census, and addressing such  questions as: Why was a certain program initiated? What was the intended  goal? Have significant changes occurred that affect its purpose? Does it  use prevailing leading practices?
Our December 2010 report noted potential focus areas for such a  reexamination. These include better leveraging of innovations in  technology and social media to more fully engage census stakeholders and  the general public on census issues; and reaching agreement on a set of  criteria that could be used to weigh the trade-offs associated with the need  for high levels of accuracy on the one hand, and the increasing cost of  achieving that accuracy on the other hand.
One of the areas that the Bureau would like to leverage for the 2020  Census is the use of an Internet response option. The Bureau provided the  opportunity for respondents to complete the 2000 Census short forms on  the Internet—protected by a 22-digit identification number. According to  Bureau officials, for the 2000 Census, about 60,000 short forms were  completed via the Internet. The Bureau originally planned to include the  Internet in the 2010 Census, but then decided not to, because the benefits  gained through processing less paper, as well as improvements to the  quality of data, were outweighed by the cost of developing the Internet  response option and the risks associated with the security of census data.  To examine its use for the 2020 decennial census, the Bureau will need to  review many of those same issues and address the following questions:    To what extent could an Internet response option lower data  collection costs for the Bureau?
To what extent could an Internet response option increase the quality  of data collected?
To what extent does the use of an Internet response option pose a risk  to the confidentiality of census data?
Moreover, given that the research, development, and testing efforts for  2020 will play out over the decade-long census life-cycle, what is the  optimal way to ensure continuity and accountability for an enterprise that  takes years to complete and extends beyond the tenure of many elected  political leaders? The Director of the Census Bureau can, in concept,  provide a measure of continuity, but of the 11 census directors who have  served since July 1969 (not including the current director), the average  tenure was around 3 years, and only 1 director has served more than 5  years. Further, in the decade leading up to the 2010 Census, the Bureau  was led by 4 different directors and several acting directors. The turnover  in the Bureau’s chief executive officer position makes it difficult to  develop and sustain efforts that foster change, produce results, mitigate  risks, and control costs over the long term.
The heads of a number of executive agencies serve fixed appointments,  based on Presidential nomination and Senate confirmation, including the  Director of the Office of Personnel Management (4 years), the  Commissioner of Labor Statistics (4 years), and the Commissioner of  Internal Revenue (5 years). We believe that the continuity resulting from a  fixed-term appointment could provide the following benefits to the  Bureau:    Strategic vision. The director needs to build a long-term vision for  the Bureau that extends beyond the current decennial census.  Strategic planning, human-capital succession planning, and life-cycle  cost estimates for the Bureau all span the decade.
Sustaining stakeholder relationships. The director needs to  continually expand and develop working relationships and  partnerships with governmental, political, and other professional  officials in both the public and private sectors to obtain their input,  support, and participation in the Bureau’s activities.
Accountability. The life-cycle cost for a decennial census spans a  decade, and decisions made early in the decade about the next  decennial census guide the research, investments, and tests carried out  throughout the entire 10-year period. Institutionalizing accountability  over an extended period may help long-term decennial initiatives  provide meaningful and sustainable results.
section_title: Lesson Learned #2: Assess and Refine Existing Operations Focusing on Opportunities to Tailor Census-Taking Activities to Specific Locations and Population Groups
paragraphs: As noted earlier, a key indicator of a cost-effective census is the mail  response rate, which is the percentage of all housing units in the mail-back  universe, including those that are later found to be nonexistent or  unoccupied. High response rates are essential because they save taxpayer  dollars and ensure a more accurate enumeration. According to the Bureau,  for every percentage point increase in mail response in 2010, the Bureau  saved $85 million that would otherwise have been spent on in-person  follow-up efforts. Also, according to the Bureau, it costs 42 cents to mail  back each census form in a postage-paid envelope, compared with an  average estimate of $57 for field activities necessary to enumerate each  home in person. Moreover, mail returns tend to have better quality data, in  part because as time goes on after Census Day (April 1), people move, or  may have difficulty recalling who was residing with them.
For the 2010 Census, the Bureau expected a response rate of 59 percent to  65 percent. The actual mail response rate on April 19 when the Bureau  initially determined the universe of houses to visit for nonresponse follow- up (NRFU) was just over 63 percent, well within the Bureau’s range of  estimates. Achieving this response rate was an important accomplishment  given the nation’s increasing diversity.
As illustrated in figure 2, the Bureau met its expected response rate in all  but 11 states. The highest response rate (71.7 percent) was in Minnesota,  while the lowest response rate (51 percent) was in Alaska. At the same  time, response rates in all but 2 states—Hawaii and South Carolina—as  well as the District of Columbia, declined anywhere from 0.8 to 8.2  percentage points when compared to 2000, thus underscoring the  difficulty the Bureau will face in the future in trying to sustain response  rates.
Key factors aimed at improving the mail response rate included the  mailing of an advance letter and a reminder postcard, and an aggressive  marketing and outreach program. In addition, this was the first census the  Bureau sent a second or “replacement” questionnaire to households.
Replacement questionnaires were sent to around 25 million households in  census tracts that had the lowest response rates in the 2000 Census, and 10  million replacement questionnaires were sent to nonresponding  households in other census tracts that had low-to-moderate response rates  in 2000.
To determine if these and other census-taking activities were effective, the  Bureau plans to complete over 70 studies covering such topics as  marketing and publicity, field operations, privacy and confidentiality, and  language barriers. Moreover, in July 2010, the Bureau developed a  database for cataloging all recommendations from these 2010 studies, as  well as recommendations from our office, the Department of Commerce  Inspector General’s Office, and the National Academy of Sciences, among  others. According to a Bureau official, this database will allow the Bureau  to link 2010 recommendations to 2020 research and testing, in an attempt  to ensure that all recommendations coming out of 2010 are incorporated  into 2020 research.
These studies of the 2010 Census are extremely important for informing  decisions on the design of the 2020 Census. However, some will not be  completed by fiscal year 2012, when the Bureau plans to start research and  testing for the 2020 Census. Bureau officials said they will give priority to  studies that align with the 2020 Census strategic plan. In moving forward,  it will be important for the Bureau to complete 2010 Census studies and  stay on track to ensure that study results, where appropriate, are  incorporated into 2020 research. As such, until all studies from the 2010  Census are finished, the Bureau will not have a complete picture of what  worked well, or know what improvements are needed for 2020.
Moreover, in several of the programs we reviewed, assessments were not  always focused on the value-added of a particular operation, such as the  extent to which it reduced costs and/or enhanced data quality. This  information would be useful for improving operations, identifying possible  duplicative efforts, and identifying potential cost savings for 2020.
As one illustration, a complete and accurate address list, along with  precise maps are the fundamental building blocks of a successful census.  If the Bureau’s address list, known as the Master Address File (MAF) and  maps are inaccurate, people can be missed, counted more than once, or  included in the wrong location. To build an accurate address list and  maps, the Bureau conducted a number of operations throughout the  decade, some of which were extremely labor-intensive. For example, the  Bureau partnered with the U.S. Postal Service and other federal agencies;  state, local, and tribal governments; local planning organizations; the  private sector; and nongovernmental entities. Moreover, the Bureau  employed thousands of temporary census workers to walk every street in  the country to locate and verify places where people could live, in an  operation called address canvassing. Three additional activities were  aimed at properly identifying and locating dormitories, nursing homes,  prisons, and other group living arrangements known as “group quarters.”
In a 2009 testimony, we noted that with the cost of counting each housing  unit growing at a staggering rate, it is important for the Bureau to  determine which of its multiple MAF-building operations provide the best  return on investment in terms of contributing to accuracy and coverage.  A number of operations might be needed to help locate people residing in  different types of living arrangements, as well as to ensure housing units  missed in one operation get included in a subsequent operation. However,  the extent to which each individual operation contributes to the overall  accuracy of the MAF is uncertain. This in turn makes it difficult for the  Bureau to fully assess the extent to which potential reforms such as  reducing or consolidating the number of address-building operations,  might affect the quality of the address list. As one example, while the  Bureau plans study options for targeted address canvassing as an  alternative to canvassing every street in the country, the Bureau’s  evaluation plan does not specify whether the Bureau will look across  MAF-building activities and compare how each individual operation  contributes to the overall accuracy and completeness of the address list  and at what cost.
Leveraging such data as local response rates, census sociodemographic  information, as well as other data sources and empirical evidence, might  also help control costs and improve accuracy by providing information on  ways the Bureau could more efficiently allocate its resources. For  example, some neighborhoods might require a greater level of effort to  achieve acceptable results, while in other areas, those same results might  be accomplished with fewer resources.
To the extent the Bureau targeted various activities during the 2010  Census, initial indications suggest that those efforts went well. For  example, the Bureau developed job aids to address location-specific  training challenges. In one example, partly in response to our  recommendations, to help ensure the Bureau would develop an accurate  address list in those areas affected by Hurricanes Katrina, Rita, and Ike,  the Bureau developed supplemental training materials for natural disaster  areas to help census workers identify less conventional places where  people might be living such as homes marked for demolition, converted  buses and recreational vehicles, and nonresidential space such as storage  areas above restaurants.
As another example, the Bureau budgeted around $297 million on paid  media to raise awareness and encourage public participation in the  census, about $57 million (24 percent) more than in 2000 in constant 2010  dollars. To determine where paid media efforts might have the greatest  impact, the Bureau developed predictive models based on 2000 Census  data and the evaluations of other efforts used for 2000. By better targeting  paid media buys by area and message, the Bureau expected to more  effectively reach those who have historically been the hardest to count.  However, according to the Bureau, this modeling could have been more  robust had the data from 2000 done a better job of isolating the impact of  paid media from other components of the Bureau’s outreach efforts,  among other factors.
Simply put, the Bureau made important progress in using data to  determine where to spend its resources. It will be important for the  Bureau to expand on those efforts in 2020, as well as to develop  information on the return on investment of key census operations.
section_title: Lesson #3: Institutionalize Efforts to Address High-Risk Areas
paragraphs: A key priority for the Bureau will be to fully address those areas that led  us to designate the 2010 Census a high-risk program. The problems the  Bureau encountered in managing its IT systems, developing reliable life- cycle cost estimates, and testing key operations under census-like  conditions were cross-cutting in that they affected a number of different  activities, and thus threatened the Bureau’s readiness for the census. The  Bureau has taken steps to address these vulnerabilities. In the years ahead,  it will be important for the Bureau to continue the progress it has made to  date and ensure that any changes are fully integrated into its basic  business practices.
subsections: section_title: Incorporate Best Practices for IT Acquisition Management
paragraphs: IT is critical to a successful census because it helps support the Bureau’s  data collection, analysis, and dissemination activities. However, the  Bureau has had long-standing difficulties with the development and  acquisition of automated systems. For example, during the 2000 Census,  the Bureau had to grapple with untimely and inaccurate management  information, a lack of mature and effective software and systems  development processes, inadequate testing of key systems, inadequate  security controls, and an insufficient number of experienced staff to  manage expensive and complex system projects. Both we and the  Department of Commerce Inspector General made a series of  recommendations to address these issues, and the Bureau took steps to  implement them. Still, problems reemerged during the run-up to the 2010  Census. For example, while the Bureau planned to use automation and  technology to improve the coverage, accuracy, and efficiency of the 2010  Census, in June 2005, we noted that the Bureau had not fully implemented  key practices important to managing IT, including investment  management, system development and management, and enterprise  architecture management. As a result, we concluded that the Bureau’s  IT investments were at increased risk of mismanagement, and were more  likely to experience cost and schedule overruns and performance  shortfalls.
As development of the IT systems progressed, these problems were  realized. For example, the Field Data Collection Automation program,  which included the development of handheld computers to collect  information for address canvassing and NRFU, experienced substantial  schedule delays and cost increases. As a result, the Bureau later decided  to abandon the planned use of handheld data-collection devices for NRFU  and reverted to paper questionnaires. According to the Bureau, this  change added between $2.2 and $3 billion to the total cost of the census.
The Bureau developed a new automated system to manage the paper- based approach, but the system experienced outages, slow performance,  and problems generating and maintaining timely progress reports.  Workarounds ultimately enabled the Bureau to successfully implement  NRFU. However, the Bureau was still limited in its ability to effectively  monitor productivity or implement quality-assurance procedures as  documented in its operational plans.
Therefore, as the Bureau prepares for 2020, among other actions it will be  important for it to continue to improve its ability to manage its IT  investments. Leading up to the 2010 Census, we made numerous  recommendations to the Bureau to improve its IT management procedures  by implementing best practices in risk management, requirements  development, and testing. The Bureau implemented many of our  recommendations, but not our broader recommendation to institutionalize  these practices at the organizational level. The challenges experienced by  the Bureau in acquiring and developing IT systems during the 2010 Census  further demonstrate the importance of establishing and enforcing a  rigorous IT acquisition management policy Bureau-wide. In addition, it will  be important for the Bureau to improve its ability to consistently perform  key IT management practices, such as IT investment management, system  development and management, and enterprise architecture management.  The effective use of these practices can better ensure that future IT  investments will be pursued in a way that optimizes mission performance.
section_title: Develop More Reliable Life-Cycle Cost Estimates
paragraphs: Accurate cost estimates are essential for a successful census because they  help ensure that the Bureau has adequate funds and that Congress, the  Administration, and the Bureau itself can have reliable information on  which to base decisions. However, we noted in our 2008 report that the  Bureau’s cost estimate for the 2010 Census lacked detailed documentation  on data sources and significant assumptions, and was not comprehensive  because it did not include all costs. We noted that the Bureau had  insufficient policies and procedures, and inadequately trained staff for  conducting high-quality cost estimation for the decennial census, and  therefore recommended that the Bureau take a variety of steps to improve  the credibility and accuracy of its cost estimates. Moreover, following best  practices from our Cost Estimating and Assessment Guide, such as  defining necessary resources and tasks, could have helped the Bureau  generate more reliable cost estimates.
Partly as a result of these issues, some operations had substantial  variances between their initial cost estimates and their actual costs. For  example, the Bureau initially estimated that NRFU would cost around  $2.25 billion. However, by the end of the operation, the Bureau reported  using approximately $1.59 billion, which was 29 percent lower than  budgeted. At the same time, another operation—address canvassing—was  around $88 million (25 percent) more than its initial budget of $356  million, according to a preliminary Bureau estimate.
Moving forward, it will be important for the Bureau to ensure the  reliability of the 2020 cost estimate, and the Bureau has already taken  several actions in that regard. For example, based on recommendations  from our June 2008 report, Bureau officials have stated that some of their  budget staff have been trained and certified in cost estimation. The Bureau  also has started using the Decennial Budget Integration Tool (DBiT).  According to the Bureau, once it has completed entering all needed budget  data, DBiT will consolidate budget information and enable the Bureau to  better document its cost estimates.
Further, as a part of its planning for 2020, Bureau officials said that they  have developed and provided to the Office of Management and Budget  (OMB) for its review a rough order of magnitude estimate for the 2020  Census—based on information at this early stage of 2020 planning. In  addition, the Bureau plans to develop a range of full life-cycle cost  estimates in fiscal year 2013. As noted in our cost estimating guide, a life- cycle cost estimate can be thought of as a “cradle to grave” approach to  managing a program throughout its useful life. Life-cycle costing enhances  decision making, especially in early planning and concept formulation.  Therefore, as the Bureau develops its estimates for 2020, it will be  important for the Bureau to identify all cost elements that pertain to the  program from initial concept all the way through operations and support.
Providing reliable cost estimates that are developed early in a project’s  life-cycle and accompanied by sound justification will be important in  order for Congress to make informed decisions about the levels at which  to fund future decennial censuses. More specifically, greater fiscal  transparency, before committing to a final design and a particular level of  spending, could help inform deliberations on the extent to which (1) the  cost of the census is reasonable, (2) trade-offs will need to be made with  competing national priorities, and (3) additional dollars spent on the  census yield better results.
section_title: Ensure Key Census-Taking Activities Are Fully Tested
paragraphs: The census can be seen as a large, complex, yet inherently fragile machine  comprised of thousands of moving parts, all of which must function in  concert with one another in order to secure a cost-effective count. In  short, while the census is under way, the tolerance for any breakdowns is  quite small. Given this difficult operating environment, rigorous testing is a  critical risk mitigation strategy because it provides information on the  feasibility and performance of individual census-taking activities, their  potential for achieving desired results, and the extent to which they are  able to function together under full operational conditions.
As the Bureau geared up for 2010, we expressed our concern about the  testing of key IT systems and other census-taking activities. For example,  partly because of the performance problems with the handheld computers  noted earlier, the Bureau decided not to include two census operations  (NRFU and Vacant/Delete Check) in the full dress rehearsal for the census  that was scheduled for 2008. In lieu of a full dress rehearsal, the Bureau  tested individual components of the census in isolation. However, without  a full dress rehearsal, the Bureau was unable to demonstrate that various  enumeration activities could function under near-census-like conditions.  Although the Bureau had performed many of these activities in previous  censuses, some operations—such as mailing a second questionnaire to  households that did not complete their census forms by a certain date, the  removal of late mail returns, and fingerprinting hundreds of thousands of  temporary census workers—were new for 2010 and introduced new  operational risks. While the actual enumeration generally proceeded  according to expectations, some operations, most notably the automated  system that the Bureau developed to manage the paper-based NRFU  operation noted earlier, were unable to function under operational loads in  part because of a compressed testing schedule.
Moving forward, as the Bureau refines and implements its testing plans,  our past work on census testing has shown that it will be important for its  strategy to include, but not be limited to, these key components of a sound  study:  clearly stated objectives with accompanying performance measures; research questions linked to test objectives and, as appropriate, a clear  rationale for why sites were selected for field tests; a thoroughly documented data collection strategy; input from stakeholders and lessons learned considered in developing  test objectives; and a data analysis plan including, as appropriate, methods for determining  the extent to which specific activities contribute to controlling costs  and enhancing quality.
While the Bureau does not plan to conduct its first major census test until  April 2014, as part of its research and testing for 2020 the Bureau plans to  conduct 26 tests in support of six different design alternatives between  fiscal years 2012 and 2014. These design alternatives include, for example,  improving the existing 2010 design, using administrative records for  nonresponse follow-up, or increasing the number of available response  options such as the Internet or cell phones. Key elements of the Bureau’s  research and testing strategy include:    performing many small focused field tests in lieu of a few large field  tests as was the case for the 2010 Census; setting up a virtual Local Census Office at Census Bureau headquarters  to test new census methods; and   using the American Community Survey—an ongoing Bureau survey of  population and housing characteristics that is administered throughout  the decade—as a vehicle to test specific census methods.
These tests will be important for determining the feasibility of different  design alternatives. We believe that given the number of tests and design  alternatives that the Bureau plans to evaluate, it will be important to have  a management structure in place for essential functions such as  coordinating the tests; determining priorities; tracking the results;  assessing their implications; weighing cost, accuracy, and other trade-offs;  and ensuring that findings and recommendations are funneled to  appropriate senior Bureau leadership for action.
section_title: Lesson Learned #4: Ensure That the Bureau’s Management, Culture, and Business Practices Are Aligned with a Cost-Effective Enumeration
paragraphs: On the basis of our earlier work on high-performing organizations,  fundamental reforms will mean ensuring that the Bureau’s organizational  culture and structure, as well as its approach to strategic planning, human- capital management, internal collaboration, knowledge sharing, capital  decision making, risk and change management, and other internal  functions are aligned toward delivering more cost-effective outcomes.  Indeed, some of the operational problems that occurred during the 2010  and prior censuses are symptomatic of deeper organizational issues. For  example, the lack of staff skilled in cost estimation during the 2010 Census  points to inadequate human-capital planning, while, as noted above, IT  problems stemmed from not fully and consistently performing certain  functions including IT investment management.
Moreover, the Bureau’s own assessment of its organization found that it  has a number of strengths including a culture that is committed to  accuracy, precision, objectivity, and the overall mission of the census, as  well as a workforce that understands decennial operations, procedures,  and critical subject matter. At the same time, the Bureau’s assessment  noted there were several areas for improvement. For example:  the Bureau is an insular organization and does not always embrace  open communications, transparency, innovation, and change; there were difficulties in drawing on assets and methods from across  the agency; the organizational structure makes it difficult to oversee a large  program and hampers accountability, succession planning, and  personal development, among other factors; and  staff with core skills and experience were lacking in such areas as  management of large programs and projects; cost estimating; and  sophisticated technology, systems, and development.
While reforms will be needed along a number of fronts, our recent work  on governmentwide strategic human capital management highlights some  key steps—some of which the Bureau is already taking—to help ensure it  identifies and closes current and emerging skill gaps to ensure the Bureau  has the workforce needed to effectively and efficiently design and execute  a successful census. These steps include:     developing workforce plans that fully support the Bureau’s need for  highly skilled talent, including defining the root causes of skills gaps,  identifying effective solutions to any shortages, and taking action to  implement those solutions; ensuring recruitment, hiring, and development strategies are  responsive to changing applicant and workforce needs; and evaluating the performance of initiatives to address critical skill gaps  and make appropriate adjustments.
section_title: The Bureau Has Launched an Ambitious Planning Program for 2020
paragraphs: The Bureau, recognizing that it cannot afford to continue operating the  way it does unless it fundamentally changes its method of doing business,  has already taken some important first steps in addressing these questions,  as well as other areas. For example, the Bureau is looking to reform  certain aspects of its IT systems planning, in part to ensure that the  technical infrastructure needed for 2020 will be tested many times before  operations begin. The Bureau also is rebuilding its research directorate to  lead early planning efforts, and has plans to assess and monitor the skills  and competencies needed for the 2020 headcount.
Further, the Bureau already has developed a strategic plan for 2020 and  other related documents that, among other things, lay out the structure of  planning efforts; outline the mission and goals for 2020; and describe the  research and testing phase of the next enumeration. For example, to  address major cost drivers such as field infrastructure, labor, and IT  systems, as well as, the quality of data collected, the Bureau has identified  the following four research tracks that focus on a(n):    Expanded, Automated, and Tailored Response, which attempts to  reduce paper, make it easier for the population to be counted, and  tailor response options, such as the Internet.
Reengineered Field Structure, including a Bureau-wide integrated IT  infrastructure that, for example, will allow for a real-time, Web-based  system to manage data collection in the field.
Continual Address Frame Updating and Targeting, which, for  example, expands the sources of data, to include commercial  databases and administrative records, in the Master Address File so  that a full address canvassing may not be required at the end of the  decade.
Using Administrative Records for Nonresponse, which includes a  major study to determine to what extent administrative records can be  used for nonrespondents.
The Bureau’s early planning efforts are noteworthy given the Bureau’s  long-standing challenges in this area. For example, in 1988, just prior to  the 1990 Census, we noted that the Bureau’s past planning efforts  generally started late, experienced delays, were incomplete, and failed to  fully explore innovative approaches. Planning for the 2000 Census also  had its shortcomings, including, as we noted in our 2004 report, a  persistent lack of priority-setting, coupled with minimal research, testing,  and evaluation documentation to promote informed and timely decision  making. And, while the planning process for the 2010 Census was initially  more rigorous than for past decennials, in 2004 we reported that the  Bureau’s efforts lacked a substantial amount of supporting analysis,  budgetary transparency, and other information, making it difficult for us,  Congress, and other stakeholders to properly assess the feasibility of the  Bureau’s design and the extent to which it could lead to greater cost- effectiveness compared to alternative approaches. As a result, in 2004, we  recommended that the Bureau develop an operational plan for 2010 that  consolidated budget, methodological, and other relevant information into  a single, comprehensive document.
The Bureau later developed specific performance targets and an integrated  project schedule for 2010. However, the other elements we recommended  were only issued piecemeal, if available at all, and were never provided in  a single, comprehensive document. Because this information was critical  for facilitating a thorough, independent review of the Bureau’s plans, as  well as for demonstrating to Congress and other stakeholders that the  Bureau could effectively design and manage operations and control costs,  we believe that had it been available, it could have helped stave off, or at  least reduce, the IT and other risks that confronted the Bureau as Census  Day drew closer.
The Bureau’s strategic plan for 2020, first issued in 2009, is a “living”  document that will be updated as planning efforts progress. As the  approach for 2020 takes shape, it will be important for the Bureau to avoid  some of the problems it had in documenting the planning process for the  2010 Census, and pull all the planning elements together into a tactical  plan or road map. This will help ensure the Bureau’s reform initiatives stay  on track, do not lose momentum, and coalesce into a viable path toward a  more cost-effective 2020 Census. On the basis of our work on planning for  the 2010 Census, a road map for 2020 could include, but not be limited to,  the following elements that could be updated on a regular basis:  specific, measurable performance goals, how the Bureau’s efforts,  procedures, and projects would contribute to those goals, and what  performance measures would be used;    descriptions of how the Bureau’s approaches to human-capital  management, organizational structure, IT acquisitions, and other  internal functions are aligned with the performance goals;   an assessment of the risks associated with each significant decennial  operation, including the interrelationships between the operations and  a description of relevant mitigation plans;    detailed milestone estimates for each significant decennial operation,  including estimated testing dates, and justification for any changes to  milestone estimates;   detailed life-cycle cost estimates of the decennial census that are  credible, comprehensive, accurate, and well-documented as stipulated  by OMB and GAO guidance; and  a detailed description of all significant contracts the Bureau plans to  enter into and a risk management plan for those contracts.
A comprehensive road map could generate several important benefits. For  example, it could help ensure a measure of transparency and facilitate a  more collaborative approach to planning the next census. Specifically, an  operational plan could function as a template for 2020 giving stakeholders  a common framework to assess and comment on the design of the census  and its supporting infrastructure, the resources needed to execute the  design, and the extent to which it could lead to greater cost-effectiveness  compared to alternative approaches. Further, it could be used to monitor  the Bureau’s progress in implementing its approach, and hold the agency  accountable for results. Importantly, to the extent the plan—or aspects of  it—are made available using social media tools, it could prompt greater  and perhaps more constructive civic engagement on the census, by  fostering an ongoing dialog involving individuals and communities of  stakeholders throughout the decade.
section_title: Concluding Observations
paragraphs: The Bureau goes to great lengths each decade to improve specific census- taking activities, but these incremental modifications have not kept pace  with societal changes that make the population increasingly difficult to  locate and count cost-effectively. The Bureau is fully aware of this  problem and has wasted no time in turning the corner on the 2010 Census  and launching the planning efforts needed for a more cost-effective  enumeration come 2020.
Many components are already in place, and a number of assessment and  planning activities are already occurring. At the same time, the Bureau has  also been responsive to the recommendations we have made in our past  work. As these actions gather momentum in the years ahead, it will be  important that they put the Bureau on a trajectory that boosts its capacity  to conduct an accurate count, control costs, manage risks, and be more  nimble in adapting to social, demographic, technological, and other  changes that can be expected in the future. It will also be important for  Congress to continue its strong oversight of the census to help ensure the  progress the Bureau has made thus far continues going forward. We look  forward to supporting the Subcommittee in its decision making and  oversight of the decennial census.
Chairman Carper, Ranking Member Brown, and Members of the  Subcommittee, this concludes my prepared statement. I would be pleased  to respond to any questions that you might have at this time.
section_title: GAO Contacts
paragraphs: If you have any questions on matters discussed in this statement, please  contact Robert Goldenkoff at (202) 512-2757 or by e-mail at  goldenkoffr@gao.gov. Other key contributors to this testimony include  Benjamin Crawford, Vijay D’Souza, Dewi Djunaidy, Ronald Fecso, Robert  Gebhart, Richard Hung, Signora May, Lisa Pearson, Jonathan Ticehurst,  and Timothy Wexler.
section_title: Related GAO Products
paragraphs: High-Risk Series: An Update. GAO-11-278. Washington, D.C.: February  2011.  2010 Census: Data Collection Operations Were Generally Completed as  Planned, but Long-standing Challenges Suggest Need for Fundamental  Reforms. GAO-11-193. Washington, D.C.: December 14, 2010.  2010 Census: Key Efforts to Include Hard-to-Count Populations Went  Generally as Planned; Improvements Could Make the Efforts More  Effective for Next Census. GAO-11-45. Washington, D.C.: December 14,  2010.  2010 Census: Follow-up Should Reduce Coverage Errors, but Effects on  Demographic Groups Need to Be Determined. GAO-11-154. Washington,  D.C.: December 14, 2010.  2010 Census: Plans for Census Coverage Measurement Are on Track, but  Additional Steps Will Improve Its Usefulness. GAO-10-324. Washington,  D.C.: April 23, 2010.  2010 Census: Data Collection Is Under Way, but Reliability of Key  Information Technology Systems Remains a Risk. GAO-10-567T.  Washington, D.C.: March 25, 2010.  2010 Census: Key Enumeration Activities Are Moving Forward, but  Information Technology Systems Remain a Concern. GAO-10-430T.  Washington, D.C.: February 23, 2010.  2010 Census: Census Bureau Continues to Make Progress in Mitigating  Risks to a Successful Enumeration, but Still Faces Various Challenges.  GAO-10-132T. Washington, D.C.: October 7, 2009.  2010 Census: Census Bureau Should Take Action to Improve the  Credibility and Accuracy of Its Cost Estimate for the Decennial Census.  GAO-08-554. Washington, D.C.: June 16, 2008.  2010 Census: Census at Critical Juncture for Implementing Risk  Reduction Strategies. GAO-08-659T. Washington, D.C.: April 9, 2008.
Information Technology: Significant Problems of Critical Automation  Program Contribute to Risks Facing 2010 Census. GAO-08-550T.  Washington, D.C.: March 5, 2008.
Information Technology: Census Bureau Needs to Improve Its Risk  Management of Decennial Systems. GAO-08-259T. Washington, D.C.:  December 11, 2007.  2010 Census: Census Bureau Has Improved the Local Update of Census  Addresses Program, but Challenges Remain. GAO-07-736. Washington,  D.C.: June 14, 2007.
Information Technology Management: Census Bureau Has Implemented  Many Key Practices, but Additional Actions Are Needed. GAO-05-661.  Washington, D.C.: June 16, 2005.  21st Century Challenges: Reexamining the Base of the Federal  Government. GAO-05-325SP. Washington, D.C.: February 1, 2005.
Information Technology Investment Management: A Framework for  Assessing and Improving Process Maturity. GAO-04-394G. Washington,  D.C.: March 1, 2004.
Comptroller General’s Forum, High-Performing Organizations: Metrics,  Means, and Mechanisms for Achieving High Performance in the 21st  Century Public Management Environment. GAO-04-343SP. Washington,  D.C.: February 13, 2004.
Human Capital: Key Principles for Effective Strategic Workforce  Planning. GAO-04-39. Washington, D.C.: December 11, 2003.  2010 Census: Cost and Design Issues Need to Be Addressed Soon.  GAO-04-37. Washington, D.C.: January 15, 2004.  2000 Census: Lessons Learned for Planning a More Cost-Effective 2010  Census. GAO-03-40. Washington, D.C.: October  31, 2002.
This is a work of the U.S. government and is not subject to copyright protection in the  United States. The published product may be reproduced and distributed in its entirety  without further permission from GAO. However, because this work may contain  copyrighted images or other material, permission from the copyright holder may be  necessary if you wish to reproduce this material separately.