2020 Census: Census Bureau Can Improve Use of Leading Practices When Choosing Address and Mapping Sources
Published: 2014-10-02T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: A complete and accurate address list is a key building block of a successful census, but developing such a list is costly and labor intensive. For the 2020 Census, the Bureau is reexamining approaches to control cost and maintain accuracy, including approaches to meet its address and mapping needs.
GAO was asked to examine potential private sector roles in 2020 Census address list and map development. GAO (1) evaluated the extent to which the Bureau is considering non-Bureau data source opportunities to meet such 2020 needs, and (2) reviewed the status of the Bureau's plans for meeting its key 2020 address and mapping needs.
GAO compared Bureau documentation to leading practices for planning, management, and scheduling from industry guides for project management, reviewed relevant documentation, and interviewed Bureau officials familiar with decennial census needs and data source decisions.
section_title: What GAO Found
paragraphs: The U.S. Census Bureau (Bureau) is working with stakeholders to identify various data sources to meet its address and mapping needs. For example, the Bureau has worked with state, local, and tribal governments and with commercial vendors to identify potential data sources to augment or verify data collected through its Geographic Support System Initiative (GSS-I) program. GSS-I allows government agencies at all levels to regularly share and continuously update their address lists and road data with the Bureau. Federal internal control standards and Office of Management and Budget guidance on geospatial data suggest that the Bureau should support significant data source decisions in terms of both data cost and quality.
However, the Bureau has inconsistently documented cost and quality support for decisions already made to use address and mapping data from state, tribal, and local governments, other federal agencies, and a commercial vendor. Without a systematic consideration of the quality of the variously sourced data that the Bureau plans to rely on, it cannot ensure that effective choices are being made and that possible data limitations that might affect their use are fully understood. Further, the Bureau did not document management approval in support of its data source decisions at the time that the decisions were made; without such documentation, the Bureau lacks accountability and transparency for future sourcing decisions. The Bureau does not have guidance clearly outlining the need or process for ensuring consideration of cost and quality—primary concerns of the Bureau's reexamination—or documentation of management approval for those data sources selected. By implementing a process for documenting such steps, the Bureau can ensure that data source decisions are transparent to Congress, commercial vendors, and other stakeholders.
The Bureau's approach for meeting its address and mapping needs lacks key elements of effective project management outlined in guidance GAO reviewed. Specifically, while the Bureau prepared planning documents to guide GSS-I, it did not include
clear and measurable performance goals to help it effectively meet its address and mapping needs;
milestones detailed at a level where decisions on GSS-I data sources might be tracked; and
performance measures, data, and reporting to help guide planning and track progress toward filling gaps in the Bureau's data needs.
While the Bureau has taken some positive steps—such as preparing a series of planning documents that provide high-level examples of measurable goals, schedules, and deadlines—the absence of detailed goals, schedules, deadlines, metrics, or data on monitoring progress toward outcomes, as well as the absence of a detailed integrated plan that incorporates these elements, means any limitations of the GSS-I strategy may not be fully known or apparent until late in the decade. Without these elements, it will be difficult for the Bureau to ensure that it is adequately evaluating the costs and benefits of alternative data sources, measuring and reporting its progress, or holding managers accountable for results.
section_title: What GAO Recommends
paragraphs: GAO recommends the Bureau implement processes for reviewing the cost and quality of data source selections and for documenting support for those decisions; document management approval of key data source decisions; and—for remaining data source decisions—develop a detailed plan with measurable goals, track performance against these goals, and set a timeline.
The Department of Commerce generally agreed with GAO's findings and recommendations.

section_title: Letter
paragraphs: The U.S. Census Bureau (Bureau) relies on a complete and accurate  address list, along with precise maps, as the fundamental building blocks  of a successful census. An accurate address list is critical because it both  identifies all households that are to receive a census questionnaire and  serves as the control mechanism for following up with households that fail  to respond to the initial questionnaire. Precise maps are critical for  counting the population in the proper locations—the basis of  congressional reapportionment, redistricting, and allocations of federal aid  to state and local governments. If the Bureau’s address list and maps are  inaccurate, people are more likely to be missed, counted more than once,  or counted at the wrong location.
Our prior work has shown that developing an accurate address list has  been both labor intensive and costly. For example, in 2000, the Bureau  tried to construct an address list largely from scratch; in addition, for both  the 2000 and 2010 Census, Bureau field staff updated the Bureau’s  address list throughout the nation by going door to door to add missing  housing units. In 2012, we reported that the Bureau’s 2010 door-to-door  effort—known as address canvassing—required 140,000 temporary  workers to verify 145 million addresses at a cost of $444 million. (The  Bureau proceeded with full address canvassing in 2010 after concluding  that anything less than door-to-door visits could produce unreliable  results.)
With a life-cycle cost of around $13 billion, the 2010 Census was the most  expensive population count in U.S. history, costing over 50 percent more  than the $8.1 billion 2000 Census (in constant 2010 dollars). To manage  tradeoffs between controlling costs and maintaining accuracy, the Bureau  is examining several approaches for the 2020 Census. By September  2015, the Bureau plans to announce its preliminary design decision— including the approaches it expects to use to meet its address listing and  mapping needs. In addition, the Bureau is currently researching how to  reduce the amount of fieldwork and the cost of address and map updates.  As part of this effort, the Bureau is researching plans to reduce the scope  of its door-to-door address canvassing by targeting areas that it believes  have experienced change and for which the Bureau has no other address  and geospatial data sources capturing those changes. Moving forward,  the Bureau is considering what role, if any, the private sector could play in  address and map building for 2020.
In light of the Bureau’s use of information from private sector vendors,  federal agencies, and other entities to meet its address and mapping  needs, you asked us to examine the potential private sector role in  address list and map development for the 2020 Census. Specifically, we  reviewed (1) the extent to which the Bureau is considering non-Bureau  sources of data to meet its key address and mapping needs for the 2020  Census, and (2) the status of the Bureau’s plans for meeting those needs,  paying particular attention to leading practices for project management.
To meet the first objective, we reviewed documents regarding data  sources the Bureau considered for meeting key address and mapping  needs, and determined the extent to which the Bureau documented the  support we identified as necessary for significant data source decisions.  To meet the second objective, we reviewed the Bureau’s efforts in  developing its address and mapping needs for 2020 Census operations  and in preparing to make its data sourcing decisions. We compared  Bureau documentation to leading project planning and management  practices related to results-oriented strategic planning and reporting that  we identified in our previous work and from other sources. For both  objectives, we interviewed Bureau officials familiar with decennial census  needs and data source decisions. For more details on our scope and  methodology, see appendix I.
We conducted our work from February 2014 to September 2014 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: Because of rising costs, a fundamental reexamination of how the Bureau  conducts the decennial census—including improving the cost- effectiveness of the actual count, or enumeration—is needed. In response  to external pressures to contain costs, including our 2010 report calling  for such a reexamination, the Bureau is researching and testing  innovations and improvements (as necessary) in an effort to conduct the  2020 Census at a lower cost per housing unit than the cost estimate of  the 2010 Census, while still maintaining high quality. (The 2010 Census  cost estimate was approximately $94 per housing unit, in constant 2010  dollars.)
Census costs have risen over the years: the cost of the 2010 Census  represents a 38 percent increase in the cost per housing unit over costs  for the 2000 Census; this in turn was a 76 percent increase over 1990  Census costs. According to Bureau officials, without substantial and bold  innovation, the cost of conducting the 2020 Census likely will continue  this trend, and may become prohibitive. (Figure 1 illustrates the increase  in cost per housing unit from 1970 through 2010.)
According to the Bureau’s 2020 Census Business Plan, the rising costs of  the 2010 Census were largely driven by several factors, including  substantial investments in a major national update of its address list  during 2009, just prior to the enumeration in 2010. The address list— referred to by the Bureau as its Master Address File (MAF)—is a data file  that contains a list of all known living quarters in the United States and  Puerto Rico. Since 2000, the Bureau has used addresses provided by the  U.S. Postal Service (USPS) Delivery Sequence File (DSF) as a starting  point to update the MAF. The Bureau uses the MAF to support the  decennial census as well as the American Community Survey and other  ongoing demographic surveys.maintains its Topologically Integrated Geographic Encoding and  Referencing system (TIGER): this system contains spatial geographical  information that associates MAF address data with TIGER geography  data on the Bureau’s maps.
In conjunction with the MAF, the Bureau  To determine whether and how to reengineer address canvassing for the  2020 Census, the Bureau is conducting ongoing efforts to improve its  map and address databases. For example, the Bureau’s Geography  Division is working with USPS, other federal agencies, and state, local,  and tribal governments on a new program called the Geographic Support  System Initiative (GSS-I). This initiative allows government agencies at all  levels to regularly share and continuously update their address lists and  road data with the Bureau. According to Bureau documents, GSS-I is  relying on partnering with federal, state, tribal, and local government  entities—as well as the private sector—to meet two Census-related data  needs: obtaining accurate, complete, and timely information about where  people live (such as address data), coordinates of residential  structures, and other map features (such as street centerlines), and detecting information changes, so that the Bureau can identify such  things as new roads and structures and update the MAF/TIGER  database in response.
According to the Bureau’s current plans, state, local, and tribal  governments (which maintain address lists for purposes such as  emergency response and property assessment) would have the  opportunity to share addresses with the Bureau throughout the decade,  rather than only during the 2 years prior to the census, as was done for  the 2010 Census.
One of the Bureau’s efforts to improve its address and mapping database  involves two projects within the Bureau’s 2020 Research and Testing  Program. These projects are using modeling to predict where changes  (e.g., address additions and deletions) are likely to occur in the MAF.
These models may be used to identify areas where update activities are  required to assure the MAF is as complete and accurate as possible. If  the Bureau decides to limit its address canvassing, this information is  intended to help determine which areas meet acceptable quality  thresholds for address coverage, as well as to identify areas in which  address canvassing would be more effective in assuring a complete and  accurate address list.
In another, related effort—also integrated with the 2020 Research and  Testing Program—the Bureau’s Geography Division has a team in place  to interactively review address and mapping information (including  imagery and other source materials) in order to identify areas in which  counts of addresses in the MAF are consistent with numbers of housing  units on the ground, as well as to identify areas in need of updating.  Bureau officials said that they are planning to test the modeling projects in  2014 and 2015, and to compare them to the interactive review effort, in  order to establish evidence of what mix of modeling and imagery-based  reviews might best identify areas most in need of updating. Details on  how the comparison will be made and tested are not yet available for our  review.
Additionally, in a test beginning in September 2014, the Bureau will  conduct a Partial Block Canvassing Test—a component of their larger  Address Validation Test. For this test, Census staff will canvass in areas  even smaller than the usual blocks of geography used for canvassing,  according to Bureau officials. This is being done under the belief that if  the Bureau can demonstrate operational success at canvassing such  small areas—what it refers to as “partial block canvassing”—it may be  able to reengineer its canvassing operation by targeting efforts to similarly  small areas. Doing so would eliminate the expense of canvassing an  entire geographic block when only a part of it is in need of update. In  another effort, the Bureau is investigating the role and possible  contributions the private sector can make in improving its address and  mapping databases. According to Bureau officials, reliance on the private  sector is necessary in order to maintain the major upgrades that were  made to its address and mapping databases for the last decennial.
As we have previously reported, it is important for the Bureau to remain  on schedule to keep downstream activities on track. As part of its 2020  Census schedule development, the Bureau divided the 14-year life cycle  of the 2020 Census into five phases. The life cycle began in fiscal year  2009 with the Options Analysis phase. The second phase, Early  Research and Testing, comprises work being done through the Bureau’s  GSS-I program and the 2020 Research and Testing Program. This work  is intended, in part, to explore how MAF/TIGER updates could be  modified to control costs or improve quality. Figure 2 illustrates the  sequencing of the five 2020 Census phases.
section_title: The Bureau Is Considering a Variety of Sources to Meet Key Address and Mapping Needs, but Has Not Systematically Documented Its Decisions
subsections: section_title: The Bureau Has Begun Making Decisions About Data Sources to Meet Key Address and Mapping Needs
paragraphs: The Bureau is taking a number of actions to help it develop a better  understanding of the different sources available for updating address and  mapping data and to better position it for cost reduction opportunities in  data acquisition while increasing the quality of the MAF/TIGER database.  According to Bureau officials, while more significant decisions on data  sources remain, they have decided to initially use the following data  sources to help improve address and mapping information in the 2020  Census MAF/TIGER database:  sufficiently reliable address and geospatial data through GSS-I,  provided by state, local, and tribal governments with active address  and mapping efforts underway; address, aerial, and spatial data from other federal agencies; and  imagery via a commercial source.
GSS-I partnerships. As the Bureau continuously updates and maintains  data in the MAF/TIGER database, Bureau officials have decided that  state, local, and tribal governments that have data of sufficient quality— and that participate in GSS-I partnerships—will be the primary source of  address and geospatial data in their geographic areas. For governments  whose data passes a series of Bureau content and quality checks, the  GSS-I partnership program data will be collected throughout the decade.  According to current plans, state, local, and tribal governments that  reliably maintain address lists (for purposes such as emergency response  and property assessment) would be invited to share addresses with the  Bureau throughout the decade, rather than only during the 2 years prior to  the census, as was done for the 2010 Census.
Federal agencies. Bureau officials have decided that some address,  aerial, and spatial data will be collected from other federal agencies. Thus  far, Bureau officials indicated that the federal agency data being used to  update the MAF/TIGER address and spatial data for 2020 includes  address data from the USPS Delivery Sequence File, and satellite  imagery provided by the U.S. Department of Agriculture’s (USDA)  National Agriculture Imagery Program and the National Geospatial- Intelligence Agency (NGA). These address, aerial, and spatial data are  used to identify areas with growth or reduction in the number of housing  units.
Commercial sources. Bureau officials have decided to use a commercial  imagery service, giving the Bureau the capacity to store and manage  imagery it has already collected from other sources—such as local  governments—and the ability to use imagery provided from the  commercial vendor for areas where the Bureau lacks imagery data. The  Bureau is exploring other options as well, including in-house use of  imagery from other federal agencies or from additional commercial  vendors.
Additional data sources. In addition to these data source decisions,  Bureau officials have been conducting ongoing research and outreach  with a variety of entities to identify additional data source options the  Bureau could use to meet its address and mapping needs. Table 1  describes data sources the Bureau has already identified that might  possibly meet its address and mapping needs.
As an example of work with entities outside the Bureau, during fiscal year  2011, the Bureau invited officials from USPS and the U.S. Geological  Survey to participate in GSS-I research and development working groups  to help identify sources for address and feature updates for the  MAF/TIGER database. In September 2011, the Bureau hosted a meeting  with state and local governments, nonprofit organizations, and various  national associations to discuss techniques for address list development  and maintenance, as well as potential pilot programs for data sharing  among these groups.
The Bureau has also taken steps to work with commercial vendors to  review potential sources of commercial data that the Bureau could use to  augment or verify data collected through GSS-I. For example, in July  2013, the Bureau completed a market research project with one vendor to  evaluate the prospects for using commercially available source data in  the event a future need arose for supplemental data from areas of the  country where local, state, or tribal governments lack spatial data to  provide to the Bureau. During the project, the Bureau obtained  information on quality assurance and quality control processes it can use  to maintain accuracy of spatial data within the MAF/TIGER database.  Additionally, in July 2014, the Census Bureau issued a request for  information conducting market research to identify available sources for  street centerline data and address data. The solicitation stated that the  Bureau was engaging industry in order to identify potential vendors and  their ability to provide spatially accurate road networks, addresses, and  other geospatial data to supplement the information already available in  the MAF/TIGER database. As of August 2014, Bureau officials were  continuing to visit vendors, attend conferences with geospatial industry  leaders to learn about their address and mapping technologies, and  interact with Census Scientific Advisory Committee and National  Academy of Sciences panels.
section_title: The Bureau Has Not Systematically Documented Its Decisions on Data Sources
paragraphs: Officials at the Bureau explained the rationale for the data source  decisions that have been made thus far and provided us with background  documents and testimonial evidence about each of the decisions.  However, they provided inconsistent support for the decisions  themselves. Federal internal control standards and Office of  Management and Budget guidance on geospatial dataBureau should be supporting its significant data source decisions—such  as for its address and mapping needs—in terms of both cost and quality,  as well as with clear management approval for the decisions. Further,  these documents should be readily available for examination.
Cost considerations. According to leading practices for data sourcing,  decisions should be based on consideration of cost. Such considerations  could be documented in a variety of ways, including market research,  minutes of meetings with relevant discussion, summaries of data cost  research, transaction costs, or pricing schedules. The decisions on data  sources the Bureau has made thus far—using local, tribal, and state  governments; other federal agencies; and a commercial imagery server— involved acquiring free data. For each of those decisions there is no  additional charge to the Bureau for obtaining data from the respective  source, largely obviating the need to justify cost. For instance, the Bureau  established a memorandum of understanding with state, local, and tribal  governments participating in GSS-I, indicating that no funds will be  exchanged between the Bureau and participants for sharing address and  map data. Bureau officials told us that getting such updates at no charge  is a key to reducing expensive broader canvassing costs later in the  census cycle. The Bureau’s recent congressional budget justification  documents echo this argument as well. Additionally, Bureau officials have  stated that they are able to obtain data from other federal agencies at no  cost and can rely on a vendor-provided imagery service that the Bureau  already had access to under other paid licensing arrangements. No  additional charges are associated with such access.
Moving forward, by improving its use of leading practices to determine the  relative cost-effectiveness of using data sources in updating the  MAF/TIGER database, the Bureau can better ensure sufficient  consideration of costs. When deciding on each additional source,  consideration includes indirect costs, such as the incremental costs of  data processing or quality assurance.
Quality considerations. According to leading practices for data sourcing,  decisions should be based on consideration of data quality, including data  accuracy, completeness, and timeliness. Such considerations can be  documented by market research, minutes of meetings with relevant  discussion, summaries of data quality research, relevant test or  evaluation results, schedules of data updates and availability, reporting  on quality measures, and evidence of successful historical use of the  data. Thus far, the Bureau has partially supported consideration of quality  across its data sourcing decisions. For example, its decision to rely on  address and mapping data submitted from state, tribal, and local  governments was based on expectations that many of the governments  would be able to provide data of sufficient quality. Bureau officials stated  they are relying on procedures to assess the content of submitted data  files on a case-by-case basis. Bureau timelines indicate the quality  reviews will likely extend over many years, and Bureau officials do not  know for sure how many (or which) government sources will be good  enough to use. For their decision to rely on USPS, USDA, and NGA for  address and mapping data, Bureau officials provided evidence that they  reviewed research on the quality of data sources from each of the  agencies. For example, since the 2000 Census, the Bureau has  successfully relied on USPS address data, and—leading up to the 2010  Census—on USDA aerial imagery as well. Over the years, several  evaluations have helped document limitations of the USPS data in  particular, so that the Bureau targets its use of them. In addition, the  Bureau documented a review of how the quality of NGA imagery sources  can help meet Bureau needs.
The Bureau summarized detailed market research for several alternative  sources of imagery from commercial vendors and other federal agencies  related to the positional accuracy and current geographic coverage of the  imagery data and the rate at which imagery data were updated. However,  it is not clear how the Bureau used that information or whether the quality  of the imagery data source exceeded quality standards (or had  limitations) compared to others. Bureau officials stated that they selected  the commercial imagery service during meetings assessing and  comparing options, because it met their needs and its use was included in  other licensing arrangements that had already been paid for. However,  Bureau officials could provide no contemporaneous documentation of the  results of those comparative assessments. By having a systematic  process to consider the quality of the data that the Bureau relies on from  various sources, the Bureau can help ensure that effective choices are  being made and that possible limitations in data that might affect their use  are better understood.
Documenting decisions. Internal control standards require  documentation of significant management decisions, which can be  documented with decision memorandums, memorandums of  understanding, or meeting minutes indicating acceptance of data source  recommendations, or other documentation of senior management  approval for deciding upon a specific data source. We found that at the  time decisions were made, the Bureau did not document management  approval in support of sourcing decisions for any of the data sources  discussed above. Bureau officials provided extensive documentation on  GSS-I, which indicated that governments across the country would be  playing a major role as partners with the Bureau by providing data:  however, there was no formal decision documenting (as an accountability  check) that senior Bureau leadership had agreed that the net benefits of  these sources are greater than alternative data sources. Regarding the  decision to rely on several federal agencies for various types of  geospatial data, the Bureau similarly lacks a formal record establishing  the Bureau’s approval of these sources as official inputs to the 2020  Census. The extensive public interactions documented between Bureau  and other agency officials leaves little doubt that the decision to rely on  data from these other federal agencies is known and acceptable to senior  Bureau officials, yet evidentiary support for the Bureau’s decision to rely  on some agencies and not other sources is absent. For the decision to  rely on a commercial imagery source, Bureau officials told us that the  chief of the Geography Division approved the use of commercial imagery  server software (to host freely available imagery acquired from federal,  state, and local sources) during a January 2012 meeting, although no  record of the meeting was produced and there was no separate  documentation of the decision. Developing evidence of management  approval for data sourcing decisions can help ensure that as the Bureau  moves forward, stakeholders—such as Congress and commercial  vendors—have greater transparency regarding the data sources being  considered to meet the Bureau’s key address and mapping needs, how  decisions are being made, who made them, and on what basis. Such  evidence would enhance the accountability of senior Bureau officials for  decisions at the time those decisions are made.
Cost and quality are two key traits linked to the goals and objectives of  the Bureau’s agency-wide strategic plan and feature prominently in other  broad strategic documents for the 2020 Census. Yet we found the Bureau  does not have guidance outlining the need or the process for ensuring (1)  systematic justification for decisions related to using specific data sources  in terms of cost, quality, or other important considerations, and (2)  documentation of management approval for such decisions. Although the  Bureau’s initial data sourcing decisions involved acquiring data at little  cost to the Bureau, according to Bureau officials, they have much more to  resolve about meeting their address and mapping data needs, with future  decisions potentially involving other stakeholders and significant cost. By  developing more rigorous support and evidence of such decisions, the  Bureau can ensure transparency to Congress, commercial vendors, and  other stakeholders. In turn, these efforts could lead to increased  stakeholder support for the Bureau’s plans for the 2020 Census and could  enable the Bureau to consistently support why the data sources it  selected are better than alternatives. Rigor in decision making can also  help reduce the Bureau’s risk of failing to select the most cost-effective,  accurate, complete, and timely address and mapping data for updating  the MAF/TIGER database.
section_title: The Bureau Has Not Fully Applied Leading Planning Practices to Identify What Other Address and Mapping Data Sources It Needs for 2020
paragraphs: The Bureau’s efforts to design a cost-effective enumeration (starting with  complete and accurate address and mapping data) present a significant  project management challenge, one that demands meticulous planning.  However, we found that the Bureau’s approach to meeting its address  and mapping needs is missing key elements that comprise a rigorous,  integrated plan, including clear and measurable goals, decision  milestones at a level where decisions on GSS-I data sources might be  tracked, and performance data that management could use to track  progress. Without consolidating these elements in a plan that lays out  how Bureau officials should monitor progress, it will be difficult for Bureau  management (and others) to know that the Bureau is on track to meet its  address and mapping needs; it will also be difficult to pinpoint  improvement opportunities.
To identify key elements of effective project management, we reviewed a  number of guides for project management and business process  reengineering.planning, we found that the guides contained many elements in common,  including the following:   Although there is no one best approach to project  the project plan should consider all phases of the project and should  have clear and measurable goals, or targets;   schedules, milestones, and deadlines should be clearly stated; and performance data should be gathered and reported to determine and  monitor progress toward goals.
Additionally, OMB Circular A-11 specifies that an agency’s general goals  should be sufficiently precise to direct and guide agency staff in actions  that carry out the agency’s mission and aid the agency in developing  annual performance goals. Without these elements of effective project  management, the Bureau cannot ensure it will make informed decisions  about its data needs in a timely manner.
Measurable goals. As discussed earlier, to help address its data needs,  the Bureau decided to rely on state, local, and tribal government address  and geospatial data that met its quality standards. However, Bureau  officials have not set measurable goals for that effort. For example, the  chief of the Geography Division estimated that these government sources  may be able to provide data coverage for about two-thirds of the over  3,200 counties in the country.provide any detailed support for that estimate, and said that they  recognize the need for better information in order to make decisions on  these and remaining needs, and are working to gather it. Knowing what  percentage of the country—and perhaps, what different regions and  different types of geographies, such as rural and urban—may be covered  by sufficiently reliable sources would help the Bureau (and others) better  know both the nature and magnitude of the remaining data gaps. Whether  in terms of estimated numbers of addresses, numbers of housing units, or  in terms of area, a measurable goal—such as the amount of address and  mapping data expected to be sufficiently reliable—could help inform  estimates of the level of effort needed to achieve goals: in turn, this can  help inform decisions about resource allocations and other elements of  project planning. Measurable goals can also help improve communication  with other stakeholders who might be able to help the Bureau fill the data  gap. Bureau officials acknowledge that they need to determine and report  more precisely the extent of the coverage gaps in addresses and  mapping data obtained from reliable government sources.
However, Bureau officials could not  Similarly, the Bureau’s strategic plan for GSS-I includes high-level goals.  However, the plan, its related management plan, and other planning  documents lack the specificity needed regarding clear and measurable  goals or targets representing the address and mapping results GSS-I  needs to achieve. Such goals or targets can help Bureau staff ensure  they are effectively taking steps toward meeting address and mapping  needs and might help in the identification of helpful sources. For example,  the Bureau’s GSS-I Strategic Plan has as one of its goals that the  Geography Division be efficient, effective, and adaptable, with a strategic  objective of efficient and effective source data acquisition. The plan  goes on to list potentially useful strategies, such as expanding the use of  available imagery for source data assessments and embracing the most  efficient data acquisition methods using viable technologies. However, the  GSS-I planning documents do not describe measurable goals or targets  (such as the estimated numbers of addresses) expected or needed from  state, local, and tribal governments under GSS-I for the efforts to be  successful, or to have met stated needs, goals, or related objectives. In  addition, the documents do not identify what coverage of the housing in  any given area is sufficient to meet the Bureau’s needs.
The Bureau’s GSS-I Strategic Plan includes one example of the type of  detailed, measurable goal that could better guide planning and decision  making. That data need, or target, is described as “maintain a standard of  7.6 meters or better positional accuracy for existing streets and newly  acquired streets.” However, this specific information is the only  occurrence in the document with this level of detail. Bureau documents  elsewhere provide examples of what could serve as measurable goals, if  they were formally stated as quality standards needed from all potential  sources of address data. For example, a description of the Bureau’s  process for reviewing government partner data files describes  assessment of, among other characteristics, whether addresses include  each of several different required elements and whether road centerline  data are updated within 2 years; however, it does not describe measures  of how complete the data must be to meet the Bureau’s needs. A detailed  integrated plan that includes complete descriptions of what the Bureau  deems sufficient quality data for its address and mapping needs can help  focus efforts of the Bureau and its private and public partners on  satisfying them.
Decision milestones. Bureau officials provided a high level master  activity schedule through the 2020 Census that includes some decision  points, such as go/no go decisions for whether it will reengineer its  address canvassing operation (limiting it to only targeted areas) and the  deadline for creating and finalizing the address canvassing workload.  Additionally, officials provided a version with much more detail through  September 2015, when the Bureau plans to announce its preliminary  design decisions. However, Bureau staff indicated that they have not yet  refined the schedule at the lower level beyond September 2015, such as  at a level where decisions about GSS-I data sources might be tracked.  Bureau officials also provided a schedule that describes in detail the  implementation of a workflow management system used to track the  handling of address and other files received from partners, but the system  was implemented in 2013 and does not include any future milestones.  Whether immediately included in the master activity schedule or not,  without schedules moving toward near term goals and decisions included  in a more rigorous plan, the Bureau and Congress cannot determine  whether the Bureau is on track to meet its key address and mapping  needs for the 2020 Census.
Specifically, while work is under way in these areas, we found that the  Bureau does not have milestones or deadlines for remaining decisions in  the following areas:   to what extent state, tribal, and local governments can be relied on as  acceptably reliable data sources; how the Bureau will obtain address, boundary, and feature data for  areas not covered by state, tribal, and local governments; and how good its various data sources are or need to be so as to be  acceptably reliable. That is, address data obtained from the sources  must meet minimum standards for timeliness, accuracy, and  completeness—sufficient coverage of the nation’s housing.
Bureau officials indicated that they do not yet have milestones such as  these, because, among other reasons, those decisions need to be  informed by the 2014 Census Test, occurring in summer 2014, and the  results of other tests in 2015. While this creates some uncertainty  regarding what the future milestone dates might need to be, without at  least preliminary milestones pending the results of census tests, the  Bureau may not have sufficient time left to complete remaining activities,  especially given the fact that some of the activities may require long lead  times for completion (see figure 3). For example, it may take significant  time to research and select which data sources are needed, depending  on how the Bureau decides to address them. The Bureau has separately  set a deadline of December 31, 2016, for finishing the development and  awarding of major contracts for systems that will support the 2020  Census, recognizing the lead time as necessary. Furthermore, the  Bureau’s 2020 Lifecycle Risk Register states that, “many of the decisions  on the final 2020 Census design may not be finalized until late in the  research time frame, which could include decisions to outsource some of  the development effort. However, acquisitions require established lead  times that include set processes and review milestones, both at the  agency and at the department level. If 2020 Census design decision  milestones do not allow the requisite lead times for acquisition processes  and reviews, then the Census Bureau may not be able to procure the  necessary products and services in sufficient time to align with the 2020  Census development life cycle.” The Bureau identified this risk and rated  it as medium on its scale of low to high, underscoring that it values  managing to a timeline of key milestones and deadlines.
Monitoring progress. Bureau officials could not provide us with the  performance reports they are using to monitor progress toward ensuring  that key 2020 address and mapping needs will be met. However, in  August 2014, Bureau officials provided us with copies of presentations  made to external groups reporting on progress made in reviewing data  from state, tribal, and local government partners under GSS-I. For  example, one presentation indicated that as of February 17, 2014, as part  of the GSS-I effort, the Bureau had contacted 375 partners, 247 of which  had provided files. A Bureau report provided to the Office of Management  and Budget on August 4, 2014, justifies resource requests for GSS-I and  contains performance metrics, but the metrics on address data from state,  local, and tribal governments are in terms of numbers of partner files  acquired and processed in 2013 and 2014. There is no context given,  such as how many potential partners there are, how many governments  the Bureau anticipates participating or needs to participate, what level of  participation the Bureau seeks to obtain year-by-year, or how complete  the coverage of addresses needs to be. Without such context, these  numbers do not provide complete information on the extent to which the  Bureau is making progress. Furthermore, the metrics the Bureau provided  do not address the extent to which the MAF/TIGER data are being  updated or improved, such as in numbers of addresses, housing unit  structures, linear miles of roads, or geographic square areas. If fewer  than expected (or needed) state, tribal, and local partners are agreeing to  participate, if less data are being updated than expected, or if the  resulting database is not complete enough, then management needs to  know in real time so that it can prioritize efforts to access other data  sources that may be needed to meet its key address and mapping needs.  By developing more accurate and timely documentation of progress on  obtaining or updating address and mapping data, the Bureau can also  better illustrate to stakeholders that it is effectively managing its data  source decisions.
Our previous work has found that the Bureau has inconsistently followed  key planning practices: for example, in 2012, we found that the Bureau’s  high-level schedule for the 2020 Census did not include milestones or  deadlines for key decisions needed to support the transition between the  planning phases for 2020. While the Bureau has taken some positive  that provide  steps, such as preparing a series of planning documentshigh-level examples of measurable goals, schedules, and deadlines, it  has not put in place all elements integral to large complex projects. The  absence of detailed goals, schedules, deadlines, metrics, or data on  monitoring progress toward outcomes and the absence of a detailed  integrated plan that incorporates these elements means any limitations of  the GSS-I strategy may not be fully known or apparent until late in the  decade. Without these elements, it will be difficult for the Bureau to  ensure that it evaluates the costs and benefits of alternative data sources  and measures and reports the Bureau’s progress. It will also be difficult to  hold managers accountable for results. Additionally, the Bureau may be  lacking the information necessary to make its remaining 2020 address  and mapping decisions. As a result, the Bureau is at risk of experiencing  increased costs to obtain data for remaining gaps in address and  mapping data.
section_title: Conclusions
paragraphs: Bureau efforts to update its MAF/TIGER database with accurate,  complete, and timely address and mapping data are critical for carrying  out a precise population count in a fiscally constrained environment. The  Bureau has taken initial steps to plan and implement data collection  efforts for meeting its key address and mapping needs, such as  developing planning documents for its GSS-I program and identifying a  series of potential data sources in coordination with internal and external  stakeholders. Thus far, these efforts have enabled the Bureau to consider  an array of government and commercial sources for updating its address  and mapping data through GSS-I as it has made its initial data source  decisions.
Continued Bureau efforts to prepare for remaining data source decisions  would benefit from more systematic efforts to document cost and quality  considerations for government and commercial sources. While the  Bureau has decided thus far to use data sources that have little or no  incremental acquisition cost, enhanced cost and quality assessments of  alternative data sources considered for future decisions will position the  Bureau to demonstrate transparency in its decision making to Congress,  to potential commercial vendors, and to other stakeholders. Implementing  processes for supporting data source decisions that meet key address  and mapping needs—particularly for assessments of cost and quality— will reduce the risk that the Bureau selects data sources that are not cost- effective or high quality. In addition, when decisions are being made  about how to meet key address and mapping needs, it is also important  that the Bureau document these decisions, such as through the use of  decision memorandums or minutes of meetings where decisions are  made. Implementing a process that ensures management approval is  documented for key decisions on data sources in the future will help  demonstrate accountability for those decisions.
As the Bureau moves forward in preparing and refining its design  approaches for the 2020 Census, it can take additional steps to improve  planning for data sourcing decisions. Currently, the Bureau does not have  a detailed integrated plan that incorporates measurable goals. Such a  plan would help ensure that the Bureau is collecting sufficient address  and mapping data from its private and public partners. The Bureau has  also not set a timeline regarding when it needs to make remaining data  source decisions: setting a timeline will allow it to reduce the risk of not  having enough time to adequately decide upon alternative data sources  that, if needed, might potentially reduce cost or increase quality.  Additionally, the Bureau has not established a process to monitor and  report on progress in the management of GSS-I, which will enable it to  better identify gaps in its data collection efforts, as well as to ensure and  track the actions needed to fill such gaps in time for the 2020 Census.
section_title: Recommendations for Executive Action
paragraphs: To help ensure that the Bureau more rigorously considers data sources  and remains on schedule to meet its address and mapping needs, the  Secretary of Commerce and Undersecretary of Economic Affairs should  direct the Census Bureau to take the following three actions:  In order to ensure transparency of future decision making, implement  a process for documenting the support for data source decisions  intended to meet key address and mapping needs and the support for  assessing the cost and quality of data sources the Bureau is  considering.
In order to ensure accountability for key decisions moving forward,  implement a process for documenting management approval of key  address and mapping data source decisions, such as through  decision memorandums or minutes of meetings where decisions  occurred.
In order to better ensure the Bureau meets its address and mapping  needs for 2020 and stays on schedule, develop a detailed integrated plan that includes items such as  measurable goals (e.g., estimated numbers of addresses  expected or needed from state, local, and tribal governments  under GSS-I); schedules and deadlines; and progress monitoring  and reporting, and establish a timeline identifying when remaining data source  decisions need to be made.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this report to the Department of Commerce and  received the department’s written comments on September 23, 2014. The  comments are reprinted in appendix II. The Department of Commerce  generally agreed with our findings and recommendations and provided  technical comments, which we incorporated as appropriate.
We are sending copies of this report to the Secretary of Commerce, the  Under Secretary of Economic Affairs, the Director of the U.S. Census  Bureau, and interested congressional committees. In addition, the report  is available at no charge on the GAO website at http://www.gao.gov.
If you have any questions about this report please contact me at (202)  512-2757 or goldenkoffr@gao.gov. Contact points for our Offices of  Congressional Relations and Public Affairs may be found on the last page  of this report. The GAO staff that made major contributions to this report  are listed in appendix III.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: This report reviewed (1) the extent to which the Bureau is considering  non-Bureau sources of data to meet its key address and mapping needs  for the 2020 Census, and (2) the status of the Bureau’s plans for meeting  those needs, paying particular attention to leading practices for project  management.
To categorize Bureau-identified address and mapping needs, we  obtained Bureau documentation of such needs. Because address and  mapping needs vary in their definition and level of specificity according to  their different use or purpose, they may be dynamic until the Bureau  makes sourcing decisions to meet them and some needs were not yet  fully identified during the time of our review. To address this issue, the  Bureau provided revised lists of key address and mapping needs during  the course of our review. For this report, we include key address and  mapping needs identified as of June 2014.
To review the Bureau’s approaches for developing its address and  mapping needs for 2020 Census operations and preparing it for data  sourcing decisions, we compared the Bureau’s organizational documents  for its approaches–such as strategic, program management, and  operational plans–to elements of project planning we identified from  industry guides for project management and business process  reengineering, as well as other leading management practices we  identified in our prior work on establishing a coherent agency mission and  integrated strategic goals, and on adopting leading practices for results- oriented strategic planning and reporting.
To identify data sources the Bureau considered for meeting key address  and mapping needs, we reviewed documents provided by the Bureau for  specific data sources used to obtain data for addresses, coordinates of  residential structures, and other map features, and specific data sources  used to perform change detection related to identifying new roads and  structures for updating the MAF/TIGER database. We did not seek to  identify an exhaustive list of data sources; rather, we identified those  readily attributable to documents (1) that the Bureau created to analyze  potential data sources or to explain its consideration of such sources, or  (2) that the Bureau solicited from commercial vendors and other sources,  even if such documents were more promotional than informative related  to the Bureau’s needs. We did not consider unsolicited documents sent to  the Bureau from commercial vendors proposing the use or adoption of  particular address or mapping data sources or solutions.
To review the extent to which the Bureau supported its decisions to use  data sources to meet its key address and mapping needs, we obtained a  Bureau-provided list of data sources it selected, and documentary and  testimonial evidence the Bureau identified as justifying its decision for  each source. We also reviewed our prior work on internal controls and  Office of Management and Budget guidance on geospatial data, and  determined the Bureau should be supporting its significant data source  decisions—such as for its address and mapping needs—in terms of cost,  quality, and clear management approval of decisions.
For both objectives, we interviewed Bureau officials in the Geography  Division, 2020 Research and Planning Office, and Decennial Systems  and Contracts Management Office to discuss planning and decision- making efforts for meeting key address and mapping needs for 2020  Census operations.
We conducted this performance audit from February 2014 to September  2014 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
section_title: Appendix II: Comments from the Department of Commerce
section_title: Appendix III: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: Key contributors to this report include Ty Mitchell, Assistant Director; Tom  Beall; Rob Gebhart; Andrea Levine; Mark Ryan; and Timothy Wexler.