2020 Census: Bureau Generally Followed Its Plan for In-Field Address Canvassing
Released: 2020-03-12T00:00:00

Letter

Background

The Bureau’s address canvassing operation updates its address list and maps, which are the foundation of the decennial census. An accurate address list both identifies all living quarters that are to receive a notice by mail to respond to the census, and serves as the control mechanism for following up with households that fail to respond to the initial request. Precise maps are critical for counting the population in the proper locations—the basis of congressional apportionment and redistricting.

Our prior work has shown that developing an accurate address list is challenging—in part because people can reside in unconventional dwellings, such as converted garages, basements, and other forms of “hidden” housing. For example, as shown in figure 1, what appears to be a single-family house could contain an apartment, as suggested by its two doorbells.

During address canvassing, the Bureau verifies that its master address file and maps are accurate to ensure the tabulation for all housing units and group quarters is correct. For the 2010 Census, the address canvassing operation mobilized almost 150,000 field workers to canvass almost every street in the United States and Puerto Rico to update the Bureau’s address list and map data—and in 2012 reported the cost at nearly $450 million. The cost of going door to door in 2010, along with the emerging availability of imagery data, led the Bureau to explore an approach for 2020 address canvassing that would allow for fewer boots on the ground.

To reduce costs for the 2020 Census, the Bureau took a new approach and some address canvassing work was completed in-office. The Bureau compared current satellite imagery to the contents of its master address file to determine if areas had housing changes, such as new residential developments or repurposed structures. If the satellite imagery and the master address file matched, then the Bureau considered those areas to be resolved or stable and did not canvass them in-field.

These areas that were unresolved by the in-office review were sent to in- field address canvassing. Field staff called listers used laptop computers to compare what they saw on the ground to the address list and maps. Listers confirmed, added, and deleted addresses or moved addresses to their correct map positions. The listers were trained to speak with a knowledgeable resident at each housing unit to confirm or update address data, ask about additional units, confirm the housing unit location on the map (known as the map spot), and collect a map spot either using global positioning systems (GPS) or manually. If no one was available, listers were to use house numbers and street signs to verify the address data. The data were then transmitted electronically to the Bureau.

The Bureau Completed In-Field Address Canvassing on Schedule and under Budget, but Listers Did Not Always Follow Procedures

Productivity Was Higher Than Expected

The Bureau completed in-field address canvassing on time despite nationwide hiring shortfalls. The Bureau credits this success to better- than-expected productivity. The Bureau conducted “in-field” address canvassing for approximately 35 percent of the housing units (approximately 50 million housing units) across the country (see fig. 2). The Bureau had already determined “in-office” that the other 65 percent of addresses (approximately 93 million housing units) were part of stable blocks.

The Bureau began the in-field address canvassing operation at seven of its 39 Area Census Offices on August 4, 2019, and then rolled out the operation to the remaining 32 offices on August 18, 2019. It conducted this phased approach to ensure all operations and systems worked together before commencing the operation nationwide. The total in-field address listing workload was more than 50 million addresses from the Bureau’s address file. Bureau officials reported that listers were generally more productive than expected, thus allowing the Bureau to complete the operation as scheduled on October 11, 2019 (see fig. 3).

The actual hourly productivity rate for the operation was 19.8 addresses versus the anticipated rate of 15.8 addresses. According to Bureau officials, listers were more productive due to efficiency gains from the Bureau’s new approach, including an automated time and attendance system, the use of computer laptops to collect census data, and a new operational control system that was used to electronically optimize assignments and transmit work to listers. Bureau officials stated that the high productivity also helped the operation come in under budget. The operation’s cost was $118.6 million—while the anticipated cost was $185 million—a reduction of 36 percent.

The Bureau Missed Potential Opportunities to Improve the Address List When Listers Did Not Follow Procedures

For in-field address canvassing, listers received online training, which detailed the procedures they were to follow, such as: comparing the housing units they see on the ground to the housing units on the address list, knocking on all doors so they could speak with a resident to confirm the address (even if the address is visible on the mailbox or house) and to confirm that there are no other living quarters such as a basement apartment, looking for hidden housing units, and confirming the location of the housing unit on a map with GPS coordinates collected on the doorstep.

In our observations of in-field address canvassing, the majority of listers generally followed these procedures. However, some listers we observed did not always follow procedures. For example,

Ten out of 59 listers did not work ground to book (i.e., compare what they saw on the ground to what was on their list).

Nine out of 59 listers did not walk up to the doorstep to collect the GPS coordinate. Specifically, we observed listers use mailboxes to confirm address information and collect the GPS coordinates from the mailbox. Following proper procedures is important because getting a GPS reading from the doorstep of every address contributes to the accuracy of the address file.

Fourteen of 59 listers did not consistently knock on every door as required to confirm the address and ask about “hidden” housing units.

Seventeen of 59 listers did not always look for or ask about “hidden” housing units. Not knocking on doors or asking about hidden housing units represents missed opportunities to potentially add missing addresses to the Bureau’s master address file.

Further, not all listers we observed provided the required confidentiality notices to occupants. Seven listers we observed did not provide confidentiality notices. Occupants may be more willing to provide their information if they know their responses will not be shared. We communicated the information regarding our observations to the Bureau, and on August 26, 2019, the Bureau instructed its field offices to remind listers of the appropriate procedures.

According to Bureau officials, some amount of temporary staff deviates from following procedures with every decennial census. As such, to control for this, the Bureau implemented a Quality Control (QC) component for in-field address canvassing that is designed to detect and correct deficient production listers’ work. QC started on August 11, 2019, and included a total workload of around 3.4 million addresses. For this operation, an automated system selected the sample of addresses to review; these addresses were assigned to QC listers. QC listers received instructions to begin canvassing at a specified location, usually an intersection, and to continue canvassing addresses until the system identified the work unit as “complete” for QC purposes.

An address worked by a production lister was considered to have “failed” QC if the QC lister recorded changes, or if the lister missed the address and the QC lister found it. Depending on the size of the block, after a predetermined number of addresses fail within a block, the system fails the entire block. Once a block fails, the QC lister must recanvas all the addresses in that block. Based on preliminary results, Bureau officials estimate that 4.3 percent, or about 2.2 million addresses, failed. According to Bureau officials, while they did not have a predetermined target for what was an acceptable range for the total number of addresses that failed QC, they nevertheless are reasonably confident that this was in an acceptable range for QC errors encountered during the operation. They further stated that they could not compare 2020 QC results to 2010 because the 2010 Address Canvassing Operation canvassed 100 percent of the addresses in-field, while the 2020 In-field Address Canvassing Operation only covered approximately 35 percent of the addresses across the country.

Lister productivity for QC was also higher than expected. The Bureau anticipated the QC productivity at 8.03 addresses per hour compared to the actual rate of 14.05 addresses per hour. Higher-than-expected productivity rates contributed to a reduction in costs and the actual cost of QC production was $10.3 million versus the anticipated cost of $25.6 million, a savings of $15.3 million. Additionally, Bureau officials stated that QC came in so far under budget because the use of laptops increased efficiency and the actual QC workload was lower than the budget estimate.

Planned Evaluations Will Ultimately Determine the Quality of the Operation

While the Bureau conducted real-time quality control follow-up of selected blocks during address canvassing, it also has two studies underway that will evaluate the re-engineered address canvassing approach, as well as the in-field address canvassing operation. Similar studies conducted by the Bureau in 2010 found that 95.7 percent of addresses were correctly deleted and 83.6 percent of addresses were correctly added. Both studies underway have a set of research questions designed to evaluate the accuracy and effectiveness of address canvassing. For example the Bureau seeks to answer questions such as:

What percentage of the housing units added during in-field address canvassing were correctly added (and added-in-error)?

What percentage of the housing units identified as deleted or duplicated by the listers during in-field address canvassing were correctly deleted or duplicated (and deleted-in-error)?

Answering these and other questions contained in both studies will be critical to determining the quality of the operation, as not all listers followed procedures, which may have led to errors in the address file. It is anticipated that the final report for the 2020 Census In-Field Address Canvassing Operational Assessment study will be available September 2020, and the 2020 Census Evaluation: Reengineered Address Canvassing study will be available March 2023.

The Bureau Had Successes and Challenges during In- Field Canvassing, Which Have Potential Implications for Future Operations

The Bureau Cited Successes with the Operation

In addition to completing in-field address canvassing on schedule and under budget Bureau officials highlighted other successes from the operation including:

Automated solutions for training staff. Bureau-developed training materials that used a blended training approach including instructor- led, computer-based, and hands-on training. This is a change from the 2010 paper-based and classroom-only training approach.

Efficiency gains from conducting reengineered field operations using:

New operational control systems, which were used to electronically assign and transmit work to the listers.

New automated time and expense reporting (timecards) for employees. In 2010, timecards were paper-based and the listers had to meet with their supervisors to submit them.

Enhanced software application for validating and updating addresses.

Implementation of rapid response to Hurricane Dorian, which affected areas of the Southeastern United States, resulted in minimal disruptions to the operation.

Additionally, the Bureau was able to resolve some unforeseen challenges at the seven Area Census Offices that opened early. For example, the Bureau identified issues with training login and new hires not being on the training roster and rectified those issues before the operation expanded to the rest of the country.

The Bureau Is Taking Steps to Address Challenges with Hiring and Onboarding Staff

The Bureau experienced delays in hiring for its early operations, raising concerns about hiring for peak operations. The Bureau’s target was to hire 40,300 listers by September 7, 2019, but as of September 9, 2019, the Bureau had hired 31,151 listers. Though address canvassing productivity was higher than expected, in some parts of the country the operation was at risk of falling behind because of a shortage of listers. The Bureau told us it filled the gap with listers who lived well outside of the area in which they were supposed to work—in some cases from a different state. This strategy allowed the Bureau to complete the operation on schedule; however, though the operation as a whole was under budget, the Bureau incurred unplanned costs for travel (airfare, personal mileage rates, rental cars, hotel stays, and per diem).

As we previously reported, these hiring problems are an early warning for what may occur later in the census during nonresponse follow-up, when the Bureau intends to hire between 320,000 to 500,000 enumerators to follow up with households that did not initially respond to the census. The Bureau said the hiring issues were caused by delays in processing background checks and greater-than-expected attrition. According to the Bureau, these delays arose, in part, due to early shortages of staff to review background checks and because a significant number of applicants did not completely or accurately fill out related forms. In February 2019, the Bureau began to bring on about 130 temporary staff to review forms for accuracy and completeness prior to submission for investigation and to help investigators conduct the pre-employment background checks.

Those delays in turn contributed to subsequent challenges in onboarding listers for address canvassing. For example, according to Bureau officials, the delays in early hiring for Area Census Office staff meant some offices did not have enough clerks in place to process paperwork for listers or make reminder phone calls to hire and onboard listers.

Regarding attrition, more listers quit than expected at two points in the hiring process:

Fingerprinting: The Bureau expected about 15 percent of applicants would leave the hiring process after being selected and before submitting fingerprints. However, the attrition rate was closer to 25 percent. Bureau officials told us they attributed this to selected applicants, in some cases, having to travel long distances to be fingerprinted.

Training: The Bureau found that fewer selected and cleared applicants attended training than anticipated. Bureau officials attributed this to fewer clerks being available to call trainees with reminders to attend training due to delays in clerks receiving their own background checks.

Bureau officials also attributed some of this attrition to the 60-day period between the selection of applicants and their training. This new time frame was put in place for the 2020 Census to provide adequate time for adjudication of background checks.

The Bureau has begun to address these challenges by adapting its hiring and onboarding processes for peak operations, such as nonresponse follow-up, which is to begin May 2020. For example, the Bureau: Increased the number of fingerprinting locations and machines. According to Bureau officials, it added 133 additional sites and 300 additional machines, bringing the total number of vendor sites for fingerprinting to 829.

Staffed Area Census Offices to help newly-selected applicants for positions complete their forms and initiate the background check process.

Hired additional staff to help clear background checks. The Bureau hired 200 staff at the National Processing Center and an additional 150 at the Regional Census Centers.

Changed the recruiting goals due to the attrition experienced during address canvassing. The recruiting goal has increased from 2.3 million to 2.7 million to ensure it has a large enough applicant pool. This increases the ratio of recruited applicants to positions from 5:1 to 6:1.

Completed a wage rate study and increased wages in 73 percent of counties by an average of $1.50 per hour for enumerators.

Developed an email campaign to maintain contact with individuals in the recruiting pool.

Decreased the types, and therefore the number, of positions that required a full background check.

Included additional training for replacement hires in the training schedules. A make-up session was added to the nonresponse follow- up training schedule, May 14-19, 2020.

If effectively implemented, these steps hold promise for helping to address the hiring issues.

The Bureau Experienced Challenges with Management’s Use of Information

To effectively manage address canvassing, the Bureau provides data- driven tools for the census field supervisors to manage listers, including system alerts that identify issues that require the supervisor to follow up with a lister. Operational issues such as listers not working assigned hours or falling behind schedule need to be resolved quickly because of the tight time frames of the address canvassing and subsequent operations. For the address canvassing operation, the system generated codes that covered a variety of operational issues such as unusually high or low productivity (which may be a sign of fraud or failure to follow procedures) and administrative issues such as compliance with overtime and completion of expense reports and time cards.

During the operation, more than 621,000 alerts were sent to census field supervisors. Each alert requires the supervisor to take action and then record how the alert was resolved. To assist supervisors, these alerts need to be reliable and properly used. However, nine out of 22 census field supervisors we spoke to indicated the alerts were not always useful. For example, almost 40 percent of those alerts were related to no progress being made on a block. This was due in part to listers opening all of the blocks they were assigned on their laptops in order to manage their workload, triggering the system that work had begun on all assigned blocks when in fact the lister was only working one block. We first heard about this issue from field supervisors in late August. Census field supervisors we spoke to indicated that these alerts took an inordinate amount of time to resolve, in part because almost every lister would open every block to plan his or her day.

We alerted Bureau officials in headquarters, and they notified area census offices to remind supervisors to instruct listers not to open all of their blocks at once. After the notification was sent out, Bureau officials reported that the number of alerts due to blocks not being worked declined. Bureau officials further stated that this issue would not impact nonresponse follow-up because enumerators do not receive multiple assignments, but instead receive, work, and transmit only one assignment of housing units for follow-up a day.

Another challenge faced by census field supervisors was providing feedback to listers on why addresses failed quality control. Four of 22 census field supervisors we spoke with were not aware that they had access to the reasons why addresses on a block failed quality control. Knowing where to find this information would have allowed census field supervisors to communicate this information to listers, thus improving lister performance as well as the accuracy of the data collected. We shared this information on some census field supervisor’s lack of awareness with the Bureau and on August 26, 2019, the Bureau notified its field offices to remind supervisors that detailed information on why addresses failed quality control was available on their laptops.

For nonresponse follow-up, Bureau officials told us QC information about any enumerator with a specified number of failed cases will be sent directly to the Regional Census Center rather than the census field supervisor. The Regional Census Center will decide whether the enumerator should continue working and, if so, what corrective action to take, such as retraining. However, if it is determined that an enumerator falsified data, then the enumerator would not be given new assignments and all of his or her work would then be reinterviewed.

Agency Comments

We provided a draft of this report to the Secretary of Commerce. In its written comments, reproduced in appendix I, the Bureau noted that our report made no formal recommendations and that we highlighted several successes of the in-field address canvassing operation. The Bureau also described several claims of cost savings and efficiency gains which it attributed to various address list-building activities. While we have previously reported on the Bureau’s 2020 address list-building efforts, we have not audited claims made in the Bureau’s response or elsewhere regarding potential cost savings from innovations for the 2020 Census. The Bureau also provided us with technical comments, which we incorporated as appropriate.

We are sending copies of this report to the Secretary of Commerce, the Under Secretary of Economic Affairs, the Director of the U.S. Census Bureau, and interested congressional committees. The report also will be available at no charge on GAO’s website at http://www.gao.gov.

If you or your staff have any questions about this report please contact me at (202) 512-3236 or mihmj@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff that made major contributions to this report are listed in appendix III.

Appendix I: Comments from the Department of Commerce

Appendix II: Area Census Offices Responsible for Locations Visited in This Review

Appendix III: GAO Contact and Staff Acknowledgments

GAO Contact Staff Acknowledgments

J. Christopher Mihm, (202) 512-3236 or mihmj@gao.gov In addition to the contact named above, Lisa Pearson, Assistant Director; Timothy Wexler, Analyst-in-Charge; Margaret Fisher; Robert Gebhart; Richard Hung; Cynthia Saunders; Anna Sorrentino; Kate Sharkey; Dylan Stagner; Jon Ticehurst; Peter Verchinski; and Alicia White made key contributions to this report.