2010 Census: Census Bureau Needs Procedures for Estimating the Response Rate and Selecting for Testing Methods to Increase Response Rate
Published: 2008-09-30T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The U.S. Census Bureau (Bureau) estimates that it will spend at least $2 billion to enumerate households that did not return census forms during the 2010 Census. Increasing the response rate would reduce the number of households that Bureau field staff must visit. To address concerns about reducing the cost of enumerating these households, GAO (1) analyzed how the Bureau develops, supports, and updates the response rate estimate, and the extent to which the Bureau uses the estimate to inform its 2010 planning efforts; (2) described the methods the Bureau considered for increasing response in 2010 and how it tested these methods; and (3) assessed how the Bureau identifies and selects for testing methods to increase response rate, including considering other surveys' methods. To meet these objectives, GAO analyzed the Bureau's documentation for estimating the response rate and selecting for testing methods to increase response, and interviewed experts from other survey organizations.
section_title: What GAO Found
paragraphs: The 2010 Census response rate estimate is not fully supported, systematically reevaluated, or clearly incorporated into the life cycle cost estimate and planning efforts for nonresponse follow-up--where census workers visit households that do not return their census forms. Specifically, the Bureau could not demonstrate support for one component underpinning the estimate--a general decline due to decreasing public participation in surveys--because it did not document its decisions or data sources when developing the estimate. The two other estimate components that affect responses are the short-form-only census and the replacement questionnaire. In 2001, the Bureau estimated the 2010 Census response rate to be 69 percent. However, from 2001 through 2008, the Bureau did not systematically reevaluate the estimate or consider test results from this decade to determine if the estimate should be updated. Although the Bureau revised the estimate to 64 percent after a major redesign of its nonresponse follow-up operation in 2008, the Bureau still lacks procedures for establishing when and how to reevaluate and, if necessary, update the estimate. To estimate costs and plan for nonresponse follow-up, the Bureau relies on response rate estimates for local census office types because these estimates reflect geographic differences. Officials said that the local estimates reflect components of the national estimate. However, only one of the three components from the national estimate--the replacement questionnaire--was clearly reflected in the local census office type estimates. Through various national and field tests and experiments, the Bureau tested nine methods to increase 2010 Census response and currently plans to implement two of these methods--the replacement questionnaire and two-column bilingual form. The Bureau also plans to use a communications campaign to increase response and plans to test campaign messages in 2009. In July 2006, the Bureau decided not to include an Internet response option in the 2010 Census. However, the Bureau recently announced that it is again considering including the Internet option in 2010, although it has not developed further plans for testing it. For 2010, the Bureau established test objectives and research questions to identify methods to test for increasing response. However, Bureau officials did not document the methods that they considered but decided not to test or the rationale behind those decisions. Although officials said that they considered cost, complexity of the test, and compatibility of experiments in their decisions, they did not specify how they weighed these factors to select and prioritize the nine methods they chose to test. Officials said that they consider the experiences of other survey organizations to identify potential methods to increase response, but they, along with some experts, noted that such methods may only be indirectly applicable to the decennial census. Nonetheless, testing modifications to methods the Bureau has previously considered or tested, such as testing a variety of telephone reminder messages, may yield additional opportunities for increasing response.

section_title: Letter
paragraphs: The U.S. Census Bureau (Bureau) estimates that even after adjusting for  inflation, the 2010 decennial census will be the most expensive census in  our nation’s history, costing from $13.7 billion to $14.5 billion. The Bureau  estimates that more than $2 billion will be used to employ temporary field  staff for nonresponse follow-up—its largest field operation where  enumerators interview households that did not return census forms.  Increasing the response rate would reduce the number of households that  Bureau field staff must visit during this nationwide operation. According  to Bureau officials, a 1 percent increase in the response rate can save   $75 million.
The Bureau expects to hire over 700,000 temporary workers to conduct  nonresponse follow-up with about 47 million households over the course  of 10 weeks in 2010. The Bureau initially based the schedule, staffing, and  funding it needed for nonresponse follow-up on an estimated national  response rate of 69 percent. However, in February 2008, the Director of  the Bureau initiated a replanning of the Field Data Collection Automation  program—a major acquisition that includes systems; equipment, including  handheld computers; and infrastructure for field staff to use in collecting  data for the 2010 Census. After analyzing several options to revise the  design of the 2010 Decennial Census, on April 3, 2008, the Secretary of  Commerce announced that the Bureau would no longer use handheld  computers in nonresponse follow-up and revised the estimated national  response rate from 69 percent to 64 percent. The Bureau estimated that  this option would result in a cost increase of $2.2 billion to $3 billion over  the previously reported cost estimate of $11.5 billion.
To address your concerns about reducing the cost of nonresponse follow- up operations, we reviewed the Bureau’s estimated response rate and  plans for increasing response. Specifically, we (1) analyzed how the  Bureau develops, supports, and updates the response rate estimate, and  the extent to which the Bureau uses the estimate to inform its 2010  planning efforts; (2) described the methods the Bureau considered for  increasing response rates in 2010 and what it did to test these methods;  and (3) assessed how the Bureau identifies and selects for testing methods  to increase response rate, including considering other surveys’ methods  for increasing response.
To meet these objectives, we reviewed documentation to support the  components of the response rate estimate and research literature on  methodologies to increase response to mail surveys and efforts to estimate  survey response rate. We analyzed Bureau documents related to 2000  Census evaluations, the 2010 research and testing program, the 2010  Census life cycle cost estimate, and the communications campaign. We  also interviewed Bureau officials in the Decennial Management Division  and other divisions about methods to increase self-response. Further, we  interviewed experts on survey methodology from Statistics Canada, the  U.S. Department of Labor’s Bureau of Labor Statistics, and various  academic institutions, as well as former Census Bureau officials and  researchers. We asked the experts a common set of questions in order to  compare responses and identify recurring themes. Appendix I includes a  list of experts we interviewed and additional information on our scope and  methodology.
We conducted our review from July 2007 through September 2008 in  accordance with generally accepted government auditing standards. Those  standards require that we plan and perform the audit to obtain sufficient,  appropriate evidence to provide a reasonable basis for our findings and  conclusions based on our audit objectives. We believe that the evidence  obtained provides a reasonable basis for our findings and conclusions  based on our audit objectives.
subsections: section_title: Background
paragraphs: In 1970, the Bureau moved away from conducting the census door-to-door  and began mailing census questionnaires to households to be filled out and  returned by mail. Since the 1970 Census, the Bureau has used mail as the  primary method for collecting census data. For Census 2000, the  mailout/mailback method was used for more than 80 percent of the  population. Households that fail to mail back the census questionnaires  are included in nonresponse follow-up workload, where enumerators  follow up with door-to-door visits and telephone calls or solicit census  data from knowledgeable people, such as neighbors.
The census response rate declined dramatically from 1970 through 1990.  The 1970 response rate was 78 percent. The rate decreased to 75 percent  for the 1980 Census and then decreased again to 65 percent for the 1990  Census. Although the Bureau estimated that the 2000 Census response  rate would continue to decline to 61 percent, actual response exceeded  the estimate, reaching 65 percent for the mailout/mailback universe prior  to nonresponse follow-up. The response rate is defined as the percentage  of census forms completed and returned for all housing units that were on  the Bureau’s address file eligible to receive a census questionnaire  delivered by mail or by a census enumerator. The denominator used in  calculating the response rate includes vacant housing units and other  addresses that were determined to be undeliverable or deleted through  other census operations.
Since 1970, per household census costs have increased (in constant 2010  dollars) from about $14 in 1970 to an estimated $88 in 2010—a figure that  does not include the recent, major redesign of the Field Data Collection  Automation program. According to the Bureau, factors contributing to the  increased costs include an effort to accommodate more complex  households, busier lifestyles, more languages and greater cultural  diversity, and increased privacy concerns. In addition, the number of  housing units—and hence, the Bureau’s workload—has continued to  increase. The Bureau estimated that the number of housing units for the  2010 Census will increase by almost 14 percent over 2000 Census levels  (from 117.5 million to 133.8 million housing units). As a result of multiple  factors, the total inflation-adjusted life cycle cost for the decennial census  has increased more than tenfold since 1970, as shown in figure 1.
The Bureau conducts its testing, evaluation, and experimentation program  for the decennial census primarily through the Decennial Statistical  Studies Division. The division develops and coordinates the application of  statistical techniques in the design and conduct of decennial census  programs. The Bureau conducts various tests throughout the decade to  determine the effect and feasibility of form or operational changes. For  example, in the 1990s, the Bureau tested whether adding a mandatory  response message and implementing a multiple mailing strategy, such as  sending an advance letter and reminder postcard, as shown in figures 2  and 3, would increase the response rate. Based on positive test results,  these methods were added to the Census 2000 design.
The Decennial Statistical Studies Division also conducts evaluations of  census operations typically using data collected as part of the decennial  process to determine whether individual steps in the census operated as  expected. It is also responsible for conducting experiments during the  census that may be instructive for future censuses. These experiments  usually involve using alternative processes for a subset of the population.  The Bureau is working with internal and external stakeholders on defining  and developing its program for experiments and evaluations for the 2010  Census, known as Census Program for Evaluations and Experiments, with  design work and implementation starting in 2008 and continuing through  2011. The final set of activities would include analysis, documentation, and  presentation of the research, and these activities would start in 2009 and  be completed by the end of fiscal year 2013.
section_title: The Response Rate Estimate Lacks Support, Is Not Systematically Reevaluated, and Is Not Clearly Incorporated into Planning Efforts
subsections: section_title: 2010 Census Response Rate Estimate Is Not Fully Supported
paragraphs: The Bureau developed the response rate estimate for the 2010 Census in  2001 but could not demonstrate support for one of the three components  underpinning the estimate. To establish the 2010 estimate, Bureau officials  told us that they used Census 2000 mail response data as the baseline and  then incorporated three other components to arrive at a response rate  estimate of 69 percent: an increase in response related to eliminating the  long form and moving to a short-form-only census, an increase in response  related to sending a replacement questionnaire to nonresponding  households, and a general decline in mail response due to decreasing  public participation in surveys. The components of the estimate are  outlined in figure 4, and a detailed explanation of the baseline and each  component follows.
The baseline  repone rte i the rte  chieved in  milot/milback  reas in Census   2000 when the nonrepone  follow-p  worklod was  identified.
Tritioning to   hort-form-  only census  expected to  increase  reponecause the  long form, ent  to one-ixth of  the poption in  the past, has lower  repone rte  hitoriclly.
Miling   replcement  qtionnire to  ll household  tht hve not  reponded   certin dte has  een hown to  increase mil  repone in  te.
Prticiption i  expected to  decline asrt of  trend of  decreasing public  prticiption in  survey.
subsections: section_title: Baseline Rate
paragraphs: Included in the baseline national response rate of 65 percent are the  effects of methods, such as the multiple mailing strategy and the  communications campaign (called the Partnership and Marketing  Program) that were implemented in Census 2000 and are planned to be  implemented in the 2010 Census as well. Bureau officials stated that the  communications campaign, which included paid advertising, had an  impact on the response rate achieved in 2000, but were unable to quantify  that effect and did not project the campaign’s effect for 2010.
section_title: Short-Form-Only Census
paragraphs: The Bureau estimated an increase of 1 percentage point from eliminating  the long form and moving to a short-form-only census. The Bureau  conducted an analysis comparing Census 2000 response rates for the short  form and the long form. Even though the difference in response rates for  the two form types was rather large—the short form had a response rate  that was more than 10 percentage points higher than the long form—the  overall effect on the response rate estimate was small because the long  form was sent to only approximately 17 percent of housing units in 2000.
section_title: Replacement Mailing
paragraphs: In 2001, the Bureau expected that sending a replacement questionnaire to  households that had not responded by a certain date would increase the  response rate by 7 percentage points. The magnitude of this component  was based on test results from the 1990s, which showed an increase in the  response rate related to the replacement mailing of at least 7 percentage  points, and possibly 10 percentage points or more. Bureau officials stated  that they estimated this effect conservatively because they assumed that a  higher response rate from a replacement questionnaire is more likely to  occur in tests than in the decennial census. Also, testing showed that not  all the respondents who return replacement questionnaires would be  removed from the nonresponse follow-up workload prior to being  enumerated because of the timing of the enumerator’s visit.
section_title: General Decline
paragraphs: The Bureau estimated a decrease of 4 percentage points from the baseline  level due to what it believes is a decline over time in public survey  participation. However, this assumption is not supported by quantitative  analysis or research studies but rather based on the opinion of subject  matter experts and senior officials. Further, the Bureau could not  demonstrate who was consulted, when they were consulted, or how they  decided on the amount of the general decline.
Best practices for cost estimation from our Cost Assessment Guide call  for documenting assumptions and data sources used to develop cost  estimates, such as the general decline component of the response rate  estimate. In addition, several experts we interviewed agreed that the  Bureau should have used quantitative analysis of mail response data from  other surveys to support the general decline component when it developed  the estimate in 2001. To help support the general decline component, we  suggested that the Bureau use changes in participation in the American  Community Survey, which uses a similar mailing strategy to the decennial  census and for which response is also required by law, for comparison  purposes. In May 2008, the Bureau completed an analysis of American  Community Survey response rate data from 2000 to 2007, which  demonstrated a decline of 6.6 percentage points in cooperation rates to  the initial questionnaire. Although the decline in response to the American  Community Survey may not be directly comparable to response behavior  for the decennial census, it provides some support for the Bureau’s  assumption that mail survey participation may be declining.
The Bureau does not have procedures for developing the response rate  estimate. Specifically, the Bureau has no established policies for  documenting deliberations related to or data sources used in developing  the estimate, including no 2010 Census decision memorandum to  document the original estimate prepared in 2001. Several experts also  suggested that the Bureau should develop a model for developing the  estimate and incorporate response rate data and demographic data from  other surveys.
Bureau officials, however, noted that they have not determined how to  apply data from other surveys to estimate the census response rate  because the census differs from other surveys, nor do they plan to develop  a model of the response rate estimate. Instead, Bureau officials stated that  they tried to conservatively estimate the response rate to ensure that they  are adequately prepared during nonresponse follow-up and to avoid  repeating what happened in 1990, when the Bureau overestimated the  response rate, requiring a supplemental appropriation of $110 million and  forcing it to extend nonresponse follow-up by up to 8 weeks for some  areas. In contrast, in 2000 the response rate exceeded the Bureau’s  estimate. Conservatively estimating the response rate may be a reasonable  approach. However, having clear, detailed documentation about decisions  related to or data sources used in developing the estimate would enable  the Bureau and others to better assess whether the estimate is reliable,  such as whether its assumptions are supported and realistic. An unreliable  response rate estimate can produce an inaccurate cost estimate and can  increase risk and uncertainty to operational plans that are based on the  response rate estimate.
section_title: The Bureau Does Not Systematically Reevaluate the Response Rate Estimate
paragraphs: Although the Bureau updated the response rate estimate as a result of a  major redesign of census operations undertaken in April 2008, the Bureau  still lacks procedures for establishing when and how to reevaluate the  response rate estimate. From 2001 through April 2008, the Bureau did not  reevaluate the estimate to determine whether it should be updated, even  though, after establishing the initial estimate in 2001, the Bureau  completed several evaluations from Census 2000 and conducted several  census tests that could have informed the estimate. During the first few  years of the decade, the Bureau completed 12 evaluations that discussed  aspects of the Census 2000 response rate. In addition, the Bureau  conducted five tests from 2003 through 2007 that were designed, in part, to  test methods to increase response. For example, in 2005 and 2007, the  Bureau tested a two-column bilingual form, which includes identical  questions and response options in English and Spanish. The 2005 test  demonstrated a significant positive effect on mail response rates of 2.2  percentage points overall, and a 3.2 percentage point increase in areas  with high concentrations of Hispanic and non-White populations, and the  2007 test revealed a similar impact. However, Bureau officials stated that  they were concerned that the two-column bilingual form has not been  tested in a census environment. In addition, they noted that the bilingual  form will be sent to only a portion of the country—approximately 10  percent of census tracts—with high concentrations of Spanish-speaking  populations, based on American Community Survey data.
Best practices state that assumptions, such as the response rate estimate,  should be revised as new information becomes available. Our Cost  Assessment Guide recommends that preliminary information and  assumptions be monitored to determine relevance and accuracy and be  updated as changes occur to reflect the best information available.  Further, according to several experts, the Bureau could have updated the  estimate based on response rate data and demographic data from other  surveys, and another expert suggested that the Bureau use triggering  events, such as after tests, for reviewing the response rate estimate. Based  on this expert’s experience, the Bureau could establish a change control  board chaired by senior Bureau officials to determine whether the  response rate estimate should be revised. However, Bureau officials  explained that they do not update the estimate based on results from tests  because the tests cannot replicate the decennial census environment.  Overall, the Bureau has not specified when or how it is to reevaluate its  response rate estimate. Establishing these procedures would help the  Bureau ensure that the estimate is current and reliable in order to better  inform planning efforts.
The Bureau revised the estimate down to 64 percent after announcing that  nonresponse follow-up would be changed from an automated to a paper- based operation. Prior to the redesign, the Bureau had planned to reduce  the nonresponse follow-up workload by using handheld computers to  remove households that returned their forms late—including many  replacement mailing returns—from the enumerator assignments on a daily  basis. According to Bureau officials, they revised the response rate  estimate based on the timing of mail returns in 2000 and replacement  questionnaire returns in the 2003 test. However, the revised estimate does  not fully reflect recently designed procedures to remove late mail returns  in 2010. Specifically, for 2010 the Bureau now plans to mail the  replacement questionnaire earlier, send a blanket replacement  questionnaire to some areas and a targeted replacement questionnaire to  others, and conduct three clerical removals of late mail returns  immediately prior to and during the nonresponse follow-up operation.  These operational plans are still being finalized pending further analysis.  Bureau officials said that although they hope that these revised operations  will increase response, they did not update the response rate estimate to  reflect these current operational plans because they have not tested this  approach under decennial census conditions and therefore have no basis  for estimating the potential effect of these operational changes on  response.
section_title: Local Response Rate Estimates Used in Planning Efforts Lack Support
paragraphs: To estimate costs and to plan for activities such as nonresponse follow-up,  questionnaire printing, and postage needs, the Bureau uses a life cycle cost  model that relies on response rate estimates for four primary LCO types  and not the national estimate. Using LCO-type estimates helps to account  for expected differences in response patterns across geographic settings.  For example, response rates for inner-city areas are estimated to be lower  than in suburban areas. These estimates range from 55 percent to 72  percent, but lack support for how they were developed. In determining the  2010 response rate estimates for the four LCO types, the Bureau said that  it equally applied two components from the national estimate—the  replacement mailing component and the short-form-only component—to  the LCO types in the cost model. However, the Bureau did not conduct  quantitative analysis to determine whether the replacement mailing  component in the national estimate should be applied equally for each  LCO type. By applying an equal percentage point increase across all LCO  types for the replacement mailing, the Bureau has, in effect, assumed that  the LCO type with the lowest baseline—48 percent—would experience a  higher relative increase in response—15 percent—due to the replacement  mailing than the LCO type with the highest baseline—65 percent—which  would be expected to experience an 11 percent increase in response. The  Bureau has not demonstrated support for assuming that the LCO types will  experience these different relative increases in response. Further, the  Bureau could not demonstrate whether or how the short-form-only  component and the general decline component of the national estimate  were reflected in the life cycle cost model.
Best practices for cost estimation from our Cost Assessment Guide call  for assumptions to be thoroughly documented, data to be normalized, and  each cost element to be supported by auditable and traceable data  sources. Following these best practices could allow the Bureau to enhance  its use of the response rate estimate in planning for the decennial census  and better inform stakeholders about the reliability of the estimate. It is  unclear why officials could only explain how one component of the  national response rate estimate was applied to LCO-type estimates in the  life cycle cost model. However, according to the Bureau, it is currently  documenting how these estimates are used to calculate the costs for  operations and how the estimates have changed over time. Having support  for the LCO-type estimates would better inform the Bureau’s planning  efforts for operations that directly rely on response, such as determining  workload and hiring enumerators for nonresponse follow-up.
section_title: The Bureau Plans to Incorporate Additional Methods to Increase Mail Response in 2010
subsections: section_title: Two of Nine Methods Tested to Increase Response Rate Are Planned to Be Included in the 2010 Census
paragraphs: We determined from reviewing Bureau testing documents that through  various national and field tests and experiments, the Bureau tested nine  methods to increase self-response to the 2010 Census, as shown in table 1.  The Bureau currently plans to implement two of these methods—the  replacement questionnaire and two-column bilingual form (see fig. 5).  Three other methods for increasing mail response and four methods for  increasing response through an electronic data collection system were  tested but are not planned for implementation in 2010. An additional  method the Bureau plans to include in the 2010 Census to increase  response, the Integrated Communications Campaign, has not been tested,  although the Bureau conducted a similar campaign in 2000 and plans to  test campaign messages with audiences in early 2009. Additional details  regarding the testing of these methods can be found in appendix II.
For the 2010 Census, the Bureau does not plan to include three methods it  tested to increase mail response, as shown in figures 6, 7, and 8. Two of  these methods—a telephone reminder call and messaging to distinguish  the initial and replacement questionnaires—were not found to  significantly improve response and therefore will not be implemented as  part of the 2010 Census. The third method—including a due date on the  initial mailing package—was found to generate faster responses when  tested in 2003, and increase overall response by 2 percentage points when  tested in conjunction with a compressed mailing schedule in 2006.  However, the Bureau also believes that the use of a due date alone could  cause a lower response rate because some people may not send back the  census form after the due date. According to Bureau officials, they are not  including this method in the decennial census design because they would  like to further test it—both with and without a compressed mailing  schedule—in a census environment. It will be important for the Bureau to  optimize this testing opportunity by designing the test to determine the  extent to which the faster and higher response is due to a compressed  schedule versus a due date, as well as exploring other test treatments the  Bureau has recommended in the past, such as including the due date on  multiple mailing pieces. Bureau officials said that additional testing on the  use of a due date will be conducted as part of the 2010 Census Program for  Evaluations and Experiments, which the National Academy of Sciences  recommended in its December 2007 interim report. The Bureau did not  provide further details on plans for the Census Program for Evaluations  and Experiments, but officials have said that they will consider costs and  staffing needs in deciding what to evaluate. The academy’s final report is  due in September 2009.
In addition to testing methods to increase mail response, the Bureau also  tested four methods to increase self-response through electronic data  collection systems: (1) providing for response via an interactive voice  response system;( 2) sending a letter to encourage either responding via  Internet or returning the initial questionnaire, instead of sending a  replacement questionnaire; (3) using computer-assisted telephone  interviewing; and (4) providing capability for responding via the Internet.  The Bureau found that the first two methods did not increase response in  tests and does not plan to include them in the 2010 Census. Computer- assisted telephone interviewing allows respondents to use the telephone  to connect with operators who record interview responses electronically.  The Bureau found that this method increased the overall response rate in a  Census 2000 experiment. However, in a March 2004 report, the Bureau  also stated that the method would likely be too costly in terms of  hardware, software, and staffing resources, compared to the increase in  response it might generate. Computer-assisted telephone interviewing was  not tested after the 2000 Census.
Although the Internet option was found to increase overall response  during the Census 2000 experiment, it did not increase overall response  when tested again in 2003, when a replacement mailing was also tested.  According to a July 2006 Bureau decision memo titled, “Rationale for the  Decision to Eliminate the Internet Option from the DRIS Contract,” the  Bureau decided not to include the Internet option in the design of the 2010  Census largely because it had underestimated the costs of the contract  that included developing the Internet response option. In responding to a  draft of this report, the Bureau stated that they made the decision because  test results showed that offering the Internet option did not increase  overall response and would not offer any cost savings. According to the  2006 memo, the Bureau also determined that the operational risks and  costs to develop this option outweighed the potential benefits. In terms of  benefits, the Bureau found improvements in data completeness and data  timeliness from Internet responses in tests conducted in 2000/2001 for the  American Community Survey and in 2003 and 2005 for the National Census  Tests. The Bureau noted that these benefits could translate into reduced  costs since less follow-up is required to improve data accuracy, and earlier  receipt of responses could result in fewer replacement questionnaires that  need to be mailed and fewer households that need to be enumerated  during nonresponse follow-up. However, with only 6 to 7 percent of the  test population using the Internet option, the Bureau concluded that no  cost savings could be realized from reducing the number or size of data  capture centers (facilities that process returned questionnaires) planned  for 2010. Finally, the Bureau stated that the inability to fully test the  Internet option and growing concerns about Internet security made it  unfeasible for the Bureau to implement the Internet as a response option  for 2010. Despite its July 2006 decision, the Bureau recently announced  that it is considering including the Internet option as part of the 2010  Census design; however, the Bureau has not developed plans for further  testing this option.
section_title: Testing and Evaluation Plans for the 2010 Communications Campaign Are Not Yet Fully Developed
paragraphs: Through its contractor and subcontractors, the Bureau has taken a  number of steps to inform the planning of the communications campaign  but has not yet fully developed the campaign’s testing and evaluation  plans. From late 2006 through early 2008, focus groups were conducted  with various ethnic groups and hard-to-count populations, such as  unattached singles, to identify potential barriers and motivators to  participation, to better understand methods of communication that work  for different groups, and to develop the campaign’s overall creative  expression. The Bureau also developed an audience segmentation model  using Census 2000 response data, updated through 2006 using American  Community Survey data, to provide a more detailed understanding of the  characteristics, such as home ownership and unemployment level, of  those more or less likely to respond, as well as where they live, in order to  better target communications to encourage census participation. In  addition, in 2008, a national phone and in-person survey is being  conducted to further explore barriers and motivators to response,  particularly in hard-to-count populations, which would inform campaign  message development.
According to agency officials, beginning in early 2009, campaign messages  are to be tested by showing storyboards to audiences that will use  electronic devices to vote on the messages. These messages will be tested  in 14 languages and for other populations, such as Spanish speakers in  Puerto Rico. According to the draft plan, testing of events, partnership  toolkits, promotional items, and public relations ideas will also be  conducted. However, the Bureau has not yet developed detailed plans for  this testing because, according to one official, the Bureau intends to  further develop testing plans as future fiscal year funding amounts become  available. An example of a Census 2000 communications campaign poster  is shown in figure 9.
In addition, although the Bureau expects to award a contract by the end of  fiscal year 2008 for an independent evaluation measuring the campaign’s  performance against its goals of increasing mail response, improving  accuracy, and reducing the differential undercount, and improving  cooperation with enumerators, it has not yet done so. In the past, the  Bureau has said that although evaluations have shown that the Census  2000 communications campaign increased awareness for that census, it  was difficult to link increased awareness to changes in respondent  behavior. Bureau officials said that they have attempted to analyze Census  2000 data to identify factors that influence behavior, but their research  results were inconclusive. Going forward, it will be important for the  Bureau to determine its plans for evaluating the 2010 communications  campaign so that it does not miss opportunities to collect data in the  census environment to inform future campaigns.
section_title: Bureau Lacks Procedures for Selecting for Testing Methods to Increase Response Rate
paragraphs: For 2010, the Bureau developed a strategy to identify various methods to  test for increasing response. Specifically, the Bureau established test  objectives and research questions, such as identifying the optimal mix of  response options for the public to respond to the 2010 Census and  determining security and confidentiality issues surrounding technology.  The Bureau also developed the 2010 Census Integrated Program Plan  and 2010 Census Operations and Systems Plan to better document its  planning. However, Bureau officials did not document the methods that  they considered but decided not to test in the 2010 testing cycle or the  rationale behind those decisions. Further, for methods that the Bureau  decided to test, officials could not provide support for how they selected  or prioritized these methods. Although officials said that they considered  cost, complexity of the test, and compatibility of experiments in their  decisions, they did not specify how they defined or weighed these factors  to select and prioritize the nine methods they chose to test.
To ensure thorough and comprehensive planning of the decennial census,  our past work on lessons learned from Census 2000 highlighted the  importance of documentation to support research, testing, and evaluation,  and a comprehensive and prioritized plan of goals, objectives, and  projects. While the Bureau has developed the 2010 Census Integrated  Program Plan and 2010 Census Operations and Systems Plan to better  document its planning, these and other planning documents we reviewed  did not provide support for how the Bureau selected and prioritized  methods to test. It is unclear why the Bureau lacks procedures for  documenting decisions concerning how it selected for testing methods to  increase response. Documenting decisions about methods that were not  selected for testing would help the Bureau more effectively build capacity  and institutional knowledge about changes in methods to consider in the  future.
According to Bureau officials, they consider the experience of other  survey organizations when identifying methods for increasing response  rate. For example, they said that they attend research conferences to learn  about experiences of other organizations that conduct national surveys to  identify potential methods to increase response rate. Both Bureau officials  and some survey experts noted that methods used to increase response in  other surveys may only be indirectly applicable to the decennial census.  For example, for the Economic Census, officials said that the Bureau  sends nonresponding businesses multiple reminder letters; the final letter  to large businesses informs them that the Department of Justice will be  pursuing nonrespondents. Bureau officials said that these methods are  less feasible for the decennial census to implement because of the shorter  time frame for obtaining responses and concerns about being respondent- friendly to households. In addition, one survey expert noted that methods  applicable to small-scale surveys, such as personalizing a cover letter, may  be less feasible to implement for the decennial census.
Further, survey experts we interviewed generally said that they were  unaware of additional methods from censuses undertaken by other  countries or private sector surveys that the Bureau could consider to  increase mail response. Some experts noted differences between how the  United States and other countries conduct their censuses, which may  make it difficult to directly transfer other countries’ practices to the U.S.  census. For example, some European censuses use a population register  to collect names, unlike the United States, which builds its survey frame  from phone and address lists. In addition, past research has provided  evidence that government-sponsored self-administered surveys, such as  the decennial census, tend to achieve higher response rates than  nongovernmental surveys.
Nonetheless, testing modifications to methods that the Bureau has  previously considered or tested in earlier studies may yield additional  opportunities for increasing response. For example, the Bureau could test  various telephone reminder messages stating that response is mandatory  by law or providing instructions for obtaining a new census form.  Although Bureau officials said that they have previously used the  American Community Survey to inform the census design, analyzing  respondent behavior to the American Community Survey, because of its  similar mailing strategy to the decennial census, could help the Bureau  regularly refine its survey methodology for increasing census response.  Although the survey forms are different, many concepts, such as targeting  the second mailing, modifying the appearance of the mailing, and varying  telephone and Internet messages to prompt nonrespondents, could be  tested with reasonable inference to the census.
section_title: Conclusions
paragraphs: Nonresponse follow-up is the largest field operation, and the Bureau  estimates that it will cost more the $2 billion. To control the cost of  nonresponse follow-up, it will be important for the Bureau to devise a  strategy for getting people to return their census forms. A reliable  response rate estimate is a critical element necessary for determining the  resources needed to carry out nonresponse follow-up. The Bureau did not  have support for one of the components of the 2010 response rate  estimate—a general decline in responsiveness—and the Bureau does not  have procedures for reviewing the estimate after testing to determine  whether it should be revised. Establishing procedures for developing the  response rate estimate, including documenting data sources and decisions,  would enable the Bureau and others to better assess the estimate’s  reliability. Also, establishing procedures for when, such as after tests or  other triggering events, and how to reevaluate the estimate would help the  Bureau ensure that it is providing the most current response rate estimate  for planning nonresponse follow-up and other activities, such as  questionnaire printing. The Bureau’s strategy of estimating the response  rate conservatively may be prudent given past difficulties with conducting  nonresponse follow-up after overestimating the response rate in 1990.  Nonetheless, establishing and following procedures for developing,  documenting, and reevaluating the estimate are important steps for  understanding differences between the estimate and the actual response  rate for 2010 and for evaluating the components and underlying  assumptions when developing the estimate for the next census. Successful  enumeration depends on early research, testing, and evaluation of census  methods to increase response. Establishing procedures for selecting and  prioritizing the testing of methods—such as the Internet or reminder  telephone call—including documenting methods considered but not  tested, would help the Bureau demonstrate that it has chosen an optimal  research strategy for the decennial census, more effectively build capacity  and institutional knowledge about changes in methods to consider in the  future, and enable it to more efficiently begin testing for the next census.
section_title: Recommendations for Executive Action
paragraphs: To enhance credibility of the response rate for determining cost and  planning for future census activities, to inform assumptions underlying the  2020 response rate estimate, and to improve the planning and  transparency of the Bureau’s research and testing, we are recommending  that the Secretary of Commerce direct the Bureau to take the following  three actions:    Establish procedures for developing the 2020 response rate estimate,  including documenting the data sources supporting the estimate's  components and decisions that are made in establishing the  components and analyzing 2010 data to assess the reasonableness of  assumptions used in applying the national estimate’s components to  the LCO-type estimates.
Establish procedures for reevaluating and updating the 2020 estimate,  including identifying events or changes in related operations that  should trigger a review and documenting the results of such reviews.
Establish procedures for selecting methods for increasing response  rate that will be the subject of research and testing, including  requirements for documenting how the Bureau defines and weighs  factors used to select methods and documentation on methods  considered but not tested.
section_title: Agency Comments and Our Evaluation
paragraphs: The Secretary of Commerce provided written comments on a draft of this  report on September 22, 2008. The comments are reprinted in appendix III.  Commerce generally agreed with our conclusions and recommendations  and stated that the Bureau is committed to developing and implementing a  documented, systematic methodology for establishing response rate  estimates for future censuses and reevaluating the estimates throughout  the decade. Because the recommendations in our draft report focused on  costs and planning for the 2010 Census, we revised our recommendations  to reflect actions to be taken to support future census planning, including  analyzing 2010 data to assess the reasonableness of assumptions used in  applying the national estimate’s components to the LCO-type estimates.
Commerce also provided technical corrections, which we incorporated as  appropriate. In its comments, Commerce disagreed with our statement  that the Internet response option increased response in the Census 2000  experiment. Commerce cited a summary statement from a Bureau report  that concluded that the use of multiple response modes tested in 2000  does not increase response. However, Bureau analyses from October 25,  2002, and March 2004 on the Census 2000 experiment stated that the  overall response rate increased when the Internet was offered as an  alternative response mode. We therefore made no changes in the report.
Commerce strongly disagreed with our statement about the reason why  the Bureau decided not to include the Internet option in the design of the  2010 Census. In the report, we state that underestimating contract costs  was the primary reason the Bureau eliminated the Internet option, which  we attribute to a July 19, 2006, memo documenting the Bureau’s rationale  for eliminating the Internet response option from the Decennial Response  Integration System (DRIS) contract. This memo states that the decision  “was due largely to the fact that the Census Bureau underestimated the FY  2006-2008 contractor costs proposed to develop DRIS.” We therefore left  this statement unchanged. However, we added the Bureau’s explanation  provided in its agency comment letter that the decision was based on test  results, which showed that offering this option did not increase overall  response and would not offer any cost savings.
As agreed with your offices, unless you publicly announce the contents of  this report earlier, we plan no further distribution until 30 days after its  issue date. At that time, we will send copies of this report to the Secretary  of Commerce, the Department of Commerce’s Inspector General, the  Director of the U.S. Census Bureau, and interested congressional  committees. We will make copies available to others upon request. This  report will also be available at no charge on GAO’s Web site at  http://www.gao.gov.
If you or your staff have any questions concerning this report, please  contact Mathew J. Scirè at (202) 512-6806 or sciremj@gao.gov or Ronald S.  Fecso at (202) 512-2700 or fecsor@gao.gov. Contact points for our Offices  of Congressional Relations and Public Affairs may be found on the last  page of this report. GAO staff who made major contributions to this report  are listed in appendix IV.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: The objectives of this report were to (1) assess how the U.S. Census  Bureau (Bureau) develops, supports, and updates the response rate  estimate, and the extent to which the Bureau uses the response rate  estimate to inform its 2010 planning efforts; (2) describe the methods the  Bureau considered for increasing response rates in 2010 and what it did to  test these methods; and (3) assess how the Bureau identifies and selects  for testing methods to increase response rates, including considering other  surveys’ methods for increasing response. The scope of our review was  limited to the census mailout/mailback universe, which covered more than  80 percent of households in Census 2000. The majority of households in  this category have standard city-style addresses, allowing them to receive  their census questionnaires in the mail in addition to being expected to  return their questionnaires by mail. We excluded from our review  operations and methods aimed at enumerating those not included in the  mailout/mailback universe, such as the Be Counted initiative and  Enumeration at Transitory Locations; those initiatives related to, but not  primarily focused on, increasing self-response, such as improving the  quality of the Master Address File and providing assistance through  Questionnaire Assistance Centers; methods tested prior to 2000 and  already implemented in previous censuses; and those primarily intended  to improve operational efficiency, such as postal tracking.
To determine how the Bureau develops and uses the response rate  estimate, we reviewed documentation to support the components of the  response rate estimate and research literature on efforts to estimate  survey response rate. We interviewed Bureau officials in the Decennial  Management Division about the process used to develop and update the  estimate, the assumptions on which the estimate is based, and how the  estimate is used. We also interviewed experts on survey methodology from  Statistics Canada, the U.S. Department of Labor’s Bureau of Labor  Statistics, and various academic institutions, as well as former Census  Bureau officials and survey methodologists to obtain their views on the  strengths and weaknesses of the Bureau’s process for developing and  updating the estimate. We compared responses to a common set of  questions we asked the experts in order to identify themes. We also  reviewed Bureau strategic planning documents—in particular, the 2010  Census life cycle cost model with supporting documentation, as well as  the 2010 Census Integrated Program Plan and the 2010 Census  Operations and Systems Plan—to understand the Bureau’s use of the  estimate, and evaluated the Bureau’s practices for using the response rate  estimate for generating the life cycle cost estimate against best practices  criteria in our Cost Assessment Guide and other relevant GAO products.
To describe the methods the Bureau considered for increasing response  rates in 2010 and how the Bureau tested these methods, we reviewed  Bureau analyses and evaluations of tests on methods to increase mail  response and various Bureau planning documents, such as the 2010  Census Operations and Systems Plan. In addition, we reviewed  documentation on the Bureau’s communications campaign, such as  evaluations of the 2000 Partnership and Marketing Program and the 2010  Census Integrated Communications Campaign Plan. We interviewed  Bureau officials in the Decennial Management Division to obtain  additional context about the methods they considered and tests they  conducted.
To assess how the Bureau identifies and selects methods to test for  increasing response rates, we reviewed various Bureau planning  documents, such as research and development planning group action  plans, for factors that the Bureau considers in selecting methods to test.  We interviewed Bureau officials in the Decennial Management Division  and other divisions about the process for identifying and selecting  methods for increasing response. We also reviewed our past work on  lessons learned from Census 2000 on the importance of documentation  and planning in order to evaluate the Bureau’s process for selecting and  prioritizing methods to test. We interviewed experts on survey  methodology from Statistics Canada, the U.S. Department of Labor’s  Bureau of Labor Statistics, and various academic institutions, as well as  former Census Bureau officials and researchers, about additional methods  for increasing response that the Bureau could consider and to obtain their  perspectives on the Bureau’s process for identifying, testing, and  implementing methods. We compared responses to a common set of  questions we asked the experts in order to identify recurring themes. We  also reviewed research literature on methodologies to increase response  to mail surveys to identify additional methods that the Bureau could  consider.
We interviewed the following experts in survey methodology:    Anil Arora, Director General, Census Program Branch, Statistics    Paul Biemer, Distinguished Fellow in statistics, Research Triangle    Don A. Dillman, Regents Professor, Thomas S. Foley Distinguished  Professor of Government and Public Policy, and Deputy Director for  Research and Development in the Social and Economic Sciences  Research Center, Washington State University, and former Senior  Survey Methodologist, U.S. Census Bureau    Elizabeth Martin, former Senior Survey Methodologist, U.S. Census    Colm O’Muircheartaigh, Vice President for Statistics and Methodology,  National Opinion Research Center and Professor, Irving B. Harris  Graduate School of Public Policy Studies, University of Chicago    Kenneth Prewitt, Carnegie Professor of Public Affairs, School of  International and Public Affairs, Columbia University and former  Director, U.S. Census Bureau    Clyde Tucker, Senior Survey Methodologist, Bureau of Labor Statistics We conducted our review from July 2007 through September 2008 in  accordance with generally accepted government auditing standards. Those  standards require that we plan and perform the audit to obtain sufficient,  appropriate evidence to provide a reasonable basis for our findings and  conclusions based on our audit objectives. We believe that the evidence  obtained provides a reasonable basis for our findings and conclusions  based on our audit objectives.
section_title: Appendix II: Testing of Methods to Increase Response to 2010 Census
paragraphs: Replacement mailing. The Bureau plans to include the replacement  mailing in the 2010 Census design. The replacement mailing involves  sending a new questionnaire to households after households have received  the initial census questionnaire in order to increase the likelihood that  households will respond. The Bureau tested a targeted replacement  mailing—where households that have not returned their initial  questionnaires by a cutoff date receive the replacement form—in the early  1990s and found that this method resulted in a 10 to 12 percent increase in  the mail response rate. A blanket replacement mailing—where all  households received a replacement questionnaire, including those that had  already responded—was planned for Census 2000, but the Bureau dropped  it from the design because of operational concerns that became apparent  in the 1998 dress rehearsal. A targeted replacement mailing was  incorporated into the 2010 design, based on the test results from the 1990s  and the Bureau’s plans to move to a short-form-only census. In 2003, the  Bureau tested the targeted replacement mailing and was able to confirm  the results of the tests from the early 1990s. Subsequent testing of the  replacement mailing focused on ensuring that the Bureau could implement  it successfully, including processing and removing the returns from the  nonresponse follow-up workload using the handheld computers.
The decision to eliminate the handheld computers from nonresponse  follow-up and make nonresponse follow-up a paper-based operation  diminished the Bureau’s ability to remove late mail returns (including  replacement questionnaires that are returned) from the nonresponse  follow-up workload. The Bureau then examined options to get a better  effect from the replacement mailing and decided to try to get the  replacement mailings out more quickly. In order to do this, the Bureau  decided that it would send a blanket replacement mailing to census tracts  identified as low response areas, based on Census 2000 response data and  updated demographic data from the American Community Survey. The  replacement mailing packets for these tracts can be printed and labeled in  advance—approximately 25 million total. Households not in these areas  will receive a replacement questionnaire on a targeted basis—that is, only  if they are in census tracts identified as medium response areas and if their  initial questionnaires have not been received by the cutoff date. The  Bureau estimates that it can print and label 15 million of these targeted  replacement mailing packages in 4 days. Under this schedule, the Bureau  will finish mailing out the replacement questionnaires on April 8, whereas  under the original plan the replacement questionnaire mail out would not  be completed until April 19. As a result, more households should be able to  return the replacement questionnaires in time to be removed from the  nonresponse follow-up workload.
Two-column bilingual form. The Bureau plans to use a two-column  bilingual form in certain locations in 2010. The two-column bilingual form  provides two response columns, one in English and one in Spanish. Each  column contains the same questions and response options, and  respondents are instructed to choose the language that is most  comfortable for them. The Bureau tested the bilingual form in 2005 to  determine whether it would (1) increase overall response to the census  and (2) lower item nonresponse (when a household returns the form but  does not respond to a particular question) when compared to the standard  English form. In the 2005 test, the two-column bilingual form panel  demonstrated a significant positive effect on mail response rates of 2.2  percentage points overall, and a 3.2 percentage point increase in areas  with high concentrations of Hispanic and non-White populations, but did  not achieve lower rates of item nonresponse. The Bureau conducted  another test of the two-column bilingual form in 2007, which produced  response rate results similar to the 2005 results. In 2010, the Bureau plans  to mail the two-column bilingual form to households in communities with  heavy concentrations of Spanish speakers and areas with low English  proficiency.
Telephone reminder call. The Bureau does not plan to use a telephone  reminder call in 2010. The multiple contact strategy that the Bureau used  in 2000 included a reminder postcard mailed 1 week after the initial  questionnaire packets were mailed. As part of the 2003 National Census  Test, the Bureau tested the effect of an automated telephone reminder call  in place of a reminder postcard on increasing the mail response rate. Out  of a sample of 20,000 households, the Bureau was able to obtain phone  numbers for 6,208 (31 percent), and these households received telephone  calls to remind them to return their questionnaires if they had not already  done so. The initial results indicated a significantly higher cooperation rate  for the telephone reminder call panel when compared to the control panel,  which received reminder postcards.
The Bureau later conducted a supplementary analysis to determine if the  higher cooperation rate was related to an underlying higher propensity to  cooperate among households with listed telephone numbers. This analysis  compared cooperation rates of two groups of households: those for which  telephone numbers were available and those for which telephone numbers  were not available. Among households for which a telephone number was  available, the Bureau observed no significant difference in cooperation  rates for housing units that received a reminder postcard compared to  those that received a reminder telephone call. The Bureau does not plan to  use a telephone reminder call in place of a reminder postcard in the 2010  Census.
Due date on initial questionnaire mailing package. The Bureau will  not place a due date on the initial questionnaire mailing package as part of  the 2010 Census design. The value of placing a due date on the mailing  package with the initial questionnaire is that it might evoke a sense of  urgency or importance in respondents, leading to increased response to  the census. In 2003, when the Bureau tested this method, it found that the  inclusion of a due date did not significantly affect the overall response  rate. However, it did result in a significantly higher response rate at an  earlier point in the test, which can decrease the size of the replacement  questionnaire mailing.
The Bureau tested a due date again in 2006, this time combined with a  “compressed schedule,” in which questionnaires were mailed 14 days (as  opposed to 21 days, as in the control group) before Census Day. A due  date for mailing the questionnaire back was given in the advance letter, on  the outgoing envelope and cover letter for the questionnaire mailing, and  on the reminder postcard. The final report of the test concluded that the  inclusion of the due date and compressed schedule resulted in a  significantly higher mail response rate, by 2.0 percentage points. In  addition, several measures of data completeness and coverage showed  significant improvement. Bureau officials decided not to include the use of  a due date in the 2010 Census, noting that they would like to conduct  further testing to better understand the effects of a due date versus a  compressed schedule. The Bureau is considering whether to test both the  use of a due date and a compressed schedule in its 2010 Census Program  for Evaluations and Experiments.
Messaging to distinguish the replacement from the initial  questionnaire. The Bureau does not plan to use a message to distinguish  the replacement questionnaire from the initial questionnaire in 2010. The  inclusion of a message on the replacement questionnaire informing  respondents that they did not need to return the replacement  questionnaire if they had already provided a response from the previous  mailing was thought to be a way to reduce the number of multiple returns,  and possibly improve overall response. The Bureau tested this method in  2005, and while the results showed a significant decline in the submission  of multiple returns, they also showed a significant decrease in the  response rate of 1.2 percentage points. Including a message to distinguish  the replacement from the initial questionnaire was not recommended for,  and will not be included in, the 2010 Census.
subsections: section_title: Methods to Increase Self-Response Using Electronic Data Collection Systems
paragraphs: Interactive voice response. The Bureau does not plan to use interactive  voice response in 2010. Interactive voice response allows respondents to  use the telephone to respond verbally to digitized voice files that contain  census questions and instructions. Speech recognition software is used to  determine and record responses. Interactive voice response was tested in  2000, with households given the choice of providing their census data via  interactive voice response or a paper questionnaire. The response rate for  the interactive voice response panel was not statistically different from  that of the control (mail only) group. However, the results are difficult to  interpret because a portion of the sample in the interactive voice response  panel either received the census form late or did not receive it at all.
In 2003, the Bureau tested two different strategies for getting respondents  to provide their census data using interactive voice response. One strategy,  known as a push strategy, did not include a paper questionnaire in the  initial mailing but rather pushed respondents to reply using interactive  voice response. The other strategy, known as a choice strategy, included  an initial paper questionnaire with the interactive voice response  information on it, allowing respondents to choose their response mode. All  nonrespondents received a paper replacement questionnaire. Households  that had a choice of responding via paper or interactive voice response  had overall response rates similar to households that only received paper,  with about 7 percent of respondents given the choice using interactive  voice response. Households in the push strategy had a significantly lower  response rate—4.9 percentage points lower than households that only  received a paper questionnaire. The Bureau does not plan to give  respondents the option of providing their census data using interactive  voice response in 2010.
Letter encouraging Internet response or returning initial  questionnaire instead of sending replacement questionnaire. The  Bureau will not send a letter encouraging Internet response or returning  the initial questionnaire instead of sending a replacement questionnaire in  2010. Instead of mailing a replacement paper questionnaire to  nonrespondents, a letter could be sent to households encouraging them  either to complete the paper questionnaire that they previously received or  to use the Internet to submit their responses. The Bureau tested this  method in 2005, sending a replacement mailing that contained a letter with  an Internet address and an identification number needed to access the  Internet questionnaire in place of a paper replacement questionnaire.  Compared to sending out a paper replacement questionnaire, this method  resulted in significantly fewer responses overall, decreasing the response  rate by 3.7 percentage points. The Bureau will not include a letter  encouraging the use of the Internet instead of a replacement questionnaire  in 2010, in part because the Internet option was dropped from the 2010  Census design.
Computer-assisted telephone interviewing. The Bureau will not use  computer-assisted telephone interviewing for increasing response in 2010.  Computer-assisted telephone interviewing allows respondents to use the  telephone to connect with operators, who conduct interviews and record  responses electronically. In a 2000 experiment, households were given the  option of returning a paper form or providing their census data via  computer-assisted telephone interviewing. When compared to households  that were only given paper questionnaires, computer-assisted telephone  interviewing brought about a 2.06 percentage point improvement in the  overall response rate and also had a low item nonresponse rate. However,  it entailed substantial costs for hardware, software, and programmer and  interviewer time. The Bureau has not tested computer-assisted telephone  interviewing since 2000 and does not plan to use this option in 2010.
Internet. The Internet response option allows respondents to use an  Internet-based questionnaire—with screens designed to resemble the  paper questionnaire—to respond to the census. Respondents answer  multiple-choice questions by clicking the appropriate buttons and  checkboxes and text-entry questions by typing their answers into response  fields. The Bureau gave respondents the option to respond by Internet in a  2000 experiment. Some households in the test were given the choice of  providing their census data via Internet or a paper questionnaire. The  experiment found that the Internet response option resulted in a 2.46  percentage point increase in response.
In 2003, the Bureau again tested allowing respondents the option of  providing their answers via the Internet by sending a paper questionnaire  along with instructions for responding via the Internet in the initial  mailing. Households that had a choice to respond by paper or the Internet  had a similar overall response rate to households that were provided only  paper, with about 10 percent of respondents choosing to respond by  Internet.
The Bureau decided not to include the Internet in the 2010 Census, despite  including it in the scope of the contract awarded in 2005 for the Decennial  Response Integration System. The Bureau noted that this decision was  based on a number of factors, including the Bureau’s underestimation of  the contractor costs for the first 3 years of the contract, as well as test  results that indicated that the Internet would not increase the response  rate and concerns about the security of respondents’ data prior to the  Bureau receiving it.
Communications campaign. Census 2000 included a greatly expanded  outreach and promotion campaign—including, for the first time, paid  advertising—in an attempt to increase public awareness of and promote  positive attitudes about the census. This program, called the Partnership  and Marketing Program, was considered a success after the Bureau  reversed the trend of declining mail response rates, and the Bureau made  plans to continue the program for 2010. Bureau officials stated that the  2010 campaign consolidates all census communications under a single  communications contract. This Integrated Communications Campaign  aims to motivate the entire populations of the 50 states, the District of  Columbia, Puerto Rico, and other U.S. territories to participate in the  census through partnerships (with community groups, businesses,  colleges, faith-based organizations, and other targeted groups); public  relations; events; Census in Schools; and paid advertisements in broadcast,  print, and online media.
Bureau officials noted that the advertising campaign would not be  included in the dress rehearsal because the Bureau’s experience including  the advertising campaign in the 1998 Dress Rehearsal did not provide the  feedback needed to revise the creative aspects of the campaign. To refine  the communications campaign, the Bureau conducted an audience  segmentation analysis to identify how to best reach people with the paid  advertising campaign. In addition, the Bureau conducted focus groups in  2006 and 2007 to provide information on what motivates individuals to  respond.
The communications campaign is scheduled to run from mid-2008 through  June 2010. In 2008 and 2009, most of the activities are focused on  preparing and mobilizing partnerships. Further development of specific  messages for various audiences and communication channels will take  place from November 2008 through April 2009. Starting in mid-2009, the  partnership program will begin outreach to certain hard-to-count  populations. Activities and events targeting all audiences will begin in  January 2010 and peak in March 2010.
section_title: Appendix III: Comments from the Department of Commerce
section_title: Appendix IV: GAO Contacts and Staff Acknowledgments
subsections: section_title: Acknowledgments
paragraphs: In addition to contacts named above, Lisa Pearson, Assistant Director;  David Bobruff; Don Brown; and Elizabeth Fan made key contributions to  this report. Tom Beall, Susan Etzel, Andrea Levine, Donna Miller, Ellen  Rominger, and Elizabeth Wood provided significant technical support.