2010 Census: Overseas Enumeration Test Raises Need for Clear Policy Direction
Released: 2004-06-21T00:00:00

Letter

Background

The Census Bureau has discretion under the Constitution and federal statutes to decide whether to count Americans residing overseas. The federal decennial census is conducted pursuant to the requirement imposed by Article I, Section 2, Clause 3 of the Constitution, and Section 2 of the Fourteenth Amendment, that Congress enumerate “the whole number of persons in each State” as the basis for apportionment of seats in the United States House of Representatives. Under the Constitution, the census is to be conducted every 10 years “in such Manner as shall by Law direct.” Congress has exercised its authority under the Constitution by passing the Census Act, which assigns to the Secretary of Commerce the responsibility of “tak a decennial census of population as of the first day of April” of each census year. The Secretary does so with the assistance of the Census Bureau and its Director.

The statutes governing the earliest censuses provided that enumerators should record all persons reported to them within their respective districts as having a usual place of abode there or as usually residing within that district, even though such persons might be “occasionally absent at the time of enumeration.” Statutes governing later censuses, including the current provisions in Title 13 of the United States Code, contain no similar provision, or any provision specifically governing the enumeration of inhabitants of the United States who are outside of its borders on the enumeration date.

To determine who should be included in the census, the Bureau applies its “usual residence rule,” which has been defined as the place where a person lives and sleeps most of the time. People who are temporarily absent from that place are still counted as residing there. One’s usual residence is not necessarily the same as one’s voting residence or legal residence. Noncitizens living in the United States are counted in the census, regardless of their immigration status.

Historically, the census has focused primarily on the domestic population and typically has not included any procedures designed to enumerate Americans residing outside of the United States. The first attempts to count Americans residing overseas were in the 1830 and 1840 censuses, which included procedures for counting the “crews of naval vessels at sea.” The naval personnel included in those censuses, however, were not allocated to any individual state, and thus were not included in the apportionment population.

As shown in table 1, various overseas population groups were included in the census at different times. For example, while federally affiliated personnel were typically included in the enumerations that took place from 1900 through 2000, only the 1970, 1990, and 2000 censuses used the numbers for purposes of apportioning Congress. At the same time, private citizens living abroad were included only in the 1960 and 1970 censuses, but not for purposes of apportionment.

In response to congressional direction and the concerns of various business, political, and other groups that represent overseas Americans, the Census Bureau embarked on a research and evaluation program aimed at determining the feasibility, quality, and cost of counting both federally affiliated and private citizens living abroad. The test enumeration began February 2004 and is to run through July 2004 at three sites: France, Kuwait, and Mexico.

The Bureau selected these countries based on several criteria including their geographic diversity, the fact that large numbers of U.S. citizens reside there, and because of the existence of administrative records that can be used to compare to the test census counts for evaluation purposes. The Bureau estimated the implementation costs for the 2004 test at approximately $2.5 million in fiscal year 2004. Further, the Bureau estimates that by the end of fiscal year 2004, it will have spent an additional $3.5 million for planning and preparation during fiscal years 2003 and 2004.

Americans can participate in the test census by completing a short-form paper questionnaire that is available at embassies, consulates, and other organizations that serve overseas Americans, or by completing the form on the Internet. The Bureau hired a public relations firm to develop a communications strategy to inform and motivate respondents living in the selected countries to answer the census. Responses from the paper and the Internet returns will be captured in order to analyze, among other things, the demographic characteristics of respondents and patterns of item nonresponse. The Bureau plans to conduct a 2006 overseas test if Congress appropriates requested funds in fiscal years 2005, 2006, and 2007. If Congress then indicates its desire that the Census Bureau conduct a general overseas enumeration in 2010, the Bureau will seek a supplementary appropriation in calendar year 2007 for that purpose and to conduct a 2008 overseas dress rehearsal beginning in 2007.

A sound test is essential in order for the Bureau, Congress, and other stakeholders to resolve the numerous logistical, conceptual, policy, and other questions that surround the counting of overseas Americans. They include: Who should be counted? U.S. citizens only? Foreign-born spouses?

Children born overseas? Dual citizens? American citizens who have no intention of ever returning to the United States? Naturalized citizens?

How should overseas Americans be assigned to individual states? For certain purposes, such as apportioning Congress, the Bureau would need to assign overseas Americans to a particular state. Should one’s state be determined by the state claimed for income tax purposes? Where one is registered to vote? Last state of residence before going overseas? These and other options all have limitations that would need to be addressed.

How should the population data be used? To apportion Congress? To redistrict Congress? To allocate federal funds? To provide a count of overseas Americans only for general informational purposes? The answers to these questions have significant implications for the level of precision needed for the data and ultimately, the enumeration methodology.

How can the Bureau verify U.S. citizenship? Administrative records such as passports and Social Security data have limitations. For example, Americans can reside in Mexico and Canada without a passport and many Americans overseas do not have Social Security numbers, especially dependents.

How can the Bureau ensure a complete count without a master address list? The foundation of the stateside decennial census is a master address list. Because the list is essentially the universe of all known living quarters in the United States, the Bureau uses it to deliver questionnaires, follow up with nonrespondents, determine vacancies, and determine individuals the Bureau may have missed or counted more than once. The Bureau lacks a complete and accurate address list of overseas Americans. Consequently, these operations would be impossible and the quality of the data would suffer as a result.

Can administrative records be used to help locate and count overseas Americans? Administrative records such as passport and visa files, voter registration forms, as well as records held by private companies and organizations have the potential to help the Bureau enumerate Americans abroad. However, the accuracy of these records, the Bureau’s ability to access them, confidentiality issues, and the possibility of duplication all remain open questions.

Do certain countries have requirements that could restrict the Bureau’s ability to conduct a count? According to the Bureau, in planning the overseas test, the Bureau was informed that French privacy laws prohibit asking about race and ethnicity, two questions that are included on the U.S. census questionnaire. Although the Bureau worked with French officials to address this problem, the extent to which the Bureau will encounter restrictions in other countries, or whether other countries will cooperate with the Bureau at all, is unknown.

Scope and Methodology

As agreed with your offices, our objectives for this report were to (1) assess the soundness of the Bureau’s test design and its suitability for addressing the Bureau’s specific research questions, and (2) examine what past court decisions have held about Americans’ rights and obligations abroad that could help inform whether and how they should be included in the census.

To assess the soundness of the Bureau’s 2004 overseas enumeration test design, we interviewed knowledgeable Bureau officials and reviewed existing documents that described the Bureau’s test objectives, research questions, and test plans. We then systematically rated the Bureau’s approach using a checklist of over 30 design elements that, based on our review of program evaluation literature, are relevant to a sound study plan. For example, we reviewed the Bureau’s approach to determine, among other things, (1) how clearly the Bureau presented the research objectives, (2) whether the research questions matched the research objectives, (3) whether potential biases were recognized and addressed, and (4) the appropriateness of the data collection strategy for reaching the intended sample population. See appendix II for a complete list of the 30 design elements.

We supplemented our ratings on the suitability of the test by gathering additional information through telephone and in-person interviews with representatives of several stakeholder organizations that represent various groups of Americans residing abroad. The organizations included Democrats Abroad, Republicans Abroad, Association of Americans Resident Overseas, and the American Business Council of the Gulf Countries. In addition, we interviewed representatives of the Mexican American Legal Defense and Educational Fund, National Coalition for an Accurate Count of Asians and Pacific Islanders, and California Rural Legal Assistance, Inc., to get their perspectives on the Bureau’s plans for counting American citizens living in Mexico, particularly migrant and seasonal farm workers, a group that the Bureau had trouble counting during the 2000 Census. These three organizations, while not actively involved in the planning of the overseas enumeration test, are members of the Secretary of Commerce’s Decennial Census Advisory Committee, a panel that advises the Bureau on various census-related issues.

To examine what past court decisions have held about Americans’ rights and obligations living abroad, including their right to be counted in the census, we reviewed a judgmental selection of five federal laws and/or programs that cover large numbers of Americans stateside, in order to determine how those laws and programs treat U.S. citizens should they live overseas. We examined federal election law, federal income tax law, and federal laws relating to Social Security benefits, Supplemental Social Security Income, and Medicare. For each of these laws and programs, we reviewed relevant statutes, court decisions, and legal analyses.

We requested comments on a draft of this report from the Secretary of Commerce, which were sent to us April 13, 2004 (see app. I). We address them in the Agency Comments and Our Evaluation section of this report.

Design Limitations Could Undermine the Usefulness of the Overseas Test

According to the Bureau, its objectives for the 2004 overseas test are “to determine the feasibility, quality, and cost of collecting data from U.S. citizens living overseas.” To meet those objectives, the Bureau developed eight research questions designed to gather data on such salient factors as participation levels, data quality, and the relative response from the two enumeration modes (Internet and paper questionnaire; see appendix III for a sample of the paper questionnaire). To assess the overseas test the Bureau is planning to complete a series of evaluations due in early 2005. The test objectives and related research questions are appropriate as written, but, as shown in table 2, because of various methodological limitations, the data that will result from the test will not fully answer key questions concerning feasibility, data quality, and cost. In short, the Bureau overstated the research test’s ability to answer its key research objectives and, as a result, congressional decision making on this issue will be that much more difficult.

Research Question #1: Participation Data Could be Unreliable

For the decennial census, the Bureau uses participation data as a key indicator of public cooperation with the census. The Bureau measures participation levels by what it calls the “return rate,” which it calculates as a percentage of all forms in the mailback universe (excluding vacant and nonexistent housing units) from which it receives a questionnaire.

Stateside, the Bureau is able to perform this calculation because, as noted above, it has a master address list of all known housing units in the United States, an inventory that takes the Bureau several years and considerable resources to compile.

However, no such address list exists for overseas Americans. Consequently, participation rates for overseas Americans cannot be calculated and the Bureau will only be able to tally the number of responses it receives (both overall for each site and within several demographic categories) and compare the results to counts obtained from administrative records. The sources for the records include a combination of tax, Medicare, and State Department data, as well as foreign census data if available. For a variety of reasons—some of which the Bureau has already acknowledged—the information generated from this exercise may not be relevant.

First, the administrative records were developed for different purposes, and as a result, are not well suited as a base for comparing against overseas counts; thus, their relevance is uncertain. As the Bureau has already reported, each of the records it plans to use to compare to the census counts has coverage limitations. Further, as each of these records is associated with particular demographic groups, they could introduce systemic biases (we discuss potential problems with administrative records in greater detail below, under Research Question #5).

Second, the census counts could be problematic because it is unclear who should participate in the overseas census, which in turn could confuse potential respondents. For the stateside enumeration, to determine where an individual should be counted, the Bureau uses the concept of “usual residence,” which it defines as “the place where a person lives and sleeps most of the time.” The Bureau has developed guidelines, which it prints on the stateside census questionnaire, to help people figure out who should and should not be included. However, the Bureau has not developed similar guidance for the overseas test. According to the Bureau, this was intentional; because this was an initial feasibility test, the Bureau did not want to restrict response, but rather to encourage the widest possible participation.

Further, the guidance the Bureau has developed, which is available on its Web site and promotional literature, is vague and could confuse potential respondents. For example, the guidelines inform potential respondents that, “All U.S. citizens living in France, Mexico and Kuwait, regardless of shared citizenship, can and should participate in the test. U.S. citizens on vacation or on short business trips should not.” Unclear is what constitutes a short or long business trip. Is it 3 weeks or 3 months? Does it matter whether one stays in a hotel or an apartment? Also, should naturalized U.S. citizens, some of whom may not return to the United States, participate? What about children born in the United States to noncitizens, but who only lived in the United States a short time? Should students spending a semester abroad but who maintain a permanent residence stateside be included? Without clear residence rules and appropriate guidance indicating who should be counted, it is quite possible that some people might inappropriately opt in or out of the census, which would reduce the quality of the data. Bureau officials have told us that they are working to develop residence rules that it will apply if there is a second overseas enumeration test in 2006.

Participation data might also be problematic because the Bureau’s enumeration methods strategy might not be as effective with certain groups compared to others. To the extent this occurs, it could introduce a systemic undercount. This is particularly true for dual nationals who, for cultural reasons, may not think of themselves as American citizens.

For example, according to representatives of two advocacy groups we contacted, Mexican dual nationals include migrant farm workers, a group that often consists of poor, less-educated people living in rural areas. They noted that this population group has low literacy levels and thus might not understand the questionnaire, and is not likely to have Internet access. At the same time, they are not likely to pick up a copy of the questionnaire at an embassy. Further, the barriers that make it difficult to count migrant farm workers in the United States, such as a distrust of government and the fact that they may speak indigenous languages, also make it difficult to count this group in Mexico.

Research Question #2: Information on Data Quality Will be Limited

The Bureau plans on measuring the quality of the data collected in the overseas test by tabulating item nonresponse, which refers to whether a respondent completed a particular question. The Bureau is to calculate this information by enumeration mode, test site, and various demographic categories. The Bureau also plans to compare this measure of nonresponse for key variables to those obtained in an earlier, stateside test held in 2003 by tabulating the rate respondents did not complete a particular question.

According to the Bureau, patterns of item nonresponse are critical for improving question design, training, and procedures. However, as the Bureau acknowledges in its study plan for evaluating the quality of the overseas enumeration data, item nonresponse by itself does not address the quality of the data. Thus, at the end of the test, the Bureau will have, at best, only limited information on the quality of the overseas data.

By comparison, the Bureau’s guidelines for measuring data quality in other surveys they conduct use measures such as coverage, unit response rates, imputation rates, and data collection errors. Because the Bureau lacks information on the universe of overseas Americans it will be unable to calculate these indicators. Therefore, it is misleading for the Bureau to state in its research objectives that it will determine the quality of the overseas data, when in fact it will deliver something far more limited. Comprehensive measures of data quality are critical because they could help Congress decide whether the data are sufficiently reliable to use for specific purposes. If the numbers were to be used to obtain a simple count of Americans abroad, absolute precision is not as critical. However, for other uses of the data, particularly congressional apportionment and redistricting, the quality of the data would need to be far higher.

Research Question #3: The Bureau Will be Unable to Geocode Certain U.S. Addresses

Counting people in their correct locations is essential for congressional apportionment, redistricting, and certain other uses of census data. With respect to Americans abroad, if the data are to be used for apportionment, the Bureau would need to assign respondents to a specific state. For purposes of redistricting and allocating federal funds, the Bureau would need to assign overseas respondents to specific neighborhoods and street addresses in the United States—a far more challenging task. Geocoding is the process of linking an address in the Bureau’s Master Address File (MAF) to a geographic location in the Bureau’s geographic database, known as the Topologically Integrated Geographic Encoding and Referencing (TIGER) mapping system. To obtain this information, the overseas enumeration form asks respondents to indicate their U.S. state of last residence, and their last street address within that state.

Although the Bureau is assessing the level of geography to which it can geocode the overseas population, an important limitation is that the Bureau will not be able to make this assessment for people who live in certain noncity-style U.S. addresses; that is, U.S. addresses without a housing number and/or street name. Specifically, people whose U.S. addresses consist of a post office box will be excluded from the study. For the stateside enumeration, Bureau employees canvass the country, identify noncity-style addresses, and mark the locations of those residences on a map. During the 2000 Census, around 20 percent of U.S. households had noncity-style addresses.

Indeed, the overseas enumeration questionnaire instructs respondents not to provide a post office box number for their last stateside address. However, if a respondent’s address includes a post office box or rural route number, it is unclear how they are supposed to complete this question. Also unclear is how migrant farm workers, who may not have had an address in the United States, would complete this question.

To the extent they leave the question blank, the Bureau would be unable to distinguish between those people who did so because they have a noncity- style address, or left it blank for privacy or other reasons. This could affect the accuracy of the Bureau’s assessment. Moreover, the construction of the question could introduce a systemic bias because those states with large rural areas are more likely to have noncity-style addresses.

The Bureau’s evaluation plan recognizes that respondents in the 2004 overseas enumeration test may provide noncity-style addresses which cannot be geocoded by the TIGER system. The Bureau intends to provide data on how many city-style and noncity-style addresses could be geocoded.

Research Question #4: Measuring the Marketing Program’s Effectiveness Will be Difficult

Although the Bureau has taken a number of steps to publicize the overseas enumeration, evaluating the effectiveness of that effort will present a challenge. The Bureau awarded a $1.2 million contract to a public relations firm to develop a promotion strategy for the overseas enumeration test. As part of that effort, the public relations firm identified a number of stakeholder organizations that represent U.S. citizens living overseas in each of the three test countries. The organizations included advocacy groups, universities, church groups, and corporations.

The Bureau anticipates that these stakeholders will help get the word out via e-mail, newsletters, and other media that a test census of Americans overseas is underway. In addition, the Bureau is to provide copies of the overseas questionnaire to stakeholders so that they can distribute them to their members and constituents. As noted earlier, questionnaires will also be available on the Internet, as well as at public places that Americans may visit, such as embassies and consulates. The Bureau has produced posters and pamphlets to promote the test (see fig. 1). The Bureau also plans to have articles about the census test placed in newspapers and magazines and stories run on local television and radio. Although paid advertising was not part of its original plans, the Bureau later decided to run a limited amount of paid advertising in Mexico and France.

Fill out a U.S. Census questionnaire, available It’s easy, important — and confidential.

The Bureau will attempt to gauge the effectiveness of the marketing program by measuring participation (as measured by the number of responses) and public awareness. According to the Bureau, respondents who submit a questionnaire via the Internet will be asked to complete a short survey eliciting information on how they learned about the census test and what motivated them to participate. No similar survey is planned for people who pick up their surveys at an embassy or other distribution site. Thus, the Bureau will not have a parallel set of data from a group of respondents that might be demographically or behaviorally different from Internet respondents.

The Bureau does, however, expect to conduct focus group interviews and debriefings to obtain feedback from mail respondents and stakeholder organizations. Focus group interviews targeting nonrespondents are planned as well.

Yet, as the Bureau acknowledges, participation, or the final count of U.S. citizens living in the selected countries, will only be an indicator of the number of people that heard about the test, completed the questionnaire, and submitted it to the Census Bureau. It will not be able to measure the Bureau’s success in getting Americans to respond because the universe of Americans overseas is unknown.

Public awareness will also be difficult to measure because it includes an unknown number of people who were aware that a test census was being conducted but chose not to respond. Nevertheless, as noted above, the Bureau intends to interview both respondents and nonrespondents in an effort to determine their awareness and motivation for responding or not responding to the census test. To the extent the Bureau conducts these interviews, it will be important for it to include hard-to-count groups, such as dual nationals and migrant farm workers, that may have been outside the reach of the Bureau’s marketing campaign.

Research Question #5: Use of Administrative Records Needs to be Thoroughly Tested

The Bureau plans on using administrative records such as Medicare and passport data to provide comparison information to assess (1) participation, (2) an invalid return detection system, and (3) the records’ potential use for building an address list. Specifically, the Bureau plans to compare the number of people counted at each site to federal tax, Medicare, U.S. Department of State, and foreign census records. While it is important for the Bureau to assess the utility of administrative records, it is unclear from the Bureau’s study plans how it will make this determination given what Bureau officials have said is a large disparity between administrative record counts on the number of Americans living overseas at the three test sites.

Further, because these administrative records were developed for different purposes and as a result are not well suited as a base for comparing against overseas counts, their relevance is uncertain. For example, not all American citizens who live abroad file tax returns; dependents are not always listed on tax returns; and dependents that are included in the tax form may not be U.S. citizens. After living abroad 30 days, Americans are no longer eligible for Medicare benefits; therefore, Medicare records may not be the most useful and only apply to U.S. citizens over 65 years old. U.S. State Department records are nonofficial and according to the Bureau inaccurate because the Department of State does not officially track either the number or location of U.S. citizens living in other countries. Finally, the type of administrative records kept by each country is unknown and earlier Bureau research found that census data from foreign countries do not contain the detailed information required for apportionment, redistricting, or other census uses.

Research Question #6: Effectiveness of Enumeration Methods Will be Difficult to Assess

To determine the relative response from the two enumeration modes and their relative effectiveness, the Bureau will look at such data as the timing of returns by mode and site, and whether one mode provided more valid returns than the other. Respondents can either fill out a paper questionnaire or complete the form on the Internet. The Bureau will deploy an invalid return detection system to determine whether a form is valid and responses will be tallied by mode and by country.

However, the Bureau recognizes that its analysis will contain several limitations. Key among them is that it will not be able to determine participation rates because the universe of Americans overseas is unknown. As a result, the analysis is limited to a count of the total number of forms returned at each site. In light of this and other limitations, it is unclear what conclusions the Bureau will be able to draw about the effectiveness of the two response modes.

Indeed, just because one enumeration mode results in a larger number of returns from a particular demographic groupolder Americans, for examplemay not necessarily have anything to do with the mode itself. As a result, it would be inappropriate to say that one enumeration method was more effective than the other in counting senior citizens. This is because there are other factors that can influence the mode such as advertising or accessibility to the Internet or the paper questionnaire. Overall, while it will be important to collect information on the returns by method of enumeration, this information should not be construed as a measure of the effectiveness of that mode.

Research Question #7: System to Detect Invalid Returns Has Shortcomings

Part of the processing of overseas returns involves validating that the respondents are within the scope of the enumeration; that is, that they in fact reside in one of the three overseas test areas. Thus, the Bureau needs a better method to detect invalid returns.

To determine whether responses are within scope, the overseas enumeration questionnaire asks respondents to provide their Social Security and U.S. passport numbers. Although we agree with the importance of determining whether respondents reside overseas, the Bureau’s analysis will not assess this. Rather, the Bureau defines a valid return as one where at least one person in the household checked the U.S. citizen box on the questionnaire, provided a valid Social Security or U.S. passport number, and has been subject to and passes an algorithm that analyzes data from the questionnaire. All other returns are invalid. Thus, what the Bureau is really measuring is whether a questionnaire is eligible for further processing, and not whether the respondent lives abroad. Put another way, anyone who completes a questionnaire with valid data, including Social Security numbers, would be considered a valid return, regardless of whether the individual lived in the test areas.

This is not an unlikely scenario given the way the Bureau set up its Internet site. Indeed, anyone—even if they live outside of the three test sites-can be included in the overseas count, so long as they provide the required information. The reason they can slip through the invalid return detection system is because the Bureau is unable to confirm the point of origin for questionnaires completed on the Internet. The Bureau is aware of this gap in the invalid return detection system, but has been unable to resolve this condition.

Another potential problem is the Bureau’s sole reliance on Social Security numbers to validate returns for the 2004 test. The Bureau had also wanted to use passport numbers to validate returns. Although the Bureau has been negotiating with the Department of State for access to the passport database, the Bureau does not expect this to occur in time for use in the 2004 test. Bureau officials said they were aware that there was a strong possibility that they would not be able to gain access to the passport file because the Bureau had not worked with Department of State data and that a memorandum of understanding would first need to be in place before the data would be released. Nevertheless, the Bureau believes that the impact of this would be low if it could be shown that using Social Security numbers alone to validate returns was sufficient.

A third potential problem is that some people may be reluctant to provide their Social Security and passport numbers for privacy reasons. Based on the Bureau’s research, requesting this information could reduce participation levels.

Research Question #8: Overseas Enumeration Will Not be Integrated With the Stateside Enumeration

The Bureau, based on its earlier research, has already identified a list of barriers to integrating overseas with stateside data. They include different questionnaire content for the overseas form, detecting and eliminating duplication within and between overseas and stateside enumerations, timely geocoding of addresses, and limited resources.

The Bureau plans to document the lessons learned from the 2004 overseas test and how they might apply to a more integrated test in 2006. In particular, the Bureau is to focus on the issues encountered or associated with collecting, capturing, and processing overseas data. While it will be important for the Bureau to thoroughly document these issues and their implications for integrating the two data sets, the Bureau does not intend to actually integrate any data from the overseas test with data being collected from a parallel stateside test it’s conducting at three Georgia counties and in Queens, New York. Without an actual integration, the Bureau may miss problems that will not likely be detected until a next test in 2006.

Overseas Test Design Has Other Shortcomings

In addition to the limitations noted above, the overseas enumeration test has other limitations that will affect the Bureau’s ability to answer its key research objectives.

Cost Data Will Not Identify the Cost of Conducting Future Tests or an Overseas Enumeration in 2010

Although one of the Bureau’s objectives for the overseas enumeration test is to determine the cost of collecting data from overseas citizens, the Bureau’s test design lacked a specific research question aimed at obtaining this information. More importantly, the cost information that the Bureau will collect will be of limited value because it will not be used to estimate the costs of future tests, nor model the costs of conducting a broader overseas enumeration in 2010.

The Bureau developed a cost model for the 2000 Census that provided the agency with an automated means to estimate staffing and budget requirements. The Bureau used the cost model to support the budget process, as well as to answer questions from Congress, the Office of Management and Budget (OMB), and our office. The cost model could also estimate the budgetary impact of certain assumptions and alternative census-taking scenarios.

The Bureau maintains that it would need more data than those that are now available to develop a cost model for an overseas enumeration in 2010. Nevertheless, while only at the beginning stages, the Bureau has some data points to begin developing a cost model for overseas enumeration or, at a minimum, for identifying major cost components. For example, Bureau officials told us that their agency will track and report the marketing, printing, postage, data capture, and processing costs for conducting the overseas enumeration at the three test sites. All of the costs provided by the Bureau are direct costs and could be specifically and uniquely attributed to a cost model for the overseas enumeration. Indirect costs, however—those that are not easily attributable to the overseas enumeration such as executive management or technical labor—would still need to be determined and captured.

Cost will be an important factor to consider when making a decision on whether to enumerate Americans overseas. The cost of the 2010 Census, now estimated at more than $11.3 billion in current dollars, is the most expensive enumeration in the nation’s history. Consequently, it will be critical for the Bureau to have a mechanism for accurately and quickly estimating overseas enumeration costs so that Congress, other oversight groups, and the Bureau itself can have reliable information on which to base or advise decisions.

Testing Questionnaire Would Help Ensure Clarity and Accurate Responses

Testing a questionnaire with a sample of intended respondents before it is distributed is a standard approach that survey organizations employ to ensure questions are clear and understandable, and that respondents will be able to provide accurate information. In short, testing is an important quality assurance procedure that increases the likelihood that respondents will provide the information needed and help reduce the likelihood of inaccurate responses.

The Bureau is aware of the benefits of testing questionnaires. In preparation for the 2000 Census, Congress budgeted millions of dollars for the Bureau to develop and test questionnaires during the 1990s, which it did using focus groups and one-on-one interviews. Moreover, the Bureau’s policy requires that demographic survey questionnaires be tested.

However, the Bureau did not test the overseas enumeration questionnaire. Instead, the Bureau gave stakeholders the opportunity to review and comment on the questionnaire before it went to OMB for final approval. Stakeholder feedback was generally positive. One change that was made in response to stakeholder feedback was that “United States” was added to “passport” in the question that asked for a respondent’s passport number. While sharing the questionnaire with stakeholders is important, it should not be seen as a replacement of questionnaire testing.

According to Bureau officials, the Bureau developed the overseas questionnaire by modifying the Census 2000 short form to collect data needed for the overseas count. New questions asked respondents for their employment status, Social Security number, and passport number. The Bureau believed that the new questions did not require testing. However, as stated earlier, collecting Social Security numbers and passport numbers could be problematic. While some stakeholders believed this to be acceptable, other stakeholders believe it could reduce participation, especially in Mexico where dual nationals reside. In addition, the questionnaire requests data on everyone in the household even if a person is not a U.S. citizen. These questions could be seen as too intrusive and potentially could stop someone from completing the form, thus resulting in an undercount. Since there was no testing of the questionnaire, the Bureau cannot be certain of the impact of these questions.

Data Analysis Will Not Provide Congress with Requested Information

In a 2001 report to Congress, “Issues of Counting Americans Overseas in Future Censuses,” the Bureau indicated that it would provide Congress with data on the number of people in the military, federal, private business, nonprofit, and other categories. However, the Bureau is only collecting data on the number of military and federal workers; people working in other sectors will be grouped in an “other” category. The reason that the Bureau is unable to provide the additional breakdown is because the length of the questionnaire did not allow for additional check boxes. Therefore, the Bureau collapsed people working in the private sector and nonprofit organizations and others into one category.

The Bureau Has Identified a Number of New Challenges to Counting Overseas Americans

The soundness of the test design notwithstanding, the Bureau has already identified several country-specific challenges to counting American citizens at each of the test sites. Together, they suggest that an enumeration of Americans on a more global scale in 2010 would introduce a number of unforeseen obstacles that the Bureau would need to address on a country-by-country basis. For example, shortly before the test was to begin in France, the French government contacted the Census Bureau indicating that French law prohibited the collection of race and Hispanic origin data. Furthermore, they were also opposed to the U.S. government asking for information on persons who were not American citizens. The Bureau worked with the French government and it was agreed that an advisory would be posted on the Internet site explaining that under French law it was not mandatory to respond to the questionnaire.

Problems have also surfaced in Mexico and Kuwait. Stakeholders and Bureau officials have told us that the mail system in Mexico is not always reliable. The concern is that the questionnaires may not make it to the Census Bureau, or arrive too late to be processed. In Kuwait, security concerns have prevented the Census Bureau from posting the location of sites where Americans can pick up the questionnaire. While the impact of these problems is difficult to quantify, it may prevent some Americans from completing the questionnaire and being counted.

The Rights and Obligations of Americans Overseas Vary

Americans residing abroad do not have the same rights and obligations under federal programs and activities as Americans living in the United States. In order to determine the rights and obligations of Americans residing abroad, one must examine the specific statutes governing each program. For this study, we examined whether overseas Americans can vote in federal elections; are subject to federal income tax; and can receive Social Security, Supplemental Security Income, and Medicare benefits (see table 3). Stateside, these programs cover millions of Americans; whether and how they extend to Americans living overseas could help inform the treatment of U.S. expatriates in the decennial census, to the extent there are any patterns.

Voting Rights

American citizens who reside outside of the United States have the right to vote in federal elections under the 1986 Uniformed and Overseas Citizens Absentee Voting Act (UOCAVA). Under this law, U.S. citizens residing on foreign soil can vote in federal elections as absentee voters of their last state of residence, even if they have no intention of ever returning to the United States. (American citizens residing in U.S. territories, however, cannot so vote. The territories include Puerto Rico, the Northern Mariana Islands, the U.S. Virgin Islands, Guam, and American Samoa.)

UOCAVA repealed legislation enacted in 1955 that was designed to prevent members of the Armed Forces and their families from being denied their voting rights when absent from their home or in a far-off place. The goal was to make it easier for military personnel to cast votes in any federal primary, general, or special election through absentee balloting procedures. While the 1955 law was amended several times to, among other purposes, specifically include other Americans living overseas, in 1986 Congress acknowledged that there was a legitimate need for further legislation. UOCAVA’s main purpose was to facilitate absentee ballot voting, while also providing “for a write-in absentee ballot that may be used in Federal general elections by overseas voters who, through no fault of their own, fail to receive a regular absentee ballot in sufficient time to vote and return the ballot prior to the voting deadline in their State.”

Federal Income Tax

U.S. citizens are taxed on their worldwide income, subject to certain exclusions specified in the Internal Revenue Code. In 1913, the enactment of the Sixteenth Amendment to the U.S. Constitution gave Congress “the power to lay and collect taxes on income, from whatever source derived, without apportionment among the several States, and without regard to any census or enumeration.” After ratification of this constitutional provision, Congress imposed a tax on the net income of every U.S. citizen, wherever they lived.

That the Constitution vests Congress with the power to tax Americans living overseas on their income earned outside of the United States was reinforced by the Supreme Court in 1924 when it ruled: “overnment, by its very nature, benefits the citizen and his property wherever found, and therefore has the power to make the benefit complete. Or, to express it another way, the basis of the power to tax was not and cannot be made dependent upon the situs of property in all cases, it being in or out of the United States, nor was not and cannot be made dependent upon the domicile of the citizen, that being in or out of the United States, but upon his relation as citizen to the United States and the relation of the latter to him as citizen.”

There are statutory exclusions, however. Generally, a U.S. citizen or resident may exclude a portion (ascending each year to $80,000 during or after 2002) of his earned income if he is a resident of a foreign country for a full calendar year or is physically present in a foreign country for 330 days of any 12 consecutive months and otherwise meets certain requirements. This foreign earnings exclusion principally aims to encourage U.S. citizens and residents to work abroad without worrying about how it might increase their tax liability. Indeed, in view of the nation’s continuing trade deficits, Congress deemed it important to allow Americans working overseas to contribute to the effort to keep American business competitive.

Social Security Payments

The Social Security Act provides individuals over the age of 62 who have worked for a minimum number of years with monthly benefit payments. Social Security checks generally follow recipients wherever they go around the world, subject to only a few exceptions. In fact, Social Security payments continue no matter how long a beneficiary stays outside of the United Stateseven if the individual retires overseas. Thus, if you are a U.S. citizen residing abroad you generally continue to receive your monthly benefits.

Supplemental Security Income

Supplemental Security Income (SSI) is a “need-based” program designed to help individuals who are over the age of 65, blind, or disabled. SSI benefits cease once a recipient remains outside the United States for a period of at least 30 days. If the recipient returns to this country within 30 days, SSI benefits are to continue as before.

Medicare Coverage

Since Medicare became effective almost 35 years ago, the program has excluded coverage for health care services received outside of the United States, even if those services are medically necessary. There is a limited exception for services occurring near U.S. borders.

The constitutionality of foreign exclusion was raised in a 1986 court decision. In that case, plaintiffs argued that by leaving Medicare beneficiaries without health care coverage once they leave the United States, the foreign exclusion deters these same individuals from traveling overseas and consequently infringes on their constitutional right to travel abroad. The district court, however, found the Medicare foreign exclusion to pass constitutional muster. The court reasoned that the difficulties of administering medical services abroad and the concern that Medicare funds be spent within the United States were not particularly compelling, but were rationally based. It thus concluded that the foreign exclusion satisfied the rational basis test and summarily dismissed the plaintiff’s claim.

Legislation Would be Needed To Require the Census Bureau to Enumerate Americans Abroad

The Bureau has the discretion to decide whether to count American citizens abroad. Indeed, there is nothing in the Constitution, the Census Act, or court decisions that would require the Bureau to count Americans living overseas, or to not count such individuals. Consequently, if Congress wanted to require the Census Bureau to include this population in the 2010 Census, legislation would be needed.

The issue of whether, and if so how, to count Americans living overseas was raised in federal court after both the 1990 and 2000 censuses. Massachusetts challenged the results of the 1990 Census claiming that it lost a seat in the House of Representatives as a result of the Secretary of Commerce’s decision to enumerate and apportion federal employees stationed abroad. Conversely, Utah challenged the results of the 2000 Census maintaining that it lost a congressional seat because no overseas Americans other than federally affiliated groups were included in the apportionment numbers. In both cases, the courts determined that the Census Bureau has discretion under the Constitution and the Census Act to decide whether to count Americans residing overseas.

Conclusions

The design of the Bureau’s overseas enumeration test falls short in many respects, and the data that the Bureau will collect as a result of this exercise may only partially answer key questions relating to feasibility, cost, and data quality. The Bureau overstated the test’s ability to answer its key research objectives and, as a result, congressional decision making on this issue will be that much more difficult.

The full results of the overseas enumeration test will not be available until early 2005, when the Bureau expects to issue the last of a series of evaluations. However, its experience thus far has made it clear that counting Americans abroad as an integral part of the 2010 Census would be an extraordinary challenge, one that would introduce new resource demands, risks, and uncertainties to an endeavor that is already costly, complex, and controversial.

That said, to the extent that better data on the number and characteristics of Americans abroad might be useful for various policymaking and other nonapportionment purposes, such information does not necessarily need to come from the decennial census. This data could, in fact, be acquired through a separate survey or some other type of data collection effort, although it would still be a difficult undertaking.

The Bureau is unlikely to decide—and in fact should not decide—on its own, whether or not to enumerate Americans abroad, and will need congressional guidance on how to proceed. Therefore, to give the Bureau as much planning time as possible, it will be important for Congress to soon decide whether the Bureau should be required to count this population group as part of the 2010 Census or as part of a separate data collection effort or whether there are so many obstacles to a successful overseas count regardless of the approach that the Bureau should shelve any plans for further research and testing.

Should Congress desire an overseas count—as part of the decennial census or a separate effortit should consider providing the Bureau with input on how it expects to use the data on overseas Americans. The purposes of the data drive the design of the enumeration; therefore, once the Bureau has a clear idea on how the data will be used, it would be better positioned to plan a test that will more accurately assess the feasibility of the procedures, methodology, and resource requirements needed to accomplish the type of count that Congress desires. Moreover, if a second test of enumerating Americans abroad is needed in 2006, it will be important for the Bureau to address the shortcomings of the design of the 2004 overseas test. Conducting a second test without this information and a sound design to fulfill it would not be cost-effective.

Matters for Congressional Consideration

In order to give the Bureau as much planning time as possible, Congress may wish to consider coming to an early decision on whether the Bureau should be required to enumerate overseas Americans, and if so, whether they should be counted as part of the decennial census or by some other, separate data collection effort. Should Congress desire an overseas count—be it part of the decennial census or a separate data collection effortit should consider telling the Bureau how the data would be used (e.g., for purposes of apportionment, redistricting, allocating federal funds, or a tally of the U.S. overseas population). This information would enable the Bureau to more thoroughly evaluate procedures and resources needed to meet Congress’s specific requirements, and ultimately provide Congress with better information with which to gauge the feasibility of such an approach.

Recommendations for Executive Action

To the extent that the Bureau proceeds with plans to conduct a second feasibility test in 2006, we recommend that the Secretary of Commerce direct the Bureau to take appropriate steps to improve the soundness of the test design and better address the objectives of an overseas enumeration. Specific steps should include the following 12 actions: Be more transparent with Congress and other stakeholders on what variables the Bureau is able to quantitatively measure, as well as what research questions the Bureau can and cannot answer.

Develop and pretest clear residence rules and appropriate guidelines on who should be included in the count to prevent confusing prospective respondents.

Ensure that its outreach and promotion strategy, data collection methods, and other aspects of the design are free from cultural and other biases that could introduce systemic errors.

Explore the possibility of developing more comprehensive measures of data quality.

Test the Bureau’s ability to geocode certain noncity-style addresses such as those with post office box numbers.

Research how best to market the overseas census to hard-to-count groups and other, less visible, segments of a country’s overseas American population.

Develop procedures to determine whether a return is within the scope of the enumeration—i.e., that it is truly from an overseas source.

Actually integrate overseas data with stateside data to more thoroughly test this activity.

Develop a cost model to provide the Bureau and Congress with better estimates of the costs of conducting an overseas census under different methodological and other scenarios.

Thoroughly pretest the overseas questionnaire with various groups of potential respondents to ensure the questions are clear, appropriate, and can be accurately answered by the unique population groups that are found overseas.

Add more response options to the questionnaire item concerning respondents’ primary activity. Specific information on whether an individual is retired, a student, or working for a private company, etc., could provide the Bureau with valuable data on the characteristics of overseas Americans that could be important for some of the purposes for which the data might be used, and just as important, could provide the Bureau with invaluable marketing data that the Bureau could use to develop a more effective outreach and promotion campaign.

Work with Congress and other stakeholders to develop a broad consensus on what would be acceptable levels of accuracy, completeness, participation, and other key measures of performance.

Agency Comments and Our Evaluation

The Secretary of Commerce forwarded written comments from the U.S. Census Bureau on a draft of this report on April 13, 2004, which are reprinted in appendix I. The Bureau generally agreed with our key findings, conclusions, and recommendations, and suggested some additional context, technical corrections, and clarifications, which we have incorporated.

The Bureau disagreed with our recommendation that it be more transparent with Congress and other stakeholders on what variables the Bureau is able to quantitatively measure, as well as what research questions the Bureau can and cannot answer. The Bureau believes that it has always been transparent about its plans and the likely limitations of this first test.

Although the Bureau’s test plan describes the limitations associated with answering its various research questions, nowhere does it disclose that its fundamental research objective to “determine the feasibility, quality, and cost of collecting data from U.S. citizens living overseas” will only be partially answered. Indeed, none of the documentation we reviewed, including the test project plan or briefing slides provided to congressional staff in April 2003, indicated either explicitly or implicitly that this test was, as the Bureau maintains, “only the most basic assessment of feasibility,” or that the Bureau would not be able to fully answer its key questions regarding feasibility, data quality, and cost. For example, as noted in our report, even though one of the principal objectives of the test was to determine the cost of collecting data from U.S. citizens living overseas, the Bureau’s test design lacked a research question for this objective.

The Bureau agreed, however, that as it completes its evaluations and documents its findings from the test, it will be “critical” to highlight the various qualitative and quantitative limitations that could affect congressional deliberations on this subject.

As agreed with your offices, unless you release its contents earlier, we plan no further distribution of this report until 30 days from its issue date. At that time we will send copies to other interested congressional committees, the Secretary of Commerce, and the Director of the U.S. Census Bureau. Copies will be made available to others upon request. This report will also be available at no charge on GAO’s Web site at http://www.gao.gov.

If you or your staff have any questions concerning this report, please contact me at (202) 512-6806 or by e-mail at daltonp@gao.gov or Robert Goldenkoff, Assistant Director, at (202) 512-2757 or goldenkoffr@gao.gov. Key contributors to this report were Lisa Pearson, Charlesetta Bailey, Betty Clark, Ellen Grady, Ronald La Due Lake, Andrea Levine, and Daniel Messler.

Comments from the Department of Commerce

Criteria for Evaluating Census Research Design for Overseas Enumeration Test

1. How clearly are the objectives of the research design presented? a. Are the test/research objectives and/or questions specified clearly in the design? b. Are concepts defined where necessary? 2. How sound is the research design? a. Do the research questions match the research objectives? b. Is the rationale for the determination of the size and type of sample explained? c. Are potential biases recognized and addressed (e.g., cultural bias, question item bias, or sample bias)? d. Does the sample selection reflect the full range and full cycle of entities or processes that should be considered? 3. How appropriate is the data collection strategy? a. Is the mode of data collection stated clearly? b. Is the timing and frequency of data collection considered? c. Is the data collection method appropriate for reaching the intended sample population? d. Is the data collection instrument appropriate for the sample population? e. Is a plan for administering and monitoring the data collection discussed in the design? f. How well does the design address factors that may interfere with data collection? g. How well are methods for assuring adequate response rates addressed? 4. How thoroughly did Census test the survey instrument(s)? a. Were any new survey items cognitively tested? b. Were field tests conducted to identify the best question wording and determine whether changes in questions are likely to achieve the change objectives? c. Were research studies used to address possible changes in the questionnaire? 5. How involved were relevant stakeholders in the research planning? a. Were relevant stakeholders for the research objectives identified? b. Were stakeholders involved in the planning or review of the methods of data collection? c. Were appropriate stakeholders participants in the review and testing of the questionnaire? 6. How sound is the design’s plan for reaching the target sample? a. Are the goals of the outreach strategy feasible? b. Are the methods of the outreach strategy viable? 7. How sound is the plan for implementing test site activities? a. Does the design consider possible mistakes and their consequences (including their seriousness)? b. Does the design assure that sufficient evidence will be gathered to answer the research questions? c. Does the design consider the level of difficulty in obtaining the data? 8. How good is the relationship between the research design and the time and resources allocated? a. Does the execution of the design appear feasible within the stated time frame? b. Do the proposed resources for the execution of the design appear feasible? c. Are the roles and responsibilities of key parties specified? 9. How well does the design develop a data analysis plan? a. Is the method of enumeration clearly presented? b. Are the proposed analytical techniques for analysis presented? c. Does the design provide a basis for comparing the results of the research? d. Does the design discuss the possible limitations of the findings/test results?

2004 Overseas Test Census Questionnaire

GAO’s Mission

The General Accounting Office, the audit, evaluation and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO’s commitment to good government is reflected in its core values of accountability, integrity, and reliability.

Obtaining Copies of GAO Reports and Testimony

The fastest and easiest way to obtain copies of GAO documents at no cost is through the Internet. GAO’s Web site (www.gao.gov) contains abstracts and full- text files of current reports and testimony and an expanding archive of older products. The Web site features a search engine to help you locate documents using key words and phrases. You can print these documents in their entirety, including charts and other graphics.

Each day, GAO issues a list of newly released reports, testimony, and correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site daily. The list contains links to the full-text document files. To have GAO e-mail this list to you every afternoon, go to www.gao.gov and select “Subscribe to e-mail alerts” under the “Order GAO Products” heading.

Order by Mail or Phone

To Report Fraud, Waste, and Abuse in Federal Programs

Public Affairs