Illegal Immigration: INS Overstay Estimation Methods Need Improvement
Published: 1995-09-26T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Pursuant to a congressional request, GAO examined the basis for the Immigration and Naturalization Service (INS) estimates of persons who entered the United States legally as visitors but did not leave under the terms of their admission.
section_title: What GAO Found
paragraphs: GAO found that INS: (1) devised a creative approach for estimating overstays by estimating the number of uncounted departures; (2) determined that the number of uncounted departures could be estimated by using system data from countries for which it seems safe to assume there are few or no overstays; (3) applied its strategy using a global approach that requires the assumption that a single rate of system error applies to all countries worldwide; and (4) global approach provided a good starting point for estimating overstays, but INS should develop improved procedures for estimating overstays among foreign visitors arriving by air.

section_title: Letter
paragraphs: This letter is in response to your April 18, 1995, request that we examine the basis for the Immigration and Naturalization Service (INS) estimates of overstays—that is, persons who entered the United States legally as visitors but did not leave under the terms of their admission. INS estimates that about half of the illegal resident aliens in the United States during recent years have been overstays. The other half are principally persons who crossed U.S. borders illegally.
Reliable and valid estimates of the number of overstays are important to public policy-making. Higher numbers of overstays might suggest, for example, the need for stricter policies or laws for issuing temporary U.S. visas to citizens of those countries whose travelers tend to overstay their visas in significant numbers. Higher numbers of illegal border crossers might suggest a greater need for actions such as further improving border security. Overstay data are also needed to monitor travel from countries whose citizens are not required to obtain a U.S. tourist visa.
subsections: section_title: Background
paragraphs: The task of estimating overstays presents a difficult challenge. Although INS has a data system for tracking the dates when individual foreign visitors arrive in and depart from the United States, the agency cannot assume that all persons whom the system does not record as having left have, in fact, overstayed their lawful periods of entry.
Most foreign visitors tracked by the INS data system travel to the United States by air. The INS data system works as follows: Airline personnel distribute arrival/departure forms (I-94s) to foreign visitors. An INS inspector collects the arrival portion of the form as each visitor enters the United States; at that time, the inspector staples the departure form in the visitor’s passport. The airlines are responsible for collecting the departure forms when visitors leave and for sending those forms to INS. But for some visitors who may have actually left the United States, INS apparently has no record of the departures. It is thus not clear, in any particular instance, whether an “unrecorded departure form” represents a person who overstayed or whose actual departure went uncounted.
section_title: Objectives, Scope, and Methodology
paragraphs: Our objectives were to examine the basis of INS overstay estimates and, as appropriate, to suggest improved estimation procedures. In reviewing the agency’s methodology, we examined published documents that describe INS procedures and interviewed staff about their methods of overstay estimation, the kinds of data that are available, and the potential for devising improved overstay estimates. We focused primarily on tourist air arrivals since they represent the majority of the foreign visitors in the INS data system. We developed new estimation procedures and applied them to tourist visitors arriving by air in the United States from nine countries from October 1990 to March 1991. We also talked with officials from the Department of State about their use of INS estimates and asked a number of experts to review a draft of this report.
We conducted our review in accordance with generally accepted government auditing standards.
section_title: GAO’s Analysis
paragraphs: INS considers the pool of unrecorded departure forms to represent two groups of persons—true overstays and uncounted departures. For true overstays, no departure forms were ever turned in because these visitors never left the United States. For uncounted departures, there was a paperwork error of some sort; that is, these visitors did leave but their departures were not counted by INS (either because the airline did not collect their departure forms or for some other reason). INS has termed the unrecorded departure forms in this latter group “system error.” Earlier we noted that one could not simply assume that all unrecorded departure forms represent overstays. Here we see that some unrecorded departure forms represent system error. In short, true overstays equal unrecorded departure forms minus the number attributable to system error.
In order to estimate true overstays, INS thus needs to estimate system error. INS does this using 11 “index countries” that it believes have so few true overstays that all unrecorded departure forms can be considered uncounted departures (system error). If INS finds, for example, that 10 percent of airline travelers arriving from these index countries were not counted as departures, the agency then assumes that the percentage of uncounted airline departures is about 10 percent in all countries. That is, INS uses one estimate of the percentage of uncounted departures for all countries of the world.
INS’ specific method of estimating overstays from each country is to subtract an estimate of uncounted departures from the total number of unrecorded departure forms for each country, based on a global estimate of the proportion of departure forms that are missing because of system error. INS then totals the individual countries’ overstay estimates to produce an estimate of the number of true overstays from all countries.
While this method is creative, it makes a sweeping assumption. INS itself had characterized its procedures as more likely to overestimate than underestimate the number of overstays. Officials from the Department of State also questioned the accuracy of INS’ estimates of overstays for certain countries.
In a detailed review, we found that estimates based on INS’ global approach are necessarily marked by considerable uncertainty because of the assumption that rates of system error are the same for index countries and all other countries. Although INS’ procedures assume it is safe to ignore the potential for differences in system error between index countries and other countries, airline-specific data for the index countries showed evidence to the contrary: First, INS’ airline-specific data confirmed that citizens of different countries tend to fly to the United States on different airlines. Second, INS’ airline-specific data for index countries showed that there is substantial variation in system error across airlines.
We also found an algebraic error in INS’ computation formula. INS’ use of an incorrect formula resulted in an error on the order of 7 to 10 percent in INS’ overstay estimates for air travelers.
subsections: section_title: GAO’s Method
paragraphs: We devised a method that uses INS’ airline-specific data for index countries as the basis for estimating separate rates of system error for each airline. We explain this method in detail below, but the main point is that it allows us to estimate system error separately for each airline—rather than rely on the assumption that a single rate of system error applies to all flights from all countries of the world.
Specifically, we began by estimating system error separately for each airline that regularly flies between one or more index countries and the United States. For example, we used the rate of unrecorded departure forms for index country passengers on United Airlines to estimate system error for all United Airlines flights. We repeated this procedure, estimating system error separately for Delta Air Lines, Air Micronesia, and so forth, on the basis of index countries’ data for each airline.
Then, we estimated overstays for nonindex countries—proceeding separately, country by country and airline by airline. For example, we estimated overstays from the Philippines who flew to this country on United Airlines using the specific rate of system error we had estimated for United Airlines; then we estimated overstays from the Philippines who flew in on Air Micronesia using the estimated rate of system error for Air Micronesia, and so forth. Essentially, we assumed that system error on United Airlines flights between the Philippines and the United States was about the same as for United Airlines flights between index countries and the United States. We made the same assumption for Air Micronesia flights, and so forth.
For some airlines, this approach was not possible because they do not regularly fly between any index country and the United States. For these “nonqualifying” airlines, additional assumptions were necessary. In making assumptions, we wished to be as conservative as possible. We therefore calculated two sets of estimates using alternative assumptions, as described below.
subsections: section_title: Assumption 1 for Nonqualifying Airlines: Same Overstay Rates
paragraphs: We first assumed that passengers on nonqualifying airlines have the same overstay rate as passengers from the same sending country who arrived in the United States on qualifying airlines (for which the separate calculations could be performed).
The example of Mexico can illustrate how our method works when this assumption is used. Separate estimates of system error were available for American Airlines, Delta Air Lines, United Airlines, Air Micronesia, and other “qualifying” airlines (which fly between index countries and the United States)—but not for airlines such as Mexicana de Aviacion and Aeromexico. We were thus able to calculate separate overstay estimates for Mexican visitors who arrived on each qualifying airline. For Mexican visitors who arrived on “nonqualifying” airlines that do not regularly fly between index countries and the United States (such as Mexicana de Aviacion and Aeromexico), we assumed that the overstay rate was the same as for Mexican visitors arriving on qualifying airlines. Using these procedures, we obtained overstay estimates for nine nonindex, or sending, countries.
section_title: Assumption 2 for Nonqualifying Airlines: Same Rates of System Error
paragraphs: We also performed a second set of calculations for the nine countries, using the alternative assumption that, for a particular sending country, passengers on both types of airlines (qualifying and nonqualifying) had the same rates of system error.
Again using the example of Mexico to illustrate, we calculated separate overstay estimates for passengers on each qualifying airline (American Airlines, United, Delta, Air Micronesia, and so forth)—exactly as before. We then assumed that for Mexicana de Aviacion, Aero Mexico, and other nonqualifying airlines, the rate of system error was the same as calculated for Mexicans who arrived on qualifying airlines.
section_title: GAO’s Estimates and INS’
paragraphs: We believe our method represents an improvement on two counts: First, the assumptions required are more limited than those required by INS’ global method because, to the extent possible, our method relies on airline-by-airline estimates of system error. Second, we used a correct, mathematically derived formula, as well as appropriately weighted data.
Table 1 presents our two sets of estimates, based on the alternative assumptions for nonqualifying airlines, for nine sending countries.
Table 1 also presents estimates of overstays based on INS’ global method; one column presents estimates calculated by INS using the incorrect formula, and the other column presents estimates that we calculated using INS’ general global method but correcting the error in INS’ formula.
While results vary for individual countries, overstay estimates are, on average, lower when using the more detailed airline-by-airline method than when using the global method. The contrast between results using the global method and corresponding results using the airline-by-airline method is greatest when the global method is applied using the correct computation formula.
section_title: GAO’s Alternative Assumptions
paragraphs: The airline-by-airline overstay estimates shown in table 1 are, on average, lowest when passengers on nonqualifying airlines are assumed to have the same overstay rates as passengers from the same country who arrived on qualifying airlines. Although the nine countries’ overstay estimates based on assumption 1 (totaling about 25,000) are dramatically lower than INS’, these might be the better of the two sets of airline-by-airline estimates.However, taken together, our two sets of estimates for overstays from the nine sending countries (25,000 and 39,000) define a range that depicts the likely uncertainty of the estimates that arises from the lack of detailed data for visitors who fly here on nonqualifying airlines.
There is a specific logic for using the two alternative assumptions. That is, the known quantity for each sending country and airline is the number—or percentage—of unrecorded departure forms. If the percentage of unrecorded departure forms is higher for passengers on “nonqualifying” airlines than for passengers on “qualifying” airlines (as is the case for Mexico, for example), this difference is likely attributable either (1) to the fact that system error is higher for nonqualifying airlines (whereas equal overstay rates apply for passengers on both groups of airlines) or (2) to the fact that the overstay rate is higher for the nonqualifying airlines (whereas the same rate of system error applies) or (3) to some combination of higher system error and higher overstay rates for the nonqualifying airlines. A range of estimates based on our two alternative assumptions covers these three possibilities. For each country, the size of the range is determined by the empirically observed difference between the rates of unrecorded departure forms for qualifying and nonqualifying airlines.
section_title: Limitations and Potential Improvements
paragraphs: All overstay estimates require some assumptions—and are therefore subject to some degree of uncertainty. At the time we drafted this report, the only estimates other than those that we made were INS’ estimates based on its global approach. The method we developed reduces the uncertainty relative to INS’ global approach because the assumptions are more limited. However, it is important to realize that even the range for airline-by-airline estimates described above does not take account of all sources of uncertainty. There is variation in system error across index countries—even for passengers on a particular airline. And it is certainly possible that, even on a particular qualifying airline, system error might be higher (or lower) for passengers from sending countries (for example, the Bahamas or Mexico) than it is for passengers from index countries.
Although we believe the procedures we devised are a step in the right direction, we also believe further improvements are possible. In responding to an oral briefing on our work, INS officials agreed that our method constitutes a step forward, and they suggested modifications that build upon—and that they believe might improve—our approach. Specifically, INS suggested increasing the number of index countries and using more detailed data on variation in system error by port of entry. Other expert reviewers provided a variety of suggestions, including using more complex models and conducting special studies to obtain empirical estimates of system error in nonindex countries.
section_title: Recommendation to the Commissioner of INS
paragraphs: We recommend that the Commissioner of INS have new overstay estimates prepared for air arrivals from all countries, using improved estimation procedures such as those discussed in this report, including, as appropriate, the potential improvements suggested by INS or by reviewers of this report.
section_title: Agency Comments
paragraphs: INS agreed that data on the individual airlines could improve its estimates of overstays. INS developed a new version of the airline-by-airline approach and revised its criteria for selecting index countries—replacing its “original” set of index countries with a new set of 17. In its written comments, INS compared estimates obtained with its old and new estimation procedures.  INS concluded that (1) the two methods produced differing estimates for certain countries, and (2) the worldwide estimate of overstays obtained with its new procedures (104,000 for the visitor category and time period in question) essentially matched the number estimated with its old procedures (107,000). More recently, INS told us that it will no longer use its previous approach and has adopted the airline-by-airline method (including the corrected estimation formula).
By adopting an airline-by-airline approach and correcting its formula, we believe INS has taken an important step forward. This is partly because achieving more accurate estimates of overstays from individual countries is essential for certain policy-related decisions—particularly those involved in administering the Visa Waiver Pilot Program. In addition, contrary to INS’ conclusion, we believe that the agency’s change to an airline-by-airline method may make a substantial difference in the worldwide estimate. As we explain in appendix II, depending on which sets of index countries INS uses, the worldwide overstay estimate achieved with airline-by-airline procedures may be 17- to 25-percent lower than the global estimate. (See pp. 56-58.)
Although INS’ adoption of an airline-by-airline method is an important step in the right direction, we believe the improvement of overstay estimation methods is a “work in progress.” As discussed in appendix II, we believe INS’ selection of index countries deserves further study. Also, certain aspects of INS’ airline-by-airline procedures differ from those that we used, and we have some potentially important concerns about those differences. Finally, the experts who reviewed a draft of this report suggested possible strategies for reducing the uncertainty that characterizes both our estimates and INS’, but INS has not had an opportunity to consider the experts’ suggestions. For these reasons, we believe our recommendation to the Commissioner of INS is still appropriate.
As agreed with your office, unless you publicly announce its contents earlier, we plan no further distribution of this report until 30 days from its date of issue. At that time, we will be sending copies of this report to the Immigration and Naturalization Service, the Department of State, and other interested parties. We will also make copies available to others upon request.
If you or your staff would like to discuss any of the issues we present here, please call me at (202) 512-2900 or Judith A. Droitcour, who served as project director on this study, at (202) 512-5885. Other major contributors to this report are listed in appendix IV.
section_title: Technical Appendix
paragraphs: This appendix presents detailed information on the following technical topics: (1) the estimation of “system error” (percentage of uncounted departures) and INS’ “global” assumption that the level of system error is the same worldwide (across all airlines and all countries of citizenship); (2) the formula we developed for estimating overstays (given an estimate of system error) and an explanation of the error we identified in INS’ computation formula; and (3) a new set of procedures for estimating overstays that, by incorporating data on specific airlines, avoids the need for INS’ global assumption and, at the same time, uses the correct computation formula and appropriately weighted data. Additional sections of this appendix discuss (4) possible further improvements in estimation procedures and (5) problems with overstay estimation procedures for visitors arriving by land.
subsections: section_title: Background
paragraphs: Each year, millions of foreigners legally enter the United States as “nonimmigrant” visitors (that is, on a temporary basis and for a specific purpose, such as tourism). The INS Nonimmigrant Information System tracks records of legal nonimmigrant visitors by country of citizenship; visa category (tourist visit, business, or other purpose such as temporary work); mode of travel (air, land, or sea); and for each air passenger, the airline on which he or she entered the United States. As indicated by the NIIS data, the large majority of foreign visitors are tourists who enter by air. (It is important to note that many Canadian visitors are not counted by this system. The same is true for many Mexican visitors who use border-crossing cards. Data on foreign students entering the United States to attend school are maintained in a separate system and are not part of the estimates discussed in this report.)
INS also attempts to determine whether or not each foreign visitor (identified by name, date of birth, country of citizenship, and passport number) exits the United States by 9 months after his or her expected date of departure. INS’ use of a minimum 9-month overstay period for purposes of calculating the overstay estimates discussed in this report means that transients who overstayed for only a few weeks or months are not included in these figures. Rather, INS defines estimated overstays as foreign visitors who—having been here for a year or more—appear to have settled in the United States. However, as explained below, there is (1) considerable error in counting visitor departures and thus (2) uncertainty in the estimation of overstays.
Briefly, when a foreign visitor legally enters the United States, he or she fills out an arrival/departure form (I-94 form; see figure I.1). The arrival portion of the I-94 form is detached and retained by INS. The departure portion of the form is stapled to the visitor’s passport. Each visitor is supposed to turn in the departure portion of the form when exiting the United States. The INS data system attempts to match the arrival with the corresponding departure portion of the form. Thus, it is possible to determine the number of foreign visitors for whom departure forms have not been recorded (“unrecorded departure forms”).
In some instances, unrecorded departure forms correspond to visitors who overstayed. Often, however, departure forms have gone unrecorded because either (1) the forms were not turned in, not collected, or if collected by an airline, not returned to INS; or conceivably, (2) the forms were returned to INS but not correctly recorded as of a 9-month period following the expected departure date.
I-94 Arrival/Departure Record - Instructions This form must be completed by all persons except U.S. Citizens, returning resident aliens, aliens with immigrant visas, and Canadian Citizens visiting or in transit.
Type of print legibly with pen in ALL CAPITAL LETTERS. Use English.  Do not write on the back of this form.
This form is in two parts.  Please complete both the Arrival Record (Items 1 through 13) and the Departure Record (Items 14 through 17).
When all items are completed, present this form to the U.S. Immigration and Naturalization Service Inspector.
Item 7 -If you are entering the United States by land, enter LAND in this space.  If you are entering the United States by ship, enter SEA in this space. 12.Address While in the United States (Number and Street)
27. TWOV Ticket Number Warning - A nonimmigrant who accepts unauthorized employment is subject to  deportation. Important - Retain this permit in your possession; you must surrender it when you  leave the U.S. Failure to do so may delay your entry into the U.S. in the future. You are authorized to stay in the U.S. only until the date written on this form.  To  remain past this date, without permmission from immigration authorities, is a  violation of the law. Surrender this permit when you leave the U.S.: - By sea or air, to the transportation line; - Across the Canadian border, to a Canadian Official; - Across the Mexican border, to a U.S. Official.
Students planning to reenter the U.S. within 30 days to return to the same school, see "Arrival-Departure" on page 2 of Form I-20 prior to surrendering this permit.
section_title: System Error
paragraphs: The fact that many visitors who actually leave the United States are not on record as having departed constitutes a major problem for overstay estimation. For air passengers, the airlines have been charged with collecting departure forms and transferring these to INS. The agency plans to begin a new airline data collection effort soon; while INS characterizes this effort as still “in the design phase,” it may have some potential for improving INS air departure data. One other possible—but likely expensive—solution would be for INS officials to collect departure forms at airports. For land travelers, the key reason why departure records are missing is that roadways exiting the United States lack facilities for collecting departure forms.
Although uncollected I-94 departure forms seem to be the chief problem, some portion of uncounted departures may possibly result from other factors, such as the loss of forms after they have been collected, incorrect keypunching (which can prevent a match between entry and exit forms), or inadvertent deletion of information from a computer file.
INS uses the term “system error” to refer to uncounted departures. Estimates of system error are not negligible and seem quite large relative to the number of overstays. Yet no one knows, with any precision, how large the system error component is. Consequently, any estimate of overstays is necessarily somewhat uncertain.
The estimation problem can be briefly described as follows. As shown in figure I.2, the three major known quantities in the INS Nonimmigrant Information System are the total number of expected departures, that is, the number of departures that should occur by the time the visitors’ legal periods of stay (plus the subsequent 9-month period) have expired; the number of counted departures, that is, the number of foreign visitors who turned in departure forms that were received and correctly recorded and stored by INS; and the difference between expected departures and counted departures, which we term “unrecorded departure forms.” departures)
In sum, the estimation challenge is that the “unrecorded departure forms” category includes both those foreign visitors who failed to depart (overstays) and those who did depart but were not counted as having departed (uncounted departures or system error). The data do not indicate how many foreign visitors with unrecorded departure forms fall into each of these two groups. It is clear, however, that being able to estimate the number of uncounted departures (system error) would allow the number of overstays to be estimated.
subsections: section_title: INS Devised a Strategy to Estimate System Error
paragraphs: INS devised a creative strategy to estimate system error. The key is to examine the rates of unrecorded departure forms in “index” countries—such as Australia, Sweden, Finland, and Switzerland—for which INS believes it is safe to assume that there are only negligible numbers of overstays. If this is so, then for visitors who are citizens of these (index) countries, the rates of unrecorded departure forms and the rates of uncounted departures (or system error) would, for all intents and purposes, be equal.
Initially, INS defined 12 index countries: Australia, Belgium, Finland, Kuwait, Netherlands Antilles, New Zealand, Norway, Saudi Arabia, Singapore, Suriname, Sweden, and Switzerland. Later, Kuwait was dropped from the list (because of overstays resulting from the Gulf war).INS reported that the 12 countries were selected according to five criteria, one of which is low rates of unrecorded departure forms.
INS realized that the data for foreign visitors from the index countries provide a potential window onto the level of system error that occurs in other countries. INS applied this strategy separately for visitors who arrived by air, land, and sea. This approach involves an assumption that travelers who arrive by one mode of transportation generally depart by the same mode—for example, that visitors who arrive by air generally depart by air. This may be less likely for citizens of certain countries. Notably, Mexicans and Canadians who arrive by air may be more likely to depart by land than visitors from countries that do not border the United States.
section_title: INS Calculated System Error for All Airlines Grouped Together
paragraphs: Using the index country data for visitors who arrived by air, INS calculated a single estimate of system error that combined data for all index countries’ air passengers and the airlines they used to fly into the United States. By calculating a single rate of system error—and applying it globally—INS procedures assumed that this “single point estimate of system error applies to every country of origin within a particular category of admission.” (Warren, 1990, p. 81.) An INS official noted that “it is possible that some areas of the world have different rates of system error. For example, some airlines might do a better job of emphasizing the collection of I-94 departure forms.” (Warren, 1990, p. 81.)
In fact, certain index countries and airlines appear to be considerably more assiduous in collecting departure forms than others. Given this fact, INS’ approach rests on assuming that the rate of system error estimated on the basis of index countries’ air passengers (and the airlines that they use to fly into the United States) is, on average, at least reasonably representative of system error for air passengers from other countries—and the different airlines that they may use to fly into the United States. Of course, no data exist to show whether system error is the same or different in index countries and in other (nonindex) countries; this lack of data represents the major source of uncertainty in current overstay estimates.
The implications of using a global estimate of system error—as opposed to an airline-by-airline approach—are as follows. If the “typical airline” serving the index countries (that is, serving Australia, Switzerland, and so forth) is more assiduous in collecting departure forms than the typical airline serving a given nonindex country (for example, India), then system error is actually lower for the airlines serving the index countries than for the airlines serving India. But the INS procedures would not account for this, so system error would be underestimated for India (or an analogous sending country). Consequently, INS’ estimated number of overstays for India (or an analogous country) would be too high. The logical converse could also be true in some instances.
Because the airlines collect the forms, we believe they are a key factor in estimating system error. But various other factors may also play a role in determining the rate of system error for visitors from a particular country.
section_title: INS Used the Average Plus One Standard Deviation
paragraphs: INS calculates its “global” estimate of system error by taking the average percent of unrecorded departures across the 11 index countries plus the standard deviation of the 11 numbers. (See table I.1.) INS adds one standard deviation to the average in order to avoid underestimating system error. Adding an adjustment of this sort seems to be justified because one criterion INS used to select the index countries was low rates of unrecorded departure forms. The fact that INS used this criterion means that the index countries may not only have zero (or near-zero) rates of overstays, but they may also have lower-than-average rates of system error. We believe, as INS does, that adding an adjustment may help to correct this possible bias. However, we have not assessed the adequacy of this corrective factor.
Tourist air arrivals, October 1986-March 1987. INS calculated the mean with each country (large or small) given an equal weight.
section_title: System Error Varies
paragraphs: We believe a main limitation of the INS estimator is its global quality. That is, INS calculated one estimate of system error based on the airlines that index country passengers flew to enter the United States—and used this estimate for all sending countries and all airlines that citizens of the sending countries flew. But system error is not constant.
Examining index country data for one 6-month period, we found that the rate of unrecorded departure forms, which INS takes as the rate of uncounted departures in index countries, varies across airlines as well as across countries. (See table I.2.) Dividing the airlines into two groups—those with generally lower levels of system error and those with higher levels—it is apparent that the airlines with lower levels of system error tend to be based in index countries, mainly Australia and New Zealand or northern European countries. By contrast, those airlines with higher levels of system error are—in the majority of cases—American carriers based in the United States.
Further consideration indicates that, when flying between the United States and other countries, airlines in the group with lower levels of system error generally tend to fly mainly between the United States and Australia, New Zealand, or northern European countries. By contrast, the airlines with higher levels of system error not only fly from index countries to the United States, but also serve many other countries around the world (for example, countries in Latin America or Asia), bringing travelers from these diverse regions to the United States. On a worldwide basis, these airlines with higher system error would logically be more frequently used by visitors to the United States than airlines in the other group.
Potentially significant patterns like these suggest that, if an estimation procedure could be based on the specific airlines that are actually used by visitors from each foreign country, that procedure might improve estimates of system error. And improved estimates of system error would reduce the uncertainty associated with overstay estimates based on INS’ global approach.
section_title: Using System Error to Estimate Overstays
paragraphs: INS uses its global system error figure to estimate the number of overstays from each sending country. In this section, we (1) present a formula for estimating the number of overstays, given an estimate of system error, and (2) contrast this formula with the INS computation formula (in which we found an error).
Figure I.3 provides the conceptual map needed to follow the ensuing discussion. Conceptually, expected departures may be divided into true departures and overstays. True departures can, in turn, be divided into counted departures (that is, visitors whose returned and recorded departure forms indicate that they have left the United States) and uncounted departures. The category of unrecorded departure forms includes both overstays and uncounted departures.
Figure I.3: Conceptual Map for Estimating System Error EXD (known)
TD (to be estimated)
OV (to be estimated) (known)
UCD (to be estimated)
subsections: section_title: Terminology for Computations
paragraphs: As previously explained, the known quantities are: the expected departures (EXD), the counted departures (CD), and the unrecorded departure forms (URDF) category, which is obtained by subtracting counted departures from expected departures.
True departures (TD), overstays (OV), and uncounted departures (UCD or system error) must be estimated.
As discussed in the previous section, the purpose of estimating system error using index countries’ data is to estimate the percentage and number of uncounted departures for every sending country—and thereby to estimate the number of overstays. But an appropriate formula must be used.
Two additional terms are needed for this discussion.
First, as mentioned above, true departures can be divided into counted departures and uncounted departures. The symbol P refers to the true proportion of departures that are uncounted (that is, the true rate of system error). Specifically,                     P = UCD/TD.
Second, the symbol P¢  refers to an estimate of system error (an estimate of P), defined as the average proportion of uncounted departures in the index countries plus one standard deviation. Specifically,                     P¢ = Average (UCD/TD) + SD, where the subscript i denotes index countries and SD refers to the standard deviation across the index countries.
That is, the rate of system error (P) is estimated using index countries’ data. P¢  is an estimate of the proportion of true departures that went uncounted. As appropriate, P can be estimated for all visitors arriving by air from all countries taken together (using all data on passengers from index countries) or for some subset of passengers such as those arriving on a specific airline (using data on index countries’ passengers who arrived on that airline).
section_title: Appropriate Formula and INS Error
paragraphs: We derived the following formula to estimate overstays:                     OV = EXD − (CD/(1 − P¢ ))
Figure I.4 shows the derivation of this formula.
A.
Overstays as a function of expected departures and true departures: EXD = TD + OV  OV = EXD – TD  (from figure I.3) (algebra)
B.
To estimate true departures: TD = CD + UCD  TD – UCD = CD  (from figure I.3) (algebra: subtracting UCD from both sides  of equation)
TD – (TD/TD) (UCD) = CD  TD – (TD) (UCD/TD) = CD  (since TD/TD = 1) (algebra: multiplying terms)
TD – P(TD) = CD  TD (1 – P) = CD  (since, by definition, P = UCD/TD) (algebra: factoring)
TD = CD/(1 – P) (algebra: dividing both sides by (1 – P))
C.
OV = EXD – TD  OV = EXD – (CD/(1 – P)) (from A above) (substituting last line of B above)
OV = EXD – (CD/(1 – P )) (where P  is an estimate of system error  )
EXD = number of expected departures TD = number of true departures OV = number of overstays CD = number of counted departures UCD = number of uncounted departures  P is calculated based on index country data, in order to estimate P in other countries.  P  is calculated using the average value of UCD/TD for index countries plus one standard deviation.   This adjustment seems justified because index countries may have lower values of P than  other countries.
INS estimated overstays using the following incorrect formula:                     OV = [(URDF/EXD) − P¢ ](EXD) where OV refers to the overstay estimate for a sending country (visa category and mode of travel), URDF refers to the number of unrecorded departure forms for that country, EXD refers to the number of expected departures for that country, and P¢  refers to the estimated rate of system error, based on the index countries data. (See Warren, 1990, p. 106.)
As shown in figure I.5, the incorrect INS formula reduces to                         OV = EXD − CD − P¢ (TD + OV)
OV = EXD − CD − P¢ (TD) − P¢ (OV)
This result indicates that when INS subtracts out the number of uncounted departures attributable to system error, it actually subtracts out not only the proportion of true departures that went uncounted, but also—wrongly—the same proportion of overstays. Assuming P¢  is an adequate estimate of the rate of system error, the effect of using the incorrect INS calculation formula shown here is to underestimate the number of overstays.
OV = (EXD) (INS formula (Warren, 1990, p. 106))
OV = [((EXD – CD)/EXD) – P ](EXD) (substituting  EXD – CD for URDF, based on definition of URDF in figure I.2)
OV = (EXD) – P (EXD) (algebra: multiplying EXD by the two terms inside parentheses)
OV = EXD – CD – P (EXD) (canceling out the EXDs in the first term on the right-hand side of the equation)
OV = EXD – CD – P (TD + OV) (substituting TD + OV for EXD, based on figure I.3)
section_title: GAO Method for Overstay Estimation
paragraphs: To obtain an estimate of overstays that takes advantage of INS data on the specific airlines that foreign visitors from different countries actually use, we developed the four-step approach described below. Three points regarding the assumptions made and not made by our method are crucial:
First, our method does not assume that a single estimate of system error characterizes each country of the world, regardless of which airlines are used. It does assume that the separate level of system error estimated for each airline that serves one or more index countries (for example, American Airlines, Delta, or British Airways) is about the same for passengers from index countries and for passengers from each nonindex country.
Second, our method does assume that system error can be adequately described by the airline of entry to the United States.
Third, with respect to nonqualifying airlines, which do not serve index countries, our method does require that an assumption be made. Specifically, we used two alternative assumptions as follows: We developed one set of estimates based on the assumption that for visitors from a particular sending country (for example, India), the overstay rate is the same for (1) passengers entering the United States on qualifying airlines, such as American Airlines or Delta, which serve one or more index countries, and (2) passengers entering the United States on other (nonqualifying) airlines. We also developed a second set of estimates based on the very different assumption that—again for visitors from a particular sending country—the rate of system error is the same for (1) passengers entering the United States on qualifying airlines and (2) passengers entering on nonqualifying airlines. (Essentially, where it was not possible to use information on the specific airline, we turned to available information on the specific country.)
Three caveats are appropriate: First, there is variation in system error across countries—even for a given airline (refer to table I.2 on p. 25); thus, it is possible that, even for a specific airline, system error may be different for passengers from index countries and from nonindex countries (either individual nonindex countries or nonindex countries in general). Second, there is not a perfect correlation between the airline upon which a passenger enters and departs from the United States; and in fact, logic suggests that for Mexico and Canada, air passengers may be more likely to depart by land than would air passengers from index countries (since these do not border the United States)—suggesting that system error may be higher for citizens of Mexico and Canada than the levels that are estimated using an index countries’ methodology. Third, overstay rates—as well as rates of system error—may differ for travelers from the same country who arrive on qualifying and nonqualifying airlines.
Despite these caveats, we believe the assumptions we have made are more plausible than the global assumption that INS made concerning system error. The four steps of the approach we devised are described below and illustrated in table I.3.
Step 1: Estimate the rate of uncounted departures (system error) for each qualifying airline, using data on visitors from index countries who entered the United States on qualifying airlines.
Step 2: Use step 1 estimates of uncounted departures (system error) together with the appropriate formula to estimate overstays among visitors from each nonindex country who arrived on each qualifying airline.
Step 3: For each nonindex country:  (a) combine the step 2 overstay estimates for qualifying airlines to obtain the overstay rate for visitors from that country who arrived on qualifying airlines; and (b) combine the step 1 estimates of system error for qualifying airlines to obtain the rate of system error for visitors from that country who arrived on qualifying airlines.
Not applicable.
Step 4: For each nonindex country:  (a) project the step 3a overstay rate to visitors from the same country who entered the United States on nonqualifying airlines, which produces one set of overstay estimates; and (b) project the step 3b rate of system error to visitors from the same country who entered on nonqualifying airlines, which produces a second set of overstay estimates.
Conceptually, qualifying airlines are defined as those that regularly fly between at least one index country and the United States. Operationally, we defined qualifying airlines as those with at least 1,000 passengers from a single index country in a 6-month period.
subsections: section_title: Step 1
paragraphs: Using data on index countries’ air passengers, we estimated system error separately for each “qualifying airline.” Conceptually, we defined a qualifying airline as one (such as American Airlines, Delta, British Airways, SAS, SwissAir, or Qantas) that regularly serves one or more index countries—and has sufficient passengers to allow a reasonably reliable estimate of system error for that airline. Operationally, we defined a qualifying airline as one that, during the 6-month period we examined, carried at least 1,000 passengers from a single index country to the United States. In the sample data INS provided to us, we found that, according to our operational definition, there were 29 qualifying airlines.
To estimate system error for a specific qualifying airline, such as American Airlines, we used the index countries’ data for that airline alone. For example, to estimate PAA, we first calculated the proportion of uncounted departures (system error) for each index country—Australia (PAUS/AA), Switzerland (PSWZ/AA), Norway (PNOR/AA), and so forth; then, we took the weighted average of these figures.
To complete step 1, we also calculated the standard deviation of the proportion of uncounted departures across index countries for a specific airline. We then took, as the estimate of system error for a specific airline, the weighted average plus one standard deviation.
Step 1 was repeated for each of the 29 qualifying airlines.
section_title: Step 2
paragraphs: For a specific foreign country, such as Mexico, we estimated the number of overstays who had come into the United States on each qualifying airline. That is, we estimated overstays for American Airlines (OVMEX/AA), Delta (OVMEX/DEL), and so forth—performing the overstay calculations for each of the 29 qualifying airlines that transported Mexican visitors into the United States.
There are two points to note with respect to these procedures.
First, within a particular sending country, the number of overstays who had arrived on each qualifying airline was estimated using the index countries’ estimate of system error developed for that particular airline.
AA))
Step 2 was repeated separately for each qualifying airline within each country.
section_title: Step 3
paragraphs: Step 3 includes two alternative procedures (steps 3a and 3b), each of which is directed toward producing a separate set of estimates.
In step 3a, we totaled estimated overstays across qualifying airlines, within each country, and calculated the overstay rate for passengers on all qualifying airlines in that country. Specifically, we divided the estimated total number of overstays who had arrived on qualifying airlines by the total number of expected departures for those airlines—again, for a particular country.
In step 3b, we calculated the rate of system error, within each country, for visitors who arrived on qualifying airlines.
section_title: Step 4
paragraphs: Step 4 also has two components (steps 4a and 4b); again, each component is directed toward obtaining a separate set of estimates.
In step 4a, we projected the step 3a overstay rate to visitors from the same country who entered the United States on nonqualifying airlines. In other words, we used the step 3a overstay rate as the estimated overstay rate, that is, R(OV), for the entire country.
Then, to obtain the estimated number of overstays for each country, we multiplied the country-specific overstay rate times the total number of expected departures for that country. (The total number of expected departures includes expected departures for visitors who had arrived in the United States on qualifying airlines and for those who had arrived on other, nonqualifying airlines.) This produced one set of overstay estimates; expressed statistically, the number of overstays for the kth sending country (air arrivals) is estimated as shown in figure I.6.
In step 4b, we performed analogous procedures using rates of system error rather than overstay rates. This produced a second set of alternative overstay estimates.
Estimator for the number of overstays from the kth sending country: EST (OV  ) = (EXD  ) where EST(R(OV)  ) refers to the estimated overstay rate for the kth sending country and EXD  refers to the number of expected departures for that country.
Estimator for the overstay rate for the kth sending country:  EST(R(OV)  ) =     (OV  )/    (EXD  ) where OV   refers to the estimated number of overstays for the jth qualifying airline and the kth sending country, and the sum is taken across all qualifying airlines within the kth country; and where EXD   refers to the number of expected departures for the jth qualifying airline and the kth country, and again, the sum is taken across all qualifying airlines within the kth country.
Estimator of the number of overstays for the kth sending country and the jth qualifying airline: EST (OV  ) = EXD   – [CD   /(1 – P  )] where CD   refers to the counted departures for visitors from the kth sending country who arrived on the jth qualifying airline, P  refers to the estimated system error for the jth airline, and the formula is from figure 1.4.
section_title: Other Possible New Approaches
paragraphs: In response to our oral presentation of the new procedures, INS suggested some additional possibilities for improved approaches to estimating overstays. Expert reviewers of a draft of this report also made suggestions.
subsections: section_title: INS’ Suggestions
paragraphs: One alternative approach would be a modification of the separate-airlines procedures we presented above. Specifically, INS suggested an approach in which it would identify additional index countries, thus expanding the number of “qualifying airlines.” The object (and benefit) would be to increase the percentage of passengers from each sending country who arrive in the United States on qualifying airlines. In other words, the likely effect would be to substantially reduce the number of persons for whom it would be necessary to assume that the overstay rate (or, alternatively, the rate of system error) is the same as on the qualifying airlines. We believe that increasing the number of index countries could be a promising approach if appropriately implemented. But the issue of how to define specific criteria for index countries is a complex one and has not been addressed in this report.
Another approach that INS suggested would involve estimates of system error that adjust for different ports of entry (for example, Miami, New York, and so forth) as well as different airlines. If this approach proves practicable, it could also be used in conjunction with adjustments for different airlines. That is, airline and port of entry could both be taken into account.
section_title: Experts’ Suggestions
paragraphs: Other approaches or variations were suggested by expert reviewers, including ideas for small-scale empirical studies, special analyses, and complex estimation models.
subsections: section_title: Empirical Studies
paragraphs: Two experts suggested that it might be possible to conduct special studies designed to estimate system error empirically. Empirical studies might consist of relatively small-scale data collections, such as those used by the Bureau of the Census to estimate the undercount in the decennial census.
Empirical studies may be especially important for certain nonindex countries such those where system error may be atypically high (for example, Mexico and Canada), those where the majority of visitors fly to the United States on nonqualifying airlines, and so forth. There are three possibilities for empirical studies: (1) “planted travelers” studies, (2) follow-up studies of samples of visitors whose departure forms were not recorded, and (3) airport surveys of passengers flying out of the United States.
A “planted travelers” study would utilize persons hired by INS as “plants” who would exit the United States by air, using passports and departure forms supplied by INS. The agency could subsequently check its data set to determine whether records of departure forms for these persons were included. Assuming that the planted travelers were instructed to be passive rather than proactive in turning in the forms, the results could be used to estimate a maximum rate of system error for the airlines or countries in question. Depending on the number of planted travelers and the design of the study, system error estimates could be obtained for selected countries, for the system as a whole, or both.
Another type of empirical study would be a “follow-up” study based on a random sample of perhaps 400 unrecorded departure forms from a specific country, airline, or other defined group. These unrecorded departure forms could be followed up in a number of ways, depending on the country in question. The initial follow-up in all cases could consist of additional checks of INS records. Then, for countries that maintain records of reentering citizens or residents, it might be possible to determine that, in some instances, the country of origin’s files contained a record of the traveler’s return. It also might be feasible to attempt to contact individual travelers either in their home country (after their return home) or in the United States (if they overstayed). Contacted persons might be asked a few brief questions about trips in the past 2 years (airline used, route flown, month of trip) as well as personal data (date of birth). The answers to the questions could be used to establish that the correct person had, in fact, been contacted. Random subsampling of nonrespondents for further follow-up attempts might also be appropriate. Even though some unrecorded departure forms may remain “unsolved,” the percentage in question would, in all likelihood, be greatly reduced.
If a general follow-back study of, for example, Mexican visitors proved too difficult, it still might be possible to follow-back a preselected subgroup, such as Mexican visitors aged 50 or older. This would be extremely useful if analyses of index countries’ data indicated either that (1) system error did not vary substantially across age groups, or (2) system error varied across age groups according to a reliable pattern.
Yet another type of empirical study would involve surveying passengers on selected flights departing the United States. For example, it might be possible for INS inspectors to check passports immediately before passengers board. This would reveal the number of departure forms that were not collected by airline personnel. Or, a complete listing of departing visitors could be made and subsequently compared to records in INS’ data system. Still other types of empirical studies might be developed. But the key point is that the option of fielding small-scale empirical studies has the potential to provide actual data on system error in nonindex countries.
Empirical studies might make it possible to avoid difficult assumptions about system error in nonindex countries and about nonqualifying airlines. At a minimum, the data that these studies could provide would greatly reduce the effects of possible bias on the results.
section_title: Special Analyses
paragraphs: The expert reviewers also pointed to various ways to utilize existing INS data. One analytic approach would be to look for a variable that is not related to system error—but that might be related to overstays. Such a variable could form the basis of a new index. Suppose, for example, that system error does not vary across age. Then, if it is possible to identify an age group with virtually no overstays (one possible candidate would be persons over 50), persons from that age group could be used as the index for their own country and airline of arrival. And still assuming that system error does not vary across age groups, even if there were no particular age group for which it could be assumed that overstays were virtually nonexistent, a similar approach could nonetheless be used if additional relevant data could be obtained. That is, if an empirical estimate of system error could be obtained for at least one age group (for example, persons over 50) in a small-scale empirical study, this estimate could be applied to other age groups. (Refer back to the discussion of empirical studies above.) Of course the variable, age, is used here for illustrative purposes.
Other analytic approaches were also suggested. For example, when frequent travelers reenter the United States, it is sometimes evident that their previous departure forms were not collected. (An analysis of these cases might indicate a rate of uncounted departures that would be considered a minimal rate if one believes that frequent international travelers are more knowledgeable and proactive about turning in their forms.) Estimates could be obtained separately for different countries, airlines, or other subsets of visitors.
Sensitivity analyses were also suggested as a way of examining a variety of analytic options. For example, as one expert emphasized, negative estimates for specific airlines or countries occur when the rate of unrecorded departure forms is lower than the estimated rate of system error. Both we and INS have reported these negative estimates as zero overstays. However, one reviewer suggested that because a random component may influence these estimates, the overall estimates should be calculated without converting negative numbers to zero—thus reducing the number of overstays estimated when airlines and countries are combined. Whether issues like this are relatively minor or might have a substantial impact could be explored in sensitivity analyses.
section_title: Estimation Models
paragraphs: In the absence of empirical studies such as those discussed above, a more complex estimation model might improve on the airline-by-airline method. Indeed, instead of using only adjustments for airline of arrival (as we did) or airline plus port of entry (a possibility that INS suggested), experts pointed out that it is possible to adjust for other background variables (for example, age and sex). One reviewer indicated that an optimal model would adjust for as many measured variables as are relevant. Another said that the model should include enough variables to sufficiently account for variation in system error across countries. (As previously discussed, there is variation in system error across countries—even for a specific airline; refer to table I.2 on p. 25.)
section_title: Overstay Estimates for Land Entries
paragraphs: Within the INS data set being considered here, overstays who entered the United States by land and sea are much less numerous than those who entered by air. However, a sizable number of Mexican visitors enter by land—and for Mexicans, the estimated number of overstays entering by land is not negligible. System error is much higher for land entries than for air arrivals because INS does not have facilities for collecting the I-94 forms when visitors exit by land. Nevertheless, some I-94 forms are collected for persons who arrived by land because (1) some visitors mail the forms in after returning home or turn the forms in to a U.S. embassy; (2) other visitors reenter the United States, and the INS inspector collects the old departure form at that time; and (3) still other visitors who arrived by land depart by air.
Estimates of overstays who arrived by land, like the air arrival estimates, rely on estimates of system error based on unrecorded departure forms for visitors from index countries. Again, INS’ assumption is that a single estimate of system error, based on the experience of visitors from Australia, Switzerland, and so forth, applies to every sending country. For the land entries that are captured by NIIS (comprised chiefly of Mexicans), this seems especially implausible to us. (Specifically, INS assumes that the experience of visitors from countries like Australia and Switzerland who entered the United States by land is the same or at least similar to that of Mexicans who entered the United States by land.)
The key issues in estimating overstays among land entries concern (1) whether the system error estimates for foreign visitors from index countries can fairly be applied to Mexican visitors arriving by land, and more generally, (2) whether the lack of facilities to collect departure forms when visitors exit by land means that overstay estimation for land arrivals is ill-advised, given current methods and data.
One possibly troubling situation is that a number of tourists from index countries may enter the United States from Mexico by land and then fly home to Sweden, Switzerland, and so forth. Such visitors may comprise a relatively large proportion of the number of index country visitors who arrive by land. The possible result is that index country visitors who arrive in the United States by land may—in some fairly large proportion—depart by air. If, by contrast, Mexicans who enter by land generally also depart by land, the use of the index country visitors’ rate of system error would be inappropriate. Thus, the overstay estimates for land arrivals are potentially misleading. We realize that this argument is somewhat speculative, but it highlights the possible need for improved procedures for estimating overstays who did not arrive by air—and illustrates the kinds of problems that might be assessed.
section_title: Comments From the Immigration and Naturalization Service
paragraphs: The following are GAO’s comments on the June 22, 1995, letter from the Immigration and Naturalization Service.
subsections: section_title: GAO Comments
paragraphs: INS told us that as a result of this report, the agency has changed its procedures for estimating overstays. Specifically, for foreign visitors who arrive by air, INS has discontinued use of the old “global” method and has adopted the airline-by-airline method. INS also told us that the error in its computation formula has been corrected. However, INS’ written comments concluded that while the airline-by-airline method may improve estimates for individual countries, it produces a worldwide estimate that is “strikingly similar” to its old worldwide estimate.
We agree that by adopting an airline-by-airline approach, INS has taken an important step forward; however, three major issues remain concerning INS’ new methods. These are discussed in the following three sections; a final section (pp. 61-62) responds to INS’ concerns about the estimates we presented in table 1.
subsections: section_title: Does the Airline-By-Airline Method Make a Difference in the Worldwide Estimate?
paragraphs: The first issue concerns INS’ conclusion that changing to an airline- by-airline method makes essentially no difference in the worldwide estimate. Alternative INS calculations show that depending on the index countries used, the airline-by-airline method produces a worldwide estimate that is 17- to 25-percent lower than the global-method estimate. As discussed below, (1) the index countries’ methodology is potentially sensitive to the inclusion of even one inappropriate index country; and (2) INS’ global-method estimate and its airline-by-airline estimate are based on different sets of index countries—and each set may include an inappropriate index country. Specifically, we believe that Saudi Arabia and Japan may be inappropriate index countries.
Table II.1 shows the impact of removing Saudi Arabia and Japan from the index countries INS used. Of the two global-method estimates shown in the first row of the table, we believe that 126,000 (the estimate calculated without Kuwait and Saudi Arabia as index countries) is the more valid figure. The reason is that the index countries’ methodology assumes that each index country has virtually no overstays. INS originally selected 12 countries that met this standard, but during the Gulf war, overstays apparently increased for two of INS’ original 12 index countries: Kuwait and Saudi Arabia. Given the time frame involved in the estimates examined here (October 1990 to March 1991), we believe INS would ideally have removed not only Kuwait but also Saudi Arabia from its original set of 12 index countries.
Of the two airline-by-airline estimates shown in the second row of table II.1, we believe that 96,000 (the estimate calculated without Japan as an index country) may be the more valid figure. To calculate its airline- by-airline estimates, INS selected a new set of 17 index countries. Kuwait and Saudi Arabia, among other countries, were dropped, but several large countries were added, including Japan. We realize that INS’ goal in selecting these countries was to expand the set of qualifying airlines. We agree with this goal, but we question the inclusion of Japan.
Japanese travelers have lower rates of system error than any other index country on almost every airline for which a comparison can be made.Since index countries are used to estimate system error in nonindex countries, our concern is that the Japan data may not fairly reflect system error for passengers from nonindex countries. (In other words, using an atypical country as an index country can skew results—even if there are no overstays from that country.) INS has said in the past that, for this reason, Japan would not be appropriate for use as an index country. After the agency sent us its written comments, INS recalculated “test estimates” using its airline-by-airline procedures without Japan as an index country.
As summarized in table II.1, INS calculations show that had Saudi Arabia been removed from the original set of index countries, INS’ worldwide global-method estimate for the Gulf war period examined here would have been 126,000 rather than 107,000. If Japan had not been included in INS’ new, larger set of index countries, INS’ worldwide airline-by-airline estimate would have totaled 96,000 rather than 104,000. Substituting one or both of these estimates for those that INS used would yield a very different conclusion: Specifically, the worldwide airline-by-airline estimate would be 17- to 25-percent lower than the worldwide global-method estimate. (That is, 104,000 is 17 percent lower than 126,000, and 96,000 is 25 percent lower than 126,000.)
In light of the foregoing discussion, we believe that INS’ conclusion that the airline-by-airline method does not change the worldwide estimate may be an artifact of (1) retaining an index country that is inappropriate for the Gulf war time period and (2) selecting a new index country that may be atypical and, thus, inappropriate. Therefore, we also believe that, before drawing general conclusions about the effect of the airline-by-airline method, other worldwide estimates that are not affected by these possible problems should also be considered.
section_title: INS’ Airline-By-Airline Procedures Need Further Study
paragraphs: The second issue concerning INS’ new estimates arises because certain aspects of INS’ airline-by-airline estimation procedures (as we understand them at this time) differ from our own. We have some concerns about these differences, and we believe that further study is needed before INS presents its new overstay estimates as definitive. As just discussed with respect to INS’ worldwide estimates, INS’ selection of Japan as one of its new index countries may be inappropriate. This points to a need for INS to continue studying its criteria for selecting index countries. As described below, we also have concerns about other aspects of INS’ airline-by-airline procedures. These include (1) INS’ definition of a qualifying airline and (2) the way that INS derives estimates for those airlines that do not meet that definition.
INS’ definition of a qualifying airline is considerably more liberal than the definition we used. Unlike our definition, INS’ includes “home-country” airlines that do not fly between the United States and any continent where an index country is located. We agree that expanding the number of qualifying airlines is an appropriate goal. However, stretching the airline-by-airline method (to cover such airlines as A L Argentina, Lan Chile, Avianca, Mexicana de Aviacion, and Aero Mexico) means relying on estimates that are driven by two categories of index countries’ travelers that we believe may be atypical.
The first category consists of European and Japanese members of the international business community who reside in Latin America and visit the United States as tourists. Our concern is that this category may consist mainly of experienced international travelers who might be proactive in turning in departure forms. And if they visit the United States frequently (especially, if they reenter this country within 9 months of the expiration of their initial visa), then any uncollected departure forms—which should still be stapled in their passports—would be collected by INS inspectors and transferred to the INS database. Relying heavily on this category of index countries’ travelers might, therefore, result in an underestimate of system error for the home-country airlines in question and a consequent overestimation of overstays.
The second category consists mainly of European and Japanese tourists on multilegged trips. Our concern is that relying heavily on this category of index countries’ passengers might result in estimates of system error for the wrong airlines. The reason is that the airline-by-airline approach necessarily assumes that the airline of arrival is the same as the airline of departure. But, when compared to passengers with a simple round-trip itinerary, “world travelers” on trips that cover three continents (for example, Europe, the United States, and South America) seem more likely to depart the United States on a different airline from the one on which they arrived. In fact, according to one expert, few airlines are licensed for routes both (1) between Europe or Japan and the United States and (2) between Latin American countries and the United States. This raises the possibility that INS’ estimates of system error for airlines like Avianca may, in fact, reflect rates for other airlines (Lufthansa, Japan Air Lines, and so forth).
Turning to INS’ estimation of overstays among passengers arriving on airlines it classifies as nonqualifying, we are concerned that, whereas we used two alternative assumptions, INS used only one (our assumption 2, as described in table 1 on p. 8). Overall, of course, only 5 percent of tourist air arrivals use airlines that do not qualify under INS’ liberal definition.But that 5 percent may include citizens of countries marked by high overstay rates, such as Poland. Use of assumption 1 versus assumption 2 could make a difference of roughly 2,000 overstays for Poland. We believe a range of estimates based on two alternative assumptions might be more meaningful. The need for such a range would be most apparent for sensitivity analyses that omit certain large index countries (such as Japan) or that use a stricter definition of qualifying airlines.
section_title: Additional Improvements May Be Possible
paragraphs: The third issue is that some uncertainties are common to our estimates and INS’—and that improvements aimed at reducing these uncertainties may be possible. For example, passengers from seemingly typical index countries may not, in general, represent the true level of system error for passengers from nonindex countries—even when traveling on the same airline. Another example is that both we and INS converted negative overstay estimates, which occur for several countries and airlines, to zero overstay estimates. However, because a random component may influence these estimates, it might be preferable to calculate combined-airline and combined-countries estimates without converting the negative numbers to zero. (See p. 41.) The effect of this change would be to reduce the number of estimated overstays.
The special analyses, empirical studies, and more complex models suggested by experts who reviewed this report could shed light on these issues. (See pp. 38-42.) But INS has not yet had an opportunity to consider them.
section_title: INS’ Concerns About Our Estimates
paragraphs: Our table 1 estimates for nine sending countries include a set of estimates in which assumption 1 was used for passengers on nonqualifying airlines. That is, one set of our estimates is based on the assumption that the same overstay rate characterizes (1) passengers on airlines for which a separate estimate of system error is possible (our qualifying airlines) and (2) passengers from the same country who used other airlines. In its written comments, INS indicated it has empirical evidence that our assumption 1 is incorrect, but did not present that evidence.
We know of no direct way to check the assumption of equal overstay rates for qualifying and nonqualifying airlines. Thus, we presented a second alternative set of estimates for the same countries, based on a very different assumption (assumption 2 in table 1). INS also questioned assumption 2. However, when assumptions 1 and 2 are used as the basis for alternative estimates, there is a specific logic that we believe justifies the use of these assumptions. (See p. 10.)
section_title: List of Experts
paragraphs: The following experts in the fields of statistics and immigration research reviewed a draft of this report and provided us with their comments.
Frank D. Bean, Ph.D., Population Research Center, University of Texas at Austin Kenneth H. Hill, Ph.D., Department of Population Dynamics, Johns Hopkins University, Baltimore, Maryland Demetrios G. Papademetriou, Ph.D., Carnegie Endowment for International Peace, Washington, D.C.
Jeffrey S. Passel, Ph.D., Urban Institute, Washington, D.C.
section_title: Major Contributors to This Report
subsections: section_title: Program Evaluation and Methodology Division
section_title: Bibliography
paragraphs: Bishop, Y.M.M., S.E. Fienberg, and P.W. Holland. Discrete Multivariate Analysis: Theory and Practice. Cambridge, Mass.: MIT Press, 1975.
Hill, Kenneth. “Discussion.” In Bureau of the Census Second Annual Research Conference, March 23-26, 1986: Proceedings, U.S. Bureau of the Census, 1986, pp. 665-70.
Hopper, Kim. “Counting the Homeless: S-Night in New York.” Evaluation Review, 16 (1992), pp. 376-88.
Levine, Daniel B., Kenneth Hill, and Robert Warren (eds.), Immigration Statistics: A Story of Neglect. Washington, D.C.: National Academy Press, 1985.
Little, Roderick, and Donald Rubin. Statistical Analysis with Missing Data. New York: Wiley, 1987.
National Research Council (U.S.) Panel on Census Requirements in the Year 2000 and Beyond. Modernizing the U.S. Census. Washington, D.C.: National Academy Press, 1995.
Norwood, Janet. Organizing to Count: Change in the Federal Statistical System. Washington, D.C.: Urban Institute, 1995.
Norwood, Janet. “The Consumer Price Index, the Deficit, and Politics.” Policy Bites, No. 22. Washington, D.C.: Urban Institute, Mar. 1995.
Papademetriou, Demetrios G. “Undocumented Immigration in the United States: Some Thoughts About Research Challenges, Impacts, and Recent Policy Initiatives.” Studi Emigrazione/Etudes Migrations, Vol. XXV, No. 91-92 (Sept.-Dec. 1988), pp. 591-617.
Passel, Jeffrey S. “Illegal Immigration: How Big A Problem?” Paper presented at the “Conference on Latin American Migration: The Foreign Policy Dimension” sponsored by the U.S. Department of State. Washington, D.C.: Mar. 17, 1995.
Rubin, Donald B. Multiple Imputation for Nonresponse in Surveys. New York: Wiley, 1987.
Warren, Robert. “Estimates of the Undocumented Immigrant Population Residing in the United States, by Country of Origin and State of Residence: October 1992.” Revised version of a paper presented at “California Immigration 1994,” a seminar sponsored by the California Research Bureau, 1994.
Warren, Robert. “Annual Estimates of Nonimmigrant Overstays in the United States: 1985 to 1988.” In Frank D. Bean, Barry Edmonston, and Jeffrey S. Passel, Undocumented Migration to the United States: IRCA and the Experience of the 1980s. Washington, D.C.: Urban Institute Press, 1990, pp. 77-110.
Warren, Robert. “The INS Nonimmigrant Information System: Assessing the Statistics on Nondeparture.” In Bureau of the Census Second Annual Research Conference, March 23-26, 1986: Proceedings. U.S. Bureau of the Census, 1986, pp. 650-64.
U.S. Immigration and Naturalization Service, Statistical Yearbook of the Immigration and Naturalization Service, 1990. Washington, D.C.: U.S. Government Printing Office, 1991.
section_title: Related GAO Products
paragraphs: Illegal Aliens: National Net Cost Estimates Vary Widely (GAO/HEHS-95-133, July 25, 1995).
Illegal Aliens: Assessing Estimates of Financial Burden on California (GAO/HEHS-95-22, Nov. 28, 1994).
Tax Administration: Earned Income Credit—Data on Noncompliance and Illegal Alien Recipients (GAO/GGD-95-27, Oct. 25, 1994).
Benefits for Illegal Aliens: Some Program Costs Increasing, But Total Costs Unknown (GAO/T-HRD-93-33, Sept. 29, 1993).
Illegal Aliens: Despite Data Limitations, Current Methods Provide Better Population Estimates (GAO/PEMD-93-25, Aug. 5, 1993).
Immigration Enforcement: Problems in Controlling the Flow of Illegal Aliens (GAO/T-GGD-93-18, Mar. 30, 1993).
Immigration and the Labor Market: Nonimmigrant Alien Workers in the United States (GAO/PEMD-92-17, Apr. 28, 1992).
Immigration: An Issue Analysis of an Emerging Problem (GAO, Sept. 1985).
The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent.
U.S. General Accounting Office P.O. Box 6015 Gaithersburg, MD 20884-6015 Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000  or by using fax number (301) 258-4066, or TDD (301) 413-0006.
Each day, GAO issues a list of newly available reports and testimony.  To receive facsimile copies of the daily list or any list from the past 30 days, please call (301) 258-4097 using a touchtone phone.  A recorded menu will provide information on how to obtain these lists.
subsections: section_title: Address Correction Requested