Border Security Metrics Between Ports of Entry
Released: 2016-02-16T00:00:00

Introduction

U.S. borders are complex. They feature varied terrain, where disparate threats are met with varying levels of federal surveillance and resource allocation. Customs and Border Protection (CBP), within the Department of Homeland Security (DHS), is tasked with securing America's borders and promoting legitimate trade and travel. Within CBP, the U.S. Border Patrol (Border Patrol) is charged with securing the border between ports of entry. A reoccurring challenge for DHS and the Border Patrol has been establishing what a secure border means. Their efforts to develop a measure for this have created a continual evolution of metrics. Metrics can describe important phenomena, such as an estimate of how many unauthorized migrants enter the United States every year and what proportion is apprehended by the Border Patrol. These data can help evaluate the effectiveness of border security, influence decisionmaking, and inform Congress and the public.

CBP has stated that "securing our borders means first having the visibility to see what is happening on our border, and second, having the capacity to respond to what we see." Furthermore, "Border Patrol characterizes a secure border as one of low risk, where there is a high probability of detection coupled with a high probability of interdiction." Border security can be assessed at both the strategic and operational levels. At the strategic level, DHS can use performance metrics to provide understanding on whether efforts to secure the border are effective or efficient. These metrics also hold DHS and its components accountable and can affect its decisionmaking. At the operational level, Border Patrol uses the identification of risk and the estimation of its magnitude in its day-to-day operations to secure the border. Such assessment helps the Border Patrol align its resources against specific threats and can help describe how secure a border sector is. It is unclear whether metrics at these two levels interact with one another and if they do, how they influence or affect one another.

Understanding the security risks at the border and developing methods to measure the performance of the Border Patrol in allaying these risks are fundamental to congressional oversight of the Border Patrol, CBP, and DHS more broadly. Some Members of Congress have asked for a clear measure of how many unauthorized migrants cross the border and/or of the overall state of border security. However, determining unauthorized border flows is difficult for many reasons, such as unauthorized migrants' desire to avoid detection. Still, metrics, at both the strategic and operational levels, ideally provide Congress with a sense of the current state of the border that helps inform public debate and possible legislation. For example, in past discussions around comprehensive immigration reform, some have said that legislation must include measures to increase border security, while others have urged that the border first be secured before other immigration reforms take place.

This report reviews the use of metrics at the department (i.e., DHS) and agency level (i.e., CBP ). The report begins by describing the performance metrics at the strategic level that DHS is developing (the level of unauthorized entry of migrants) and has used (the recidivist rate and the effectiveness rate). Next, the report summarizes DHS's, and the former Immigration and Naturalization Service's (INS's), use of metrics since the early 2000s. The report also addresses the Border Patrol's metrics at the operational level. It concludes with a discussion of select issues Congress may consider when evaluating measures of border security. The Appendix includes information on additional data and methodologies that have been reviewed or recommended by scholars and think tanks.

Performance Metrics9

DHS's, and the former INS's, efforts in developing different performance metrics at the strategic level to depict the state of the border have resulted in an array of measures that focus either on border security as a whole or on a component of it ( Figure 1 ). Over time, performance metrics have come and gone, offering little continuity and opportunity for historical assessment. In recent years, DHS has been developing a new measure to estimate unauthorized entry of migrants into the United States. While DHS has shown intent to continually improve its measurement of border security, this new metric is subject to influence by external factors that are outside of the control of DHS and may not directly measure border security.

DHS currently reports the recidivism rate and the rate of interdiction effectiveness. These measures, though subject to limitations, are meant to estimate the Border Patrol's effectiveness in apprehending migrants and deterring migrants from re-entering the United States. DHS's continual re-evaluation of its performance metrics has created a history of past metrics that have either attempted to comprehensively measure border security, such as optimal deterrence and operational control, or have reported only single metrics, such as apprehensions.

Recent Efforts at DHS16

DHS has been working to create a new generation of performance metrics in an effort to better measure border security at the strategic level. For instance, in 2011 DHS announced that CBP was developing a new "border conditions index" (BCI), which would have included measures of estimated unauthorized flows between ports of entry, wait times to cross the border at ports of entry, the efficiency of legal flows at ports of entry, public safety, and quality of life in border regions. In 2013, DHS officials determined that the BCI would not satisfy the demands for a single comprehensive measure of border security and discontinued its development.

Abandoning efforts to establish a single, comprehensive measure of border security, in FY2015 DHS began to develop a means of estimating another component of border security: the level of unauthorized entry into the United States. The measure would only cover migration, not drug smuggling or other security concerns. It appears that DHS opted to fashion what may be a baseline figure upon which it can build broader, more nuanced estimates of performance. Through the use of this metric, DHS hopes to inform its strategic efforts and decisionmaking, while also informing the public and Congress.

Level of Unauthorized Entry of Migrants

According to DHS officials, the agency has begun efforts to estimate the level of unauthorized entry of migrants into the United States. DHS commissioned the Institute for Defense Analyses (IDA) to produce a baseline estimate for this new metric through the use of the repeated trials method (RTM), also known as the capture-recapture method. The basic model calculates the probability of apprehension by examining a group of migrants who have been apprehended and removed previously, with the assumption that the migrant will continue to attempt crossing the border.

The probability of apprehension for a given trip is calculated as:

The probability of apprehension can then be used to estimate the level of unauthorized entry by:

An advantage of RTM is that it relies on observable administrative enforcement data—apprehensions and recidivists—to calculate key border security metrics: apprehension rates and unauthorized flows. The model has limitations grounded in certain assumptions, such as that the pool of migrants is fixed, the probability of being apprehended is the same for each attempt, and those who have been arrested have not succeeded in entering the United States between arrests. The basic model also does not account for migrants who are deterred from attempting to cross the border again because it uses the basic assumption that migrants will continue to attempt crossing the border until they are successful. Furthermore, given that this metric measures migrant volume and is influenced by the push and pull factors of migration that are outside the control of DHS's components (e.g., economic, social, and political factors in source countries and countries of destination), it does not speak directly to the effectiveness of border security efforts.

DHS is working to factor deterrence into RTM in order to control for migrants who will be deterred from attempting to re-cross the border. DHS is using a survey that measures the characteristics, volumes, tendencies, and effects of migrant flows between the United States and Mexico. The survey, Encuestas sobre Migración en la Frontera Norte de Mexico (EMIF Norte), is conducted by the Colegio de la Frontera Norte (COLEF) and other agencies of the Mexican government, who sample potential migrants at busy departure points in Northern Mexico and migrants removed from the United States.

With the incorporation of deterrence, the probability of deterrence is then calculated as:

EMIF Norte provides information on other migrant populations, outside of Mexicans. RTM models are most useful when sensitive to how migrants from other source countries could affect results.

In addition to RTM, DHS is developing two other methods to validate its estimates. One method is based on the Encuesta Nacional de Ocupación y Empleo (ENOE), a representative Mexican household survey. The second method is based on an econometric analysis of apprehension records used to validate deterrence. These new measurements are to be included in DHS's State of the Border report that is to be produced alongside its 4 th Quarter BorderStat Report. DHS also plans to create "BorderStat," modeled on law enforcement's CompStat, which would provide current and historical trends for performance metrics along with statistics specific to subpopulations among the unauthorized population who enter the United States.

Technology

In addition to estimating the level of unauthorized entry of migrants into the United States, DHS has also begun to turn its attention to evaluating the performance of the technology used to secure the border. A 2014 Government Accountability Office (GAO) report found that CBP had identified how technologies contribute to its mission but that it had not yet created performance metrics to evaluate them. DHS databases allow for CBP to collect data on "asset assists," which are "instances in which a technology, such as a camera, or other asset such as a canine team, contributed to an apprehension or seizure." Currently, Border Patrol agents are not required to record these data and therefore the data on asset assists are incomplete. GAO stated that "requiring the reporting and tracking of asset assist data could help CBP determine the extent to which its surveillance technologies are contributing to CBP's border security efforts." According to DHS officials, the agency's FY2016 border research agenda will include an analysis on the effectiveness of border investments on outcomes. The study is to focus on the change in unauthorized entries and forecast future unauthorized entries (incorporating law enforcement and non-enforcement determinants). The study is to also include an analysis of the effectiveness of CBP border investments.

Other Metrics42

At the strategic level, DHS has used two other performance metrics: the percentage of people apprehended multiple times along the Southwest border, or the recidivism rate (introduced in 2013), and the rate of interdiction effectiveness along the Southwest border between ports of entry, or the effectiveness rate (introduced in 2014). These performance metrics, as reported in DHS's FY2015-FY2017 Annual Performance Report, were used to assess the department's efforts in reaching its goal of securing the U.S. borders. These metrics signal DHS's efforts to improve measuring practices, but they are not without limitations.

Recidivism Rate

Since 2000, the Border Patrol has tracked the number of unique subjects the agency apprehends per year by analyzing biometric data (i.e., fingerprints and digital photographs) of persons apprehended. These data are used to obtain the percentage of people apprehended multiple times along the Southwest border, which represents the percentage of individuals who are recidivists (also referred to as the recidivism rate). CBP holds that "effective and efficient application of consequences for illegal border crossers will, over time, reduce overall recidivism." Therefore, the Border Patrol uses recidivism data as a method to evaluate if it is effectively deterring additional unauthorized crossings.

The recidivism rate is calculated as:

The recidivism rate has some weaknesses as a performance measurement. For example, it is not sensitive to the distance between the United States' border and migrants' source countries. The farther an individual's source country is from the United States, the more difficult it is for the migrant to attempt multiple entries. Therefore, increases in the number of migrants traveling from outside of Mexico could lower the recidivism rate, but such a dip in the rate would not necessarily indicate b etter border security. Furthermore, recidivism rates are not only determined by deterrence migrants may have experienced, but also the probability of apprehension. A decreasing recidivism rate that may be interpreted as a successful increase in deterrence could also be the result of a falling apprehension rate. Moreover, intensified enforcement could increase both rates of deterrence and apprehension, offsetting one another and resulting in no change to the recidivism rate, thereby obscuring results.

Annual Southwest border recidivism rates, tracked by the Border Patrol since 2005, increased slightly between FY2005 (25%) and FY2007 (29%), but have fallen since that time, reaching 14% in FY2014 and FY2015. DHS's Annual Performance Report for Fisc al Years 2015 -201 7 set the target for this metric at less than 17% for FY2015, FY2016, and FY2017.

The Effectiveness Rate

The rate of interdiction effectiveness along the Southwest border between ports of entry, also referred to as the effectiveness rate, reports the percentage of all detected unauthorized entrants who are either apprehended or turned back. The Border Patrol achieves this outcome by maximizing the percentage of unauthorized entrants it apprehends or that return to their source countries (turn backs) and minimizing those that evade apprehension (got aways).

The rate of interdiction effectiveness is calculated as:

DHS's Annual Performance Report for Fiscal Years 2015 -201 7 set the goal for this metric at 80% for FY2015 and 81% for FY2016 and FY2017. In FY2015, DHS met its target by interdicting 81.01% of unauthorized crossers.

The Border Patrol stated that this metric is "used to determine the effectiveness of border security operation at the tactical—or zone—level but can also affect strategic decisionmaking," such as evaluating whether it is employing the appropriate mix and placement of personnel and assets and whether they are being deployed efficiently and effectively.

The interdiction effectiveness rate has drawbacks. This measure is based only on observable events reported by the Border Patrol and therefore excludes undetected entry. The data used are also based on events and not unique subjects, allowing for potential double counting. Moreover, GAO found that this metric's inclusion of turn back and got away data, the reporting of which can vary across sectors, impedes its ability to compare the rate across sectors. While using enforcement data to measure border security can raise a number of methodological concerns, the effectiveness rate may have an advantage over other enforcement metrics because, as a ratio, it may be somewhat less sensitive to the level of enforcement resources in place.

Past Measures60

The federal government has collected data on immigration since 1892. This information has been publicly reported since the 1950s. Over time, there have been multiple efforts to expand data collection, improve analysis, and develop performance metrics. DHS's attempts to measure border security beyond output have resulted in different iterations of performance metrics. For example, in the last 15 years DHS has used five different performance metrics and, as discussed, a new metric is currently under development (see " Level of Unauthorized Entry of Migrants ").

The INS's 2001 and 2002 annual performance reports established "high priority entry corridors demonstrating optimal deterrence," as a performance metric for effectively controlling the border. Optimum deterrence was defined as "the level at which applying more Border Patrol agents and resources would not yield a significant gain in arrests/deterrence." Analyses were conducted for each corridor, including the examination of variables such as apprehensions, border related crimes, recidivism, shifts in illegal activity, smuggling fees, property values, and others. In FY2003, 9 corridors had reached optimum deterrence, and in FY2004 this increased to 11 corridors. When DHS was created in 2003, it continued to report optimum deterrence through 2004.

In 2005, the concept of operational control, also referred to as effective control, replaced optimum deterrence. Section 2 of the Secure Fence Act of 2006 ( P.L. 109-367 ) defined operational control of the border as "the prevention of all unlawful entries into the United States, including entries by terrorists, other unlawful aliens, instruments of terrorism, narcotics, and other contraband." Operational control describes the number of border miles where the Border Patrol can detect, identify, respond to, and interdict cross-border unauthorized activity. In February 2010, the Border Patrol reported that 1,107 miles (57%) of the Southwest border were under operational control. After 2010, DHS ceased to use operational control as a metric because the Border Patrol station and sector chiefs could not accurately and reliably use its coding scheme to assess different border regions and the agency did not view it as useful for evaluating border security on a mile-by-mile basis.

From 2011 to 2013, DHS used apprehensions as a performance metric for border security in annual performance reports. Though apprehension data can inform activity levels and aid CBP in identifying emerging issues and trends, multiple factors outside of U.S. border security resources and tactics affect it. Such factors could include immigration enforcement in transit countries and economic and political conditions and demographic changes within the United States and source countries. Additionally, apprehension data are imperfect indicators because they exclude migrants who successfully enter and remain in the United States and migrants who are deterred from entering the United States. The data are also limited in that they count events rather than unique subjects and therefore may result in the double counting of those who are apprehended more than once in a given time period. Furthermore, RAND reported that "a measure that reflects successful performance whether it rises or falls has limited value as a management tool." This critique refers to how a decrease in apprehensions can be seen as a success for the Border Patrol in deterring migration but it can also be seen as failure for the Border Patrol in interdicting migrants. GAO also reported that apprehension data "[do] not inform program results or resource identification and allocation decisions, and therefore until new goals and measures are developed, DHS and Congress could experience reduced oversight and DHS accountability."

Operational Metrics80

At the operational level, the Border Patrol estimates risk at the border through the use of risk assessments. These assessments are not used as metrics themselves. However, the Border Patrol's methodology for risk assessments monitors certain metrics, along with other information, in order to estimate the level of risk at the sector level.

DHS defines risk as "the potential for an unwanted outcome resulting from an incident, event, or occurrence, as determined by its likelihood and the associated consequences." The process of risk assessment identifies and evaluates potential risks or hazards and analyzes what could happen if the hazard went unchecked. The Border Patrol conducts risk assessments at an operational level to determine the level of risk present in specific border sectors. They are meant to inform and not dictate decisionmaking with respect to resource allocation and response. Furthermore, risk assessments are not designed to measure overall performance, though DHS uses certain risk measurements that may be able to speak to performance.

CBP's 2012-2016 Border Patrol Strategic Plan (BPSP) focuses on "identifying high risk areas and targeting ... response[s] to meet" threats in those areas. This plan employs a risk-based strategy, which departs from the previous, resource-based 2004 National Border Patrol Strategy. According to Border Patrol officials, they employ the "State of the Border Risk Methodology," which the agency claims is "a scientifically valid, adaptable, and quantifiable means of assessing the relative risks of the corridors and sectors of the Southwest Border." This approach has three components: "intelligence," "situational awareness," and "risk methodology."

Intelligence refers to information gathered and analyses performed by U.S. intelligence agencies that offer insight into terrorist or criminal efforts to exploit the U.S. border and undermine security measures. The Border Patrol works with other agencies to obtain and share different intelligence. The information collected through these collaborations reportedly gives the Border Patrol the opportunity to predict threats and act proactively. Situational awareness refers to understanding conditions at specific locations on the border. The Border Patrol increases its situational awareness through two methods. First, the traditional method is used in areas of high threat and activity along the U.S. border, where agents employ different tracking strategies and coordinated patrols. Second, the technical method is used in areas of low activity, where the Border Patrol uses geospatial intelligence. If activity is detected by geospatial intelligence technology, sectors are notified by DHS and they investigate the activity. As of August 24, 2015, 1,279.84 border miles (877.10 on the Southwest border and 402.74 on the Northern border) employed geospatial intelligence and 95.4% of those miles confirmed no activity present. The risk methodology is comprised of 12 indicator metrics that were created by isolating key concepts involved in apprehending undocumented migrants and seizing contraband. For example, some indicators are the average apprehensions per recidivist, daily average apprehensions, and the percentage of first time apprehensions. In addition, other metrics the Border Patrol uses are the recidivist rate and the effectiveness rate, the same metrics that DHS uses as performance metrics. However, the Border Patrol reviews these metrics at the sector level to make operational decisions, while DHS examines them for the entire border to make strategic decisions. For each sector, the Border Patrol standardizes each indicator in order to create a common unit of measure. The average measure for each indicator is noted and if there is a significant deviation from that norm in the wrong direction, meaning it is noticeably different from that sector's average, it is determined to be risky. Depending on how different the measure is from its average, the Border Patrol determines whether the risk is classified as low, medium, or high. Therefore, in this methodology, risk is determined by detecting variations or changes from the norm for each sector.

Though intelligence, situational awareness, and the risk methodology are all important to determining risk, when intelligence contradicts the risk methodology's findings, intelligence holds more weight. This is because the risk methodology's indicators examine past information while intelligence focuses on the future.

Select Issues for Congress to Consider

In reviewing current metrics and those under development, Congress may consider whether these metrics adequately measure border security. For example, is DHS's new metric in development, unauthorized entry of migrants into the United States, satisfactory or will additional metrics be needed to sufficiently measure border security? Furthermore, is a comprehensive border security metric, like operational control, still desirable or necessary or can border security be measured by other metrics that look at different components of security?

A clear measure of border security is an important tool to help Congress conduct oversight and evaluate if CBP is reaching its intended goals. Congress may want to consider what type of review and oversight should be required with respect to measurement of performance. For the purposes of maintaining accountability within DHS and the Border Patrol, Congress may also want to review what it means to be "effective," and more specifically what the targets or meaningful thresholds for different metrics should be. For example, what is an acceptable level of risk at the border?

An evaluation of metrics, at the strategic and operational levels, may also help Congress analyze how to most efficiently use resources at the border. For example, should enforcement expenditures be maintained, increased, or decreased? Furthermore, Congress could consider whether the allocation of funding between different enforcement programs should remain the same or be adjusted. Congress may also be interested in the implementation of metrics at operational and strategic levels and how they interact with one another. For instance, what changes are happening at the departmental level and how do they affect the day-to-day work of Border Patrol agents? In theory, metrics should be used not only to hold DHS accountable but also to inform decisionmaking and the development of strategies for improvement. Therefore, implementation of metrics and how they are used may be of interest to Congress.

Congress could consider whether other data and methodologies should be used to measure border security. For instance, the unintended consequences and secondary effects of border security demonstrate some of the tradeoffs present in maintaining or changing the level of security present at the border. This calls into question whether certain secondary effects are negative and acceptable and whether they impact what is considered to be effective border security. Furthermore, additional or improved data and methodologies could be used to adjust measurements to incorporate shifting migrant demographics, such as increases in unaccompanied child migrants, asylum claims, and migrants from countries other than Mexico.

Some migrants coming from countries not adjacent to the United States also face border security and immigration enforcement in transit countries, such as Mexico. Therefore, Congress may also consider whether DHS's metrics account for the actions of transit countries and whether DHS is capturing the correct data. For example, a decrease in apprehensions at the Southwest border may not be only a result of successful deterrence by the Border Patrol, it may also be the result of increased apprehensions of unauthorized migrants in Mexico, who are therefore prevented from reaching the U.S. border.

Several pieces of legislation have been introduced in the 114 th Congress that include reporting and measurement requirements. For example, some legislation would direct DHS to submit a Northern border threat analysis. Other legislation would require the Secretary of Homeland Security to gain and maintain operational control and situational awareness of the Northern and Southern borders of the United States, while also strengthening DHS's measurement and evaluation of its activities and progress.

Appendix. Additional Data and Methodologies

Research by scholars and think tanks has recommended the collection of additional data and the use of varied methodologies in order to obtain meaningful performance-based metrics to measure border security. Some have suggested collecting data other than enforcement data in order to view a greater breadth of factors impacting border security. Additionally, expanding on the types of methodologies with which enforcement and other data are collected and analyzed can allow for broader perspectives and new findings. New data and methodologies also allow for inclusion of metrics that may be able to provide a more complete or holistic view of border security.

Data

In addition to traditional enforcement statistics, survey data based on interviews with migrants and potential migrants are additional sources of information on unauthorized migration. Surveys can collect different information about their subjects than is found in enforcement data. This is because they can be conducted within the United States as well as in migrant countries of origin, and they may capture more information about successful unauthorized inflows and the deterrent effects of enforcement. In 2011, DHS commissioned a comprehensive study by the National Research Council (NRC) on the use of surveys and related methodologies to estimate the number of unauthorized U.S.-Mexico border crossings. The NRC recommended that DHS use survey data along with enforcement data to measure unauthorized flows and the effectiveness of border enforcement.

RAND has recommended the use of respondent-driven sampling for migrant surveys. This method would involve a nonrandom sample of individuals from the population of interest who invite others to participate, and they would invite more individuals to participate. Researchers can account for social networks and recruitment patterns in order to obtain estimates of population characteristics. To account for potential inaccurate responses from migrants, RAND also suggested the use of "group answers" that group illicit and licit activities together, therefore allowing migrants to answer questions without revealing any unlawful activities. This suggestion responds to migrants' possible hesitation to participate in a survey or provide accurate information for fear of subsequent law enforcement action. Surveys also pose certain limitations, such as issues about whether samples are representative of the overall migrant population, which is related to who is surveyed and where. Similarly, the composition of the migrant populations can also affect the ability of a survey to fully capture migration metrics. For instance, recent decreases in the proportion of unauthorized Mexican migrants entering the United States may call for the development of survey methodology that accounts for different source countries.

Since migrant flows cannot be measured directly, some have suggested the use of models that create estimates of unauthorized entry using proxy data. With respect to migrant flows, researchers have suggested human smuggler fees as a possible proxy for border control effectiveness. In theory, more effective border enforcement would make it more difficult for a smuggler to pass the border; this additional cost to them could be passed on to migrants through higher fees. However, smuggling fees are also sensitive to other factors, such as the risks and costs associated with providing "coyote" services, the cost of attempting to cross the border without a coyote, migration alternatives, the size and structure of the coyote service industry, migrants' ability to afford services, and the specific services the coyote is contracted to provide. A model using proxy data would need to account for other influencing factors in order to create a more reliable flow estimate.

Methodologies

Different methodologies can employ the use of enforcement data and other data in order to analyze the effectiveness of border security. For example, population surveys can be used to estimate flows of migrants to and from countries. Similar to the residual method used to estimate the undocumented population in the United States, Mexico's Censo de Población y Vivienda could be used to identify emigration risk factors by monitoring annual changes in population demographics that are not explained away by births or deaths. Additionally, comparisons across each country's population surveys could produce broad estimates of unauthorized migrant flows to and from each country. However, population surveys do have limitations, such as their relatively high costs, infrequent administration, and the possibility of undercounting those without stable housing.

Regression models could be used to estimate the size of the population that is likely to migrate and the proportion that will attempt to enter the United States. Models can account for and link changes in the risk population to economic, demographic, and social factors that may influence the decision to migrate. In order to obtain statistically significant results in a regression model, a sizable sample would be needed. Furthermore, the model would need to be sensitive to specific time periods and certain migrant cohorts.

Stratified sampling of border crossers would involve the Border Patrol randomly placing resources across sectors, taking into account the likelihood of resources being placed in a certain region and the number of apprehensions made there. This methodology could be used to estimate total apprehensions that would have been made if resources were placed in all sectors. Incidentally, this method could also assist in improving Border Patrol performance by increasing the chance of detecting new transportation routes and strategies by allowing for random placement. Another method for estimating detection in a sampled area would involve "red teaming," where migrants or agents are recruited to attempt crossing the border in order to establish the probability of interdiction.