Asylum: Variation Exists in Outcomes of Applications Across Immigration Courts and Judges
Published: 2016-11-14T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Tens of thousands of foreign nationals in the United States apply annually for asylum, which provides refuge to those who have been persecuted or fear persecution on protected grounds. EOIR's immigration judges decide asylum application outcomes in court proceedings. In 2008, GAO reported that EOIR data from October 1994 through April 2007 showed significant variation in the outcomes across immigration courts and judges (grants versus denials) of such applications.
The Senate Appropriations Committee report for DHS Appropriations Act, 2015, included a provision for GAO to update its 2008 report. This report examines (1) variation in asylum applications outcomes over time and across courts and judges; (2) factors associated with variability; and (3) EOIR's actions to facilitate asylum applicants' access to legal resources. GAO analyzed EOIR data—using multivariate statistics—on asylum outcomes from fiscal years 1995 through 2014, the most current data available at the time of GAO's analysis; reviewed EOIR policies and procedures; and interviewed EOIR officials and immigration judges about court proceedings and legal access programs. GAO observed asylum hearings in 10 immigration courts selected on the basis of application data and other factors.
section_title: What GAO Found
paragraphs: GAO analyzed the outcomes of 595,795 asylum applications completed by the Department of Justice's Executive Office for Immigration Review (EOIR) between fiscal years 1995 and 2014, and identified outcome variation both over time and across immigration courts and judges. From fiscal years 2008 through 2014, annual grant rates for affirmative asylum applications (those filed with the Department of Homeland Security (DHS) at the initiative of the individual and referred to an EOIR immigration judge) ranged from 21 to 44 percent. In the same period, grant rates for defensive asylum applications (those initiated before an immigration judge) ranged from 15 to 26 percent. Further, EOIR data indicate that asylum grant rates varied by immigration court. For example, from May 2007 through fiscal year 2014, the grant rate was 66 percent (affirmative) and 52 percent (defensive) in the New York, New York, immigration court and less than 5 percent (affirmative and defensive) in the Omaha, Nebraska, and Atlanta, Georgia, immigration courts.
GAO found that certain case and judge-related factors are associated with variation in the outcomes of asylum applications. For example, applicants who were represented by legal counsel were granted asylum at a rate 3.1 (affirmative) and 1.8 (defensive) times higher than applicants who were not represented. After statistically controlling for certain factors, such as judge experience and whether or not the applicant had dependents, GAO found variation remained in the outcomes of completed asylum applications across immigration courts and judges. For example, from May 2007 through fiscal year 2014, GAO estimated that the affirmative and defensive asylum grant rates would vary by 29 and 38 percentage points, respectively, for a representative applicant with the same average characteristics we measured, whose case was heard in different immigration courts. In addition, GAO estimated that the affirmative and defensive asylum grant rates would vary by 47 and 57 percentage points, respectively, for the same representative applicant whose case was heard by different immigration judges. GAO could not control for the underlying facts and merits of individual asylum applications because EOIR's case management system was designed to track and manage workloads and does not collect data on all of the details of individual proceedings. Nonetheless, the data available allowed GAO to hold constant certain factors of each asylum application, enabling GAO to compare outcomes across immigration courts and judges.
EOIR provides legal resources to targeted populations, including asylum applicants, through the Legal Orientation Program (LOP) and Legal Orientation Program for Custodians of Unaccompanied Alien Children (LOPC). EOIR and its contractor use LOP and LOPC site visits, monthly conference calls, and quarterly reports to monitor these programs. However, EOIR has not established performance measures, consistent with principles outlined in the GPRA Modernization Act of 2010, to determine whether these programs are having a measurable impact in meeting program objectives. Developing and implementing performance measures, including establishing a baseline, to determine whether LOP and LOPC are having a measurable impact would better position EOIR to make any adjustments necessary to improve the programs' performance.
section_title: What GAO Recommends
paragraphs: GAO recommends that EOIR develop and implement a system of performance measures, including establishing a baseline, to regularly evaluate the effectiveness of LOP and LOPC. EOIR concurred with GAO's recommendation.

section_title: Letter
paragraphs: Each year, tens of thousands of foreign nationals in the United States  apply for asylum, which provides refuge to those who have been  persecuted or fear persecution in their home countries on protected  grounds. U.S. immigration law provides that foreign nationals within the  United States may be granted humanitarian protection in the form of  asylum if they are unable or unwilling to return to their home country  because of past persecution, or a well-founded fear of future persecution  based on their race, religion, nationality, membership in a particular social  group, or political opinion. Decisions on asylum applications can carry  serious consequences—granting asylum to an applicant with a genuine  claim protects the asylee from being returned to a country where he or  she has been or could in the future be persecuted, but granting asylum to  an ineligible  individual jeopardizes the integrity of the asylum system.
U.S. Citizenship and Immigration Services (USCIS) within the Department  of Homeland Security (DHS) and the Executive Office for Immigration  Review (EOIR) within the Department of Justice (DOJ) share  responsibility for the U.S. asylum system. USCIS asylum officers  adjudicate affirmative asylum claims—that is, claims made at the initiative  of the individual who files an application for asylum with USCIS. If the  asylum officer does not grant asylum to an applicant who lacks lawful  status in the United States, his or her application is to be referred to EOIR  for adjudication during removal proceedings in immigration court. EOIR’s  immigration judges adjudicate defensive claims, which are those claims  first raised during removal proceedings in immigration court as a defense  to removal from the United States, as well as affirmative asylum claims  referred by USCIS. According to EOIR, from fiscal years 2011 through  2015, EOIR’s immigration judges completed more than 181,000 total  asylum applications in 58 immigration courts throughout the United  States.
In 2008, we reported on our analysis of EOIR data from fiscal years 1995  through April 2007 on the outcomes of asylum applications across  immigration courts and judges. Specifically, we reported that there was  significant variation in such outcomes during that time period. For  example, we found that the grant rate for affirmative asylum applications  ranged from 6 percent in the immigration court in Atlanta, Georgia, to 54  percent in New York, New York; the grant rate for defensive asylum  applications ranged from 7 percent in Atlanta, Georgia, to 35 percent in  San Francisco, California, and New York, New York. On the basis of our  multivariate statistical analysis, we also reported that grant rates for  affirmative and defensive asylum applications were more than three times  higher for individuals who were represented by legal counsel. As a result,  we recommended, among other things, that EOIR address disparities in  asylum outcomes that may be unwarranted and identify immigration  judges who may benefit from supplemental efforts to improve their  performance. EOIR implemented this recommendation by providing  training to all immigration judges and conducting analyses comparing  asylum grant rates for judges before and after the training. We also  reported that EOIR expanded access to legal resources and information  for individuals in removal proceedings in immigration court. In addition,  since 2008, EOIR has also expanded its training program for new  immigration judges (see app. I for more information on EOIR’s training for  immigration judges). In addition, we recommended that EOIR develop a  plan for supervisory immigration judges, including an assessment of the  resources and guidance needed to ensure that immigration judges  receive effective supervision. In response, EOIR analyzed the duties and  tasks performed by EOIR’s Assistant Chief Immigration Judges (ACIJ),  who oversee EOIR’s immigration judges. Among other things, in 2010,  EOIR published the ACIJ Handbook to assist ACIJs in carrying out their  roles as supervisors in immigration court.
The Senate Appropriations Committee Report that accompanied the DHS  Appropriations Act, 2015, includes a provision for us to update our 2008  report. This report (1) describes the extent of variation in the outcomes of  completed asylum applications over time and across immigration courts  and judges; (2) discusses the factors associated with variability  in the  outcomes of completed asylum applications; and (3) examines the extent  to which EOIR has taken action to facilitate access to legal resources,  including representation, for asylum applicants.
To describe the extent of variation in the outcomes of completed asylum  applications, we analyzed record-level data from EOIR on all immigration  court proceedings from fiscal year 1995 through fiscal year 2014. Our  analysis included all completed asylum applications between October 1,  1994, and September 30, 2014, the most current data available at the  time of our analysis, that involved asylum applicants from (1) countries  with a minimum of 800 affirmative and 800 defensive completed asylum  applications during the time period; (2) immigration courts with a minimum  of 800 affirmative and 800 defensive completed asylum applications  during the time period; and (3) judges with a minimum of 50 affirmative  and 50 defensive completed asylum applications in their primary court  during the time period. This combination yielded a total of 740,922  individuals with asylum applications that were completed between fiscal  years 1995 and 2014 included in our sample. Of those, 595,795 (or 80  percent) were included in our analysis of asylum application completions  and asylum grant rates. We used data from EOIR’s case management  system to identify immigration court proceedings where an immigration  judge had made a decision on an applicant’s asylum application. We  assessed the reliability of the data used in our analyses through  electronic testing, analyzing related database documentation, and  working with agency officials to reconcile discrepancies between the data  and documentation that we received. We determined that the data on  completions of asylum applications were sufficiently reliable for the  purposes of our reporting objectives.
To discuss the factors associated with variability in the outcomes of  completed asylum applications, we used multivariate statistical methods  to estimate an expected asylum grant rate for a representative applicant,  holding constant various characteristics of individual  cases, judges, and  courts. The scope of our analysis included the outcomes of all completed  asylum applications that met the screening criteria discussed above— specifically, we created an indicator for each application that reflected  whether the applicant received asylum. This allowed us to calculate  asylum grant rates as a proportion of all completed asylum applications.  Our analysis used a mixed logistic regression model to attribute the  unique contribution of certain factors in each completed asylum  application to variation in asylum grant rates. For a complete list of the  factors used in our analysis, see table 7 in appendix II. We modeled the  data separately by affirmative and defensive applications and time period.  Each applicant was associated with a judge, court, country of nationality,  and language. The results of our analysis allowed us to estimate the  range of expected affirmative and defensive asylum grant rates for a  representative applicant with the same average characteristics we  measured whose case was heard in different courts or by different  judges. EOIR’s case management system was designed to track and  manage cases and does not collect data on all of the details of individual  proceedings. As a result, our analysis could not hold constant the  underlying facts and merits of individual asylum applications because  EOIR’s case management system does not collect that information.  Nonetheless, the data available allowed us to control for certain factors of  each asylum application, enabling us to compare outcomes across  immigration courts and judges. For a detailed discussion of our  multivariate statistical methods, see appendix III.
To examine the extent to which EOIR has taken action to facilitate access  to legal resources, we reviewed the documents on EOIR’s legal  resources programs, including congressional budget justifications and  EOIR documentation on individual programs and initiatives. In particular,  we focused our review on the Legal Orientation Program (LOP) and the  Legal Orientation Program for Custodians of Unaccompanied Alien  Children (LOPC) because these are among EOIR’s longest-standing legal  access programs, they have received more funding than EOIR’s Office of  Legal Access Program’s (OLAP) other programs, and they have served  tens of thousands of individuals at immigration courts throughout the  United States. Regarding LOP and LOPC, we examined documentation  outlining program objectives and deliverables, quarterly and annual  reports, site visit reports and evaluation reports. In addition, we analyzed  data on the number of individuals served by LOP (fiscal years 2008  through 2015) and LOPC (fiscal years 2011 through 2015). We also  interviewed EOIR officials responsible for overseeing the program. We  analyzed all of this information in light of principles outlined in the  Government Performance and Results Act (GPRA) of 1993, as updated  by the GPRA Modernization Act (GPRAMA)  of 2010, to assess EOIR’s  efforts to measure progress and results against performance measures.
To address all three objectives, we reviewed EOIR documents describing  EOIR asylum applications statistics; policies and procedures related to  immigration court proceedings; manuals and documents describing EOIR’s case management system; guidance and training provided to  judges for adjudicating asylum applications; and documentation on  EOIR’s programs aimed at providing legal resources to individual  in  proceedings in immigration court. We also interviewed EOIR  headquarters officials responsible for overseeing immigration court  proceedings, EOIR’s case management system, and immigration judge  training and guidance and legal access programs. We visited 10  immigration courts in Tacoma, Washington; Seattle, Washington; New  York, New York; Newark, New Jersey; Elizabeth, New Jersey; Los  Angeles, California; Adelanto, California; Miami, Florida; Krome, Florida;  and, Arlington, Virginia. At these locations, we interviewed supervisory  immigration judges, immigration judges, court administrators and  observed removal proceedings for individuals applying for asylum. We  interviewed supervisory immigration judges, immigration judges and court  administrators by telephone at two additional immigration courts in El  Paso, Texas and Atlanta, Georgia. We selected these sites based on a  variety of factors, including number of asylum cases adjudicated, courts  with dockets of individuals who are being detained by DHS throughout the  course of their immigration court proceedings, as well as courts with  dockets of individuals who are not detained, and a range of grant rates,  and circuit court jurisdiction. We also interviewed DHS attorneys  responsible for representing the U.S. government in removal proceedings  in immigration court and staff from pro bono/advocacy organizations  proximate to each immigration court to gain their perspectives on (1)  potential factors associated with changes in asylum applications  outcomes over time and (2) immigration court efforts to facilitate access  to legal resources for asylum seekers. The results from our site visits  cannot be generalized more broadly to all immigration courts or  immigration judges. However, they provided important context and  insights into EOIR’s efforts to assist immigration judges in adjudicating  asylum applications, perspectives on training and guidance provided to  immigration judges, and EOIR efforts to facilitate access to legal  resources.
We conducted this performance audit from January 2015 to October 2016  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives. Appendix II provides additional  information on our scope and methodology.
subsections: section_title: Background
subsections: section_title: DOJ’s Process for Adjudicating Asylum Applications
paragraphs: Immigration court hearings are adversarial proceedings in which an  individual appears before an immigration judge for adjudication of  charges of removability from the United States, and may raise defense to  removal by seeking asylum and other forms of relief or protection from  removal. Immigration judges are to assess whether an applicant has  credibly demonstrated that he or she is eligible to be granted asylum, and  whether a grant of asylum is warranted as a matter of discretion. An  applicant is eligible for asylum if he or she (1) applies from within the  United States; (2) suffered past persecution, or has a well-founded fear of  future persecution, based on race, religion, nationality, membership in a  particular social group, or political opinion; and (3) is not statutorily barred  from applying for or being granted asylum. When making a credibility  determination, the immigration judge is to consider the totality of the  circumstances and all relevant factors. EOIR follows the same  procedures for defensive asylum applications and affirmative asylum  referrals from USCIS. For affirmative asylum referrals, the immigration  judge reviews the case de novo, meaning that the judge evaluates the  applicant’s affirmative asylum application anew and is not bound by the  USCIS asylum officer’s previous determination. As shown in Figure 1, in  conducting removal proceedings and adjudicating cases before them,  judges in immigration courts conduct an initial master calendar hearing to,  among other things, ensure the applicant understands the immigration court proceedings and schedule the merits hearing, during which the  judge hears claims for immigration relief, such as asylum.
An individual in removal proceedings in immigration court may request  multiple types of relief before an immigration judge, and, if deemed  removable, the judge makes a decision as to whether the individual  satisfies the applicable eligibility requirements of any requested relief.  During the merits hearing, judges hear testimony from the respondent  and any other witnesses, oversee cross-examinations by DHS’s U.S.  Immigration and Customs Enforcement (ICE) trial attorneys, and review  relevant evidence, including evidence related to claims for asylum or  other relief. Additionally,  the immigration judge may question the applicant  or other witnesses. Judges render oral or written decisions at the end of  immigration court proceedings. As a general matter, immigration judges  have discretion in rendering decisions under 8 C.F.R. § 1003.10(b), which  states that “n deciding the individual case before them, and subject to  the applicable governing standards, immigration judges shall exercise  their independent judgment and discretion and may take any action  consistent with their authorities under the Act and regulations that is  appropriate and necessary for the disposition of such cases.” In addition,  asylum is a discretionary form of relief under U.S. immigration law, in  contrast to a mandatory type of immigration relief or protection, such as  withholding of removal. With respect to asylum, section 208(a) of the  Immigration and Nationality  Act (INA) provides that the Attorney General  or the Secretary of Homeland Security may grant asylum to an applicant  who meets the definition of a refugee under INA  § 101(a)(42)(A).
section_title: Potential Outcomes of Asylum Applications in Immigration Court
paragraphs: There are five possible outcomes of an asylum application—grant, denial,  withdrawal, abandonment, or other. If a judge grants asylum, the asylee is  eligible to apply for adjustment to lawful permanent resident status after  one year. Individuals who have been granted asylum are considered  qualified individuals for the purpose of eligibility  for federal, and state or  local, public benefits. Asylees are authorized for employment in the  United States as a result of their asylum status. The other four possible  outcomes of an asylum application—denial, withdrawal, abandonment,  and other—do not result in an asylum grant, and therefore do not convey  such benefits. According to EOIR headquarters officials, the outcome of  an individual’s asylum application is not necessarily determinative of the  outcome of the individual’s overall removal case. For example, an  individual whose asylum application ends in an outcome of denial,  withdrawal, or other may still be granted other forms of relief or protection  from removal, such as withholding or cancellation of removal. Or, the  individual may be found removable and not eligible for any form of relief  or protection from removal. In this latter case, the individual would be  subject to removal pursuant to the judge’s order of removal once it has  become administratively final. According to EOIR headquarters officials,  an outcome of other for an asylum application is recorded if the judge did  not need to reach a decision on the merits of the asylum application. For  example, ICE’s Office of the Principal Legal Advisor may exercise  prosecutorial discretion in seeking, and the immigration judge may grant,  administrative closure of removal proceedings for an individual whose  case is not an enforcement priority for DHS.
section_title: EOIR’s Legal Access Programs
paragraphs: EOIR’s OLAP oversees various programs aimed at increasing access to  legal services and information for individuals appearing before  immigration court, as well as improving the effectiveness of EOIR’s  adjudication process. According to EOIR, since April 2000, OLAP has  worked to improve access to legal information and counseling and  increase the level of representation for individuals appearing before the  immigration courts. This has been carried out primarily through initiatives  that facilitate access to information and help create new incentives for  attorneys, non-profit organizations and their representatives, and law  students to accept pro bono cases. In particular, OLAP established LOP  and LOPC to provide legal information to targeted populations regarding  immigration court processes and possible forms of relief, including  asylum. LOP, which EOIR established in 2003, provides legal  information to individuals in detention who are awaiting their removal  proceedings regarding their rights and the immigration process to assist  them in making better informed decisions earlier in immigration court  proceedings. LOPC, which EOIR established in 2010, provides legal  orientation presentations to custodians of unaccompanied alien children  (UAC) released from custody from the Office of Refugee Resettlement, to  ensure the child’s appearance at all immigration court hearings. To  provide LOP and LOPC services, OLAP contracts with the Vera Institute  of Justice (Vera), which subcontracts with non-profit organizations across  the country. OLAP has also established other legal access services, including self-help materials and other programs intended to increase pro  bono representation and serve vulnerable populations.
section_title: EOIR Data Indicate that Outcomes of Completed Asylum Applications Varied Over Time and Across Immigration Courts and Judges
subsections: section_title: Asylum Application Outcomes
paragraphs: A total of 740,922 individuals with asylum applications that were  completed between fiscal years 1995 and 2014 met the criteria to be  included in our sample. Of those, 595,795 (or 80 percent) were included  in our analysis of asylum application completions and asylum grant  rates. Since 2008, when we last reported on asylum outcomes, EOIR’s  immigration judges granted asylum to between 4,508 and 6,090  individuals in our sample each year.
The number of individuals granted affirmative asylum each year has  ranged from 3,060 to 4,166. The number of individuals denied affirmative  asylum decreased each fiscal year from fiscal year 2008 to fiscal year  2014. Figure 2 illustrates outcomes of completed affirmative asylum  applications during this period.
Since our 2008 report, the number of defensive asylum applications that  were granted by immigration judges has ranged from 1,343 to 1,924 per  year (fiscal years 2008 through 2014). As illustrated in Figure 3, the  number of defensive asylum applications that were denied increased  each fiscal year from 2011 to 2014. The number of defensive asylum  applications with outcomes of “other” increased sharply from 828 in fiscal  year 2009 to 3,960 in fiscal year 2014. As previously discussed, an  outcome of “other” for an asylum application is recorded if the immigration  judge did not need to reach a decision on the merits of the application.  EOIR officials stated that the impact of DHS’s 2011 memorandum on  prosecutorial discretion for immigration enforcement is a key reason for  this increase—when DHS exercises prosecutorial discretion during  removal proceedings, judges are able to administratively close removal  cases without reaching a decision on the merits of the case, including  associated asylum applications.
section_title: Asylum Grant Rates
paragraphs: For fiscal years 1995 through 2014, EOIR data indicate that affirmative  and defensive asylum grant rates varied over time and across  immigration courts, applicants’ country of nationality, and individual  immigration judges within courts. In 2008, we reported the asylum grant  rate as the number of asylum applications granted divided by the total  number of all granted and denied applications. In other words, the grant  rate depended on the number of completed asylum applications that  ended in a decision by the immigration judge of either grant or denial. In  this report, to provide data on all of the outcomes of completed asylum  applications during our period of analysis, we included the five possible  outcomes in calculating the asylum grant rate. These include applications  that were granted and denied, as well as those that had an outcome of  withdrawn, abandoned, or other. Thus, for the purposes of this report, we  define the asylum “grant rate” as the number of granted asylum  applications divided by the total number of completed applications  (completions include applications that are granted, denied, withdrawn,  abandoned, or end in an outcome of “other”) during the period of analysis.  An expanded definition allows us to describe the outcomes of completed  asylum applications, including outcomes other than granted or denied,  and trends in those outcomes. Using this definition, annual grant rates for  affirmative asylum applications ranged from 21 to 44 percent from fiscal  year 2008 through fiscal year 2014. In the same time period, grant rates  for defensive asylum applications ranged from 15 to 26 percent. Figure 4  illustrates asylum grant rates for fiscal years 1995 through 2014.
In 2008, we reported that affirmative asylum grant rates varied by  applicants’ country of nationality. Country conditions vary from one  country to another, including countries’ political climates and human rights  records. Thus, differences in the extent to which applicants from various  countries are granted or denied asylum in the United States is not  unexpected. For the period since our 2008 report, from May 2007 through  fiscal year 2014, affirmative asylum grant rates continued to vary by  applicant nationality, as shown in Figure 5. For example, the grant rate  for affirmative applicants exceeded 50 percent for asylum applicants from some countries, including Sudan, Cameroon, Burma (Myanmar), Sri  Lanka, Yugoslavia, China, Ethiopia, Egypt, and Eritrea. For other  countries, the affirmative asylum grant rate was less than 5 percent,  including El Salvador, Cuba, Mexico, Ecuador, Nicaragua, Guatemala,  Honduras, and Philippines.
In 2008, we reported that defensive asylum grant rates also varied by  applicants’ country of nationality. From May 2007 through fiscal year  2014, defensive asylum grant rates continued to vary by country of  nationality. For example, the grant rate for defensive applicants exceeded  50 percent for asylum applicants from some countries, including  Cameroon, Iraq, China, Burma (Myanmar), Ethiopia, and Eritrea (see fig.  6). For other countries, the defensive asylum grant rate was less than 5  percent, including Laos, Philippines, and Mexico.
In 2008, we reported that affirmative and defensive asylum grant rates  varied depending on the immigration court that heard the case. We found  similar levels of variation from May 2007 through fiscal year 2014. For  example, the grant rate for affirmative applicants in the New York  immigration court was 66 percent, while the grant rate in the Omaha and  Atlanta immigration courts was less than 5 percent (see fig. 7).
From May 2007 through fiscal year 2014, defensive asylum grant rates  also continued to vary depending on the immigration court that heard the  case. For example, the grant rate in the New York immigration court was  52 percent, while the grant rates in the Omaha, Atlanta, and Bloomington,  Minnesota, courts were less than 5 percent (see fig. 8).
From May 2007 through fiscal year 2014, within each immigration court,  the asylum grant rate for individual immigration judges varied. Affirmative  asylum grant rates for individual judges ranged from 0 to 83 percent.  Among the four courts with more than 15 judges in our sample, the  difference between the lowest and highest grant rates in each court, or  the “range,” extended from 17 percentage points in Miami to 60  percentage points in New York, as illustrated in Table 1.
Figure 9 illustrates the range of affirmative asylum grant rates for judges  in their primary immigration courts from May 2007 through fiscal year  2014.
From May 2007 through fiscal year 2014, defensive asylum grant rates  across individual judges ranged from 0 to 80 percent. Among the four  courts with more than 15 judges in our sample, the difference between  the lowest and highest grant rates in each court, or the “range,” extended  from 16 percentage points in Miami to 70 percentage points in New York,  as illustrated in Table 2.
Figure 10 illustrates the range of defensive asylum grant rates for judges  in their primary immigration courts from May 2007 through fiscal year  2014.
section_title: Grant Rates for Asylum Applications Can be Affected by Case Characteristics and Outcome Variance Persists when Controlling for Certain Factors
subsections: section_title: Potential Explanations Offered for Variation in Outcomes of Asylum Applications
paragraphs: Immigration judges have discretion in rendering decisions and asylum is a  discretionary form of relief under U.S. immigration law. Immigration  judges we spoke with in six immigration courts stated that the judicial  discretion provided for in U.S. immigration law is one reason that  decisions on asylum applications, even among judges in the same court,  could vary. For example, while a judge adjudicating an asylum claim is  permitted by law to require that corroborating evidence be provided in  support of otherwise credible, persuasive, and factually specific  testimony, such testimony could also be legally sufficient for another  judge to determine that the applicant has met his or her burden of proof  without corroboration. In addition, researchers have offered potential  explanations for variation in asylum application outcomes. For example, a  study published in 2015 by researchers at the University of Texas at  Dallas found that variation in asylum application outcomes is not  unexpected given the institutional conditions under which judges operate,  including the particular requirements of the law, the lack of corroborating  evidence that is common in asylum cases, and the difﬁculty of assessing  credibility.
section_title: EOIR Data Show Variation in Outcomes of Asylum Applications Across Courts and Judges Persists when Holding Certain Case and Judge Characteristics Constant
paragraphs: On the basis of our analysis of EOIR data from fiscal years 1995 through  2014, we found that asylum grant rates varied across courts and judges,  holding constant various case and judge characteristics. While we were  able to analyze various characteristics of cases, judges, and courts, the  design of EOIR’s case management system did not allow us to measure  and control for a number of factors—such as the nature or key  characteristics of the claim of persecution or the gender of the applicant— that may be relevant to variability  in asylum grant rates. EOIR’s case  management system was designed to manage and track workloads  across immigration courts rather than to collect all data on the details of  individual proceedings. In addition, each asylum application presents  unique facts and circumstances that judges must consider in rendering  their decisions. As a result, our analysis could not hold constant the  underlying facts and merits of individual asylum applications because  EOIR’s case management system does not collect that information.  Further, according to EOIR officials, immigration judges are not required  to document each factor in the case management system they consider in  their overall adjudication of an asylum claim. Some of these factors may  be relevant to variation in asylum application outcomes, such as whether  there was an adverse credibility determination. In addition, there may be  other factors that could affect variation in asylum grant rates including  whether a particular judge was assigned to a juvenile or other docket that  does not randomly assign cases to judges. Nonetheless, the data  available allowed us to control for certain factors of each asylum  application, which enabled us to compare asylum outcomes across  immigration courts and judges.
subsections: section_title: Factors Associated with Grant Rates
paragraphs: We used multivariate statistical methods to estimate an expected asylum  grant rate for a representative applicant, holding constant various  characteristics of individual cases, judges, and courts. We estimated  rates separately for affirmative and defensive asylum applications and for  each of four time periods across fiscal years 1995 through 2014.  Analyzing asylum grant rates for a representative applicant, holding  constant factors such as whether the applicant was represented by  counsel, applicant country of nationality, and judge gender, allowed us to  compare asylum grant rates across courts and judges (see Appendix III  for more details about our analysis).
For the period since our 2008 report (May 2007 through fiscal year 2014),  we estimate that, for each of the factors noted below, a representative  affirmative asylum applicant would be granted asylum at a rate higher (or  lower) than an applicant with differing case factors, and the difference  would be statistically significant. The remaining factors we held  constant—judge gender, most circuit courts, and judge and court asylum  caseloads—had no statistically distinguishable associations with  affirmative asylum grant rates for cases heard in immigration court. For  example:
Representation. Applicants who were represented by legal counsel  were granted asylum at a rate 3.1 times higher than applicants who  were not represented.
Dependents. Applicants with at least one dependent were granted  asylum at a rate 1.7 times higher than applicants without dependents.
Date of affirmative asylum  application. Applicants who applied  within one year of entering the United States were granted asylum at  a rate 2.4 times higher than applicants who applied later.
Presidential administration under which judges were appointed.
Judges who were appointed by the Attorney General during the  administrations of Presidents George H.W. Bush, Reagan, Carter,  Nixon, or Johnson were generally more likely to grant asylum than  those appointed during the administrations of Presidents Clinton,  George W. Bush, and Obama, holding constant years of experience  as a judge. Judges appointed during the administrations of Presidents  Clinton, George W. Bush, and Obama granted asylum at rates that  were statistically indistinguishable from each other.
Judge years of experience. Judges with more experience were less  likely to grant asylum, holding constant the presidential administration  of appointment. Judges with an additional 7 years of experience were  28 percent less likely to grant asylum.
Many of the factors we analyzed had similar associations for both  affirmative and defensive asylum grant rates. From May 2007 through  fiscal year 2014, we estimate that, for each of the factors noted below, a  representative defensive asylum applicant would be granted asylum at a  rate higher (or lower) than an applicant with differing case factors, and the  difference would be statistically significant. Judge experience and the  presidential administration under which judges were appointed had no  statistically distinguishable associations with defensive asylum grant  rates. For example:
Representation. Applicants who were represented by legal counsel  were granted asylum at a rate 1.8 times higher than applicants who  were not represented.
Dependents. Applicants with at least one dependent were granted  asylum at a rate 1.7 times higher than applicants without dependents.
Date of defensive asylum application. Applicants who applied  within one year of entering the United States were granted asylum at  a rate 5.0 times higher than applicants who applied later. This  association is substantially larger than the association for affirmative  applications made during the same time, and the difference is  statistically distinguishable from zero.
Judge gender. Female judges granted asylum for defensive  applications at a rate 1.4 times higher than male judges. There was  no statistically meaningful association between judge gender and  asylum grant rates among affirmative cases.
section_title: Asylum Grant Rates Varied Significantly Across Immigration Courts for a Representative Applicant After Controlling for Certain Factors
paragraphs: From May 2007 through fiscal year 2014, we estimated that the  affirmative asylum grant rate would vary by 29 percentage points if  different immigration courts heard the case of a representative applicant  with the same average characteristics we measured. As shown in table  3, the grant rate for a representative applicant in one court would be 19  percent, whereas it would be 48 percent in another court for the same  representative applicant. The size of this range varied across the four  time periods we analyzed, from a low of 10 percentage points in the first  period to a high of 37 percentage points in the second period.
For defensive asylum completions in the most recent period, we  estimated that the defensive asylum grant rate would vary by 38  percentage points if different immigration courts heard the case of a  representative applicant with the same average characteristics we  measured. As shown in table 3, the grant rate for a representative  applicant in one court would be 18 percent, whereas it would be 56  percent in another court for the same representative applicant. The size  of this range varied across the four time periods we analyzed, from a low  of 11 percentage points in the first period to a high of 38 percentage  points in the third period.
section_title: Asylum Grant Rates Varied Significantly Across Immigration Judges for a Representative Applicant After Controlling for Certain Factors
paragraphs: From May 2007 through fiscal year 2014, we estimated that the  affirmative asylum grant rate would vary by 47 percentage points if  different immigration judges heard the case of a representative applicant  with the same average characteristics we measured. As shown in table  4, the grant rate for a representative applicant by one judge would be 13  percent, whereas it would be 60 percent before another judge for the  same representative applicant. The size of this range varied across the  four time periods we analyzed, from a low of 29 percentage points in the  first period to a high of 47 percentage points in the fourth period.
For completed defensive asylum applications in the most recent period,  we estimated that the defensive asylum grant would vary by 57  percentage points if different immigration judges heard the case of a  representative applicant with the same average characteristics we  measured. As shown in table 4, the grant rate for a representative  applicant by one judge would be 11 percent, whereas it would be 68  percent before another judge for the same representative applicant. The  size of this range varied across the four time periods we analyzed, from a  low of 22 percentage points in the first period to a high of 57 percentage  points in the fourth period.
section_title: EOIR Has Programs in Place to Facilitate Access to Legal Resources for Asylum Applicants, but Could Improve Efforts to Measure Program Results
subsections: section_title: EOIR Facilitates Legal Resources for Asylum Applicants and Monitors Results
paragraphs: Through LOP and LOPC, EOIR has provided legal orientations to  approximately 450,000 individuals. EOIR established LOP in 2003 to  provide legal orientations to persons in detention prior to their first hearing  before an immigration judge. In 2010, EOIR established LOPC to provide  legal orientation presentations to custodians of UAC released from  custody from the Office of Refugee Resettlement to help ensure the  child’s appearance at all immigration proceedings. LOP and LOPC  provide legal information through group and individual orientations, as  well as self-help workshops and pro bono legal referrals to those in  removal proceedings in immigration court. According to DOJ and EOIR,  LOP and LOPC are key efforts to, among other things, increase the  efficiency of immigration court proceedings and improve access to basic  legal services for individuals in such proceedings. For example, DOJ’s  Congressional Budget Submission for Fiscal Year 2016 reported that  LOP is designed to assist detained individuals in making better informed  decisions earlier in their immigration court proceedings, thereby improving  access to basic legal services, especially for low income individuals, while  increasing the efficiency of the court hearing and detention process.  Regarding LOPC, in February 2016, EOIR’s Director testified that LOPC  “helps the immigration court process function more efficiently and  effectively by providing valuable information to the custodians of children  who arrive in the United States without a parent or guardian.”
Since 2005, EOIR has contracted with the Vera Institute of Justice (Vera),  which subcontracts with non-profit organizations, to administer LOP and  LOPC. As of July 2016, EOIR administered LOP at 36 detention facilities,  2 non-detained locations and LOPC in 15 cities across the United States.  According to EOIR’s September 2011 blanket purchase agreement with  Vera and associated statements of work, LOP and LOPC at each site, are  to provide, among other things, individual  and group legal orientations  and the dissemination of written legal orientation materials to individuals  in immigration removal proceedings. LOP and LOPC sites provide self- help workshops, pro bono referral services and individual and group  orientations. In particular, these orientations are to review the range of  rights available to such individuals and the forms of relief from removal  that may or may not be available to them. Table 5 provides information on  the goals and objectives for LOP and LOPC in the 2011 blanket purchase  agreement.
Consistent with the 2011 blanket purchase agreement, Vera submits  quarterly reports to EOIR with data on, for example, the number of unique  individuals who attended legal orientations and the number of referrals to  pro bono legal services during the prior quarter. According to Vera’s  quarterly report summarizing LOP data through fiscal year 2015, between  about 35,000 and about 65,000 individuals attended LOP legal  orientations per year from fiscal years 2008 through 2015 (see fig. 11).  According to EOIR headquarters officials, the decrease in the number of  individuals attending LOP orientations since fiscal year 2011 mirrors the  decrease in the number of individuals detained and placed in removal  proceedings during this time period.
Regarding LOPC, according to Vera’s quarterly report summarizing  LOPC data through fiscal year 2015, legal orientations to custodians  increased from about 1,500 in fiscal year 2011, when the program  started, to almost 14,000 in fiscal year 2015 (see fig. 12).
EOIR headquarters officials stated that they review information that Vera  provides through the quarterly reports to check compliance with contract  requirements and to monitor the programs’ progress against stated  objectives. In addition, according to EOIR officials, Vera and EOIR  officials conduct site visits of LOP and LOPC sites to meet with local legal  service organizations to observe presentations and workshops and  discuss program performance. Vera also conducts monthly meetings  with site staff. EOIR officials responsible for overseeing LOP and LOPC  also noted that they hold regular discussions with Vera staff and use the  information collected by Vera to monitor progress. In particular, EOIR  officials stated that they monitor Vera’s data to identify increases or  decreases in the number of individuals served in each LOP and LOPC  location to determine site staffing levels or changes to existing sites.
section_title: EOIR Could Improve Efforts to Measure LOP and LOPC Effectiveness
paragraphs: EOIR monitors LOP and LOPC through a variety of mechanisms, but has  not established a system of performance measures, including  establishing a baseline, to regularly evaluate the effectiveness of LOP  and LOPC to determine whether these programs are having a  measurable impact in meeting program objectives. EOIR officials stated  that they have monitored LOP and LOPC performance through site visits  and monthly conference calls with Vera, as well as Vera’s quarterly  reports, annual reports, and budget analyses. Regarding LOP, EOIR  officials stated that they have used many of the findings and  recommendations from a one-time evaluation of the program in 2008 as a  guide for monitoring LOP performance. The evaluation, conducted by  Vera at the request of EOIR, found that LOP participants moved faster  through immigration courts, received fewer in absentia removal orders  and that LOP could effectively prepare individuals to represent  themselves in immigration courts, among other things. In the 2008 LOP  evaluation, Vera also concluded that immigration judges stated that LOP  increased immigration court efficiency and detention facility staff stated  that LOP improved detention conditions. EOIR’s 2011 blanket purchase  agreement related to LOP and LOPC states that EOIR can request that  the Contractor (i.e., Vera) produce various deliverables as needed and/or  as requested by EOIR, including an evaluation of LOP program outcomes  and a “performance outcome measurement program” to accurately and  continuously measure and obtain quantitative  and qualitative data on  LOP’s stated objectives. Although the 2008 evaluation provided EOIR  officials with valuable insight into the effectiveness of the LOP program,  EOIR officials stated that they have not requested that Vera conduct a  similar comprehensive outcome performance measurement program  consistent with the 2011 blanket purchase agreement. Further, as of July  2016, EOIR officials stated that they have not required Vera to develop a  formal performance outcome measurement program, or other formal  performance measures, for LOP as allowed for in the blanket purchase  agreement because of the variation in how LOP providers operate  programs at each site.
Regarding LOPC, the 2011 blanket purchase agreement states that, as  specified in an executed blanket purchase agreement task order, Vera is  to develop a system for measuring the performance of the program  against its stated objectives by collecting program data on the number of  UAC custodians served and the number of UACs represented by counsel,  among other things. In addition, EOIR officials stated that they requested  that Vera evaluate LOPC’s performance in 2014, consistent with the 2011  agreement. In August 2016, EOIR provided us a copy of the final  evaluation report. Among other things, the report states that in absentia  rates were lower for UAC whose custodians attended an LOPC  orientation. In addition, custodians who Vera surveyed after attending an  LOPC orientation reported learning about children’s legal rights and  options for relief, as well as the importance of appearing in immigration  court. In the report, Vera recommended that EOIR and Vera, among other  things, consistently identify UAC in EOIR’s case management system and  follow up with custodians to encourage attendance at orientations. EOIR  officials stated in July 2016 that they do not plan to request another  evaluation of LOPC, but will use these evaluation findings and  recommendations to make any necessary program improvements.
Consistent with requirements outlined in GPRAMA,  performance  measurement is the ongoing monitoring and reporting of program  accomplishments, particularly towards pre-established goals and  agencies are to establish performance measures to assess progress  towards goals.   While GPRAMA  is applicable to the department or  agency level, performance goals and measures are important management tools to all levels of an agency, including the program,  project or activity level. Agencies can use performance measurement to  make various types of management decisions to improve programs and  results, such as developing strategies and allocating resources, and  identify problems and take corrective action. Because of its ongoing  nature, performance measurement can serve as an early warning system  to management and as a vehicle for improving accountability to the  public.
EOIR headquarters officials stated that they have not established  performance measures for LOP and LOPC because of the variation in  how providers operate these programs at detention facilities and non- detained immigration courts, which results in the inability to have a “one- size fits all” approach. Rather, EOIR headquarters officials explained that  program managers at headquarters regularly assess LOP and LOPC  sites on an individual basis to (1) identify the number of individuals served  by each full time staff position with the local subcontracted organization  and (2) determine whether demand for services has increased or  decreased. Based on this information, headquarters officials make  necessary adjustments, such as closing a site with decreased demand for  services and opening a new site with new demand. EOIR officials stated  that this process provides flexibility  to make changes at LOP and LOPC  sites as needed in response to differing conditions at each location.  However, EOIR has not established baselines for these data it collects  from Vera and therefore does not have adequate performance  measurements against which to compare the data to determine whether  the programs are achieving stated goals. Solely tracking increases or  decreases in program data, such as changes in the number of individuals  served in detention facilities, does not allow EOIR to fully evaluate its  LOP and LOPC programs as such changes in the data may not be an  indicator of program success or increased efficacy. Developing and  implementing performance measures, including establishing a baseline,  to independently  and periodically determine whether LOP and LOPC are  having a measurable impact would better position EOIR to make any  adjustments necessary to improve the programs’ performance.
section_title: Conclusions
paragraphs: EOIR’s primary mission is to adjudicate immigration cases in a careful  and timely manner while ensuring the standards of due process and fair  treatment for all parties involved. EOIR’s LOP and LOPC aim to achieve  the dual goals of increasing the efficiency of immigration court  proceeding, while also increasing individuals’ ability to make timely  decisions about their immigration cases through receipt of early and  accurate legal information. Our multivariate analysis of EOIR data on  completed asylum applications from May 2007 through fiscal year 2014  indicates that whether an applicant has legal representation, in particular,  can have a significant effect on the outcome of the application. Since  2008, EOIR has facilitated access to legal resources to thousands of  individuals through LOP and LOPC legal orientations. While EOIR  monitors LOP and LOPC through a variety of mechanisms, establishing a  system of performance measures, including establishing a baseline, could  better position EOIR to determine whether these programs are having a  measurable impact in meeting program objectives and goals.
section_title: Recommendation for Executive Action
paragraphs: To better assess whether the LOP and LOPC are having a measurable  impact in meeting their program objectives, the Director of EOIR should  develop and implement a system of performance measures, including  establishing a baseline, to regularly evaluate the effectiveness of LOP  and LOPC and assess whether the programs are achieving their stated  goals.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this report to DOJ and DHS for their review and  comment. DOJ did not provide official written comments to include in this  report. However, in an e-mail received on October 24, 2016, an Associate  General Counsel for EOIR stated that EOIR concurred with our  recommendation. DHS did not provide written comments on our draft  report. DOJ and DHS provided technical comments, which we  incorporated as appropriate.
Should you or your staff have questions about this report, please contact  me at (202) 512-8777 or gamblerr@gao.gov. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on  the last page of this report. GAO staff that made key contributions to this  report are listed in appendix IV.
section_title: Appendix I: Executive Office for Immigration Review (EOIR) Training for Immigration Judges
paragraphs: EOIR provides training to new immigration judges and annual training to  all incumbent judges to assist them in rendering decisions on asylum  applications. Since 2008, EOIR has expanded its training program for  new immigration judges (see table 6).
In addition to training requirements for new immigration judges, EOIR  holds annual conferences for all immigration judges at EOIR  headquarters. EOIR held an annual conference in person for incumbent  immigration judges in fiscal years 2008, 2009, 2010 and 2015. According  to EOIR officials, EOIR did not host the annual conference in fiscal years  2011, 2012, and 2013 due to resource constraints. In fiscal year 2014,  EOIR planned to hold the conference, but canceled it due to the increase  in workload that resulted from the large numbers of unaccompanied  children and families who were apprehended at the southwest border and  placed in removal proceedings in the summer of 2014, according to EOIR  officials. According to the EOIR headquarters official responsible for  training, EOIR decides whether to have the conference each year based  on available funding. During years when the conference was not offered in person, EOIR video recorded conference sessions on discs and mailed  them to immigration judges to watch.
section_title: Appendix II: Objectives, Scope, and Methodology
paragraphs: Our objectives were to (1) describe the extent of variation in the outcomes  of completed asylum applications over time and across immigration  courts and judges; (2) discuss the factors associated with variability in the  outcomes of completed asylum applications; and (3) examine the extent  to which the Executive Office for Immigration Review (EOIR) has taken  action to facilitate access to legal resources, including representation, for  asylum applicants.
To address all three objectives, we reviewed EOIR documents describing  asylum applications statistics; policies, procedures related to immigration  court proceedings; manuals and documents describing EOIR’s case  management system; guidance and training provided to judges for  adjudicating asylum applications; and documentation on EOIR’s legal  access programs. We interviewed EOIR headquarters officials  responsible for overseeing immigration court proceedings; EOIR’s case  management system; immigration judge training and guidance and legal  access programs. We also visited 10 immigration courts in Tacoma,  Washington; Seattle, Washington; New York, New York; Newark, New  Jersey; Elizabeth, New Jersey; Los Angeles, California; Adelanto,  California; Miami, Florida; Krome, Florida; and, Arlington,  Virginia. At  these locations, we interviewed supervisory immigration judges,  immigration judges, court administrators and observed removal  proceedings for individuals applying for asylum. We also interviewed  supervisory immigration judges, immigration judges and court  administrators by telephone at two additional immigration courts in El  Paso, Texas and Atlanta, Georgia. Because immigration judges have  large caseloads and constrained schedules, we interviewed judges who  were available to speak with us during our scheduled interviews. We  interviewed the court administrator and supervisory immigration judge for  each immigration court we visited or contacted. During the immigration  court interviews, we used semi-structured interview questions to ask  about (1) potential factors associated with changes in asylum applications  outcomes over time, (2) training and guidance provided to immigration  judges for adjudicating asylum applications, and (3) EOIR immigration  court efforts to facilitate access to legal resources for asylum seekers. We  selected these sites based on a variety of factors, including courts that  adjudicate a large number of asylum cases, detained and non-detained  dockets, a range of grant rates, and circuit court jurisdictions. We also  interviewed staff from U.S. Immigration and Customs Enforcement (ICE),  Office of the Principal Legal Advisor, Offices of the Chief Counsel and  asylum pro bono/advocacy organizations proximate to each immigration  court interviewed to gain their perspectives on (1) potential factors  associated with changes in asylum applications outcomes over time and  (2) immigration court efforts to facilitate access to legal resources for  asylum seekers. The results from our site visits cannot be generalized  more broadly to all immigration courts or immigration judges. However,  they provided important context and insights into EOIR’s efforts to assist  immigration judges in adjudicating asylum applications, perspectives on  training and guidance provided to immigration judges and EOIR efforts to  facilitate access to legal resources.
subsections: section_title: Extent of Variation in Completed Asylum Applications
paragraphs: To describe the extent of variation in the outcomes of completed asylum  applications, we analyzed data from EOIR about completions of asylum  applications from fiscal year 1995 through fiscal year 2014, the most  current data available at the time of our analysis. We obtained EOIR  records of all immigration court removal proceedings that occurred during  the period covered by our analysis. We selected those records where the  immigration judge made the decision on the asylum application and  eliminated decisions that were made after appeals. We selected only  records for “lead” applicants, using the applicant’s alien number—a  unique registration number that the Department of Homeland Security  assigns to foreign nationals in immigration court proceedings—as the unit  of analysis. We eliminated duplicate decisions for a spouse and/or  dependent children because they derive from the decision on the lead  applicant. As in our 2008 report, we selected the immigration courts and  countries of nationality that contributed a minimum of 800 affirmative and  800 defensive asylum decisions on asylum applications from fiscal year  1995 through fiscal year 2014; our analysis in this report included  applicants from 41 countries and 28 immigration courts. We selected the  800 minimum in each category to help ensure a sufficient number of  completed asylum applications for our analysis. The results of our  analysis cannot be generalized to asylum seekers from other countries or  to other immigration courts. We used EOIR’s case management system  to identify immigration court proceedings where an immigration judge had  made a decision on an applicant’s asylum application. We assessed the  reliability of the data used in our analyses through electronic testing,  analyzing related database documentation, and working with agency  officials to reconcile discrepancies between the data and documentation  that we received. We determined that the EOIR data on completions of  asylum applications were sufficiently reliable for the purposes of  describing the extent of variation in the outcomes of completed asylum  applications. We reported separately on affirmative and defensive  completed asylum applications to control for characteristics shared by  cases in each of those groups that could affect outcomes, such as  whether the asylum application had already been reviewed by a U.S.  Citizenship and Immigration Services (USCIS) asylum officer.
We also obtained biographical information from EOIR on those  immigration judges who had served during the time period of our analysis.  We merged these biographical data with the EOIR immigration court  proceedings data. In doing so, we produced a dataset for the analysis  that combined proceedings records with information on the characteristics  of the applicants, the immigration judges, the immigration courts, and the  completion or decision on the applicants’ asylum applications. We  selected immigration judges with completions on at least 50 affirmative  and 50 defensive asylum applications from fiscal year 1995 through fiscal  year 2014. We selected the 50 minimum in each category to help ensure  having judges who completed a sufficient number of asylum applications  for our analysis. In analyzing differences in asylum decision across  immigration judges, we excluded those immigration judges who heard  fewer than 50 affirmative cases in our analyses of affirmative asylum  decisions and fewer than 50 defensive cases in our analyses of defensive  asylum decision. We also excluded cases heard by immigration judges  other than in their primary court in order to simplify the presentation and  avoid reaching inappropriate conclusions that can occur when  calculations are based on small numbers of cases. We assessed the  reliability of the data used in our analyses through electronic testing and  working with agency officials to reconcile discrepancies between the data  and documentation that we received. We determined that the EOIR  immigration judge biographical data were sufficiently reliable for the  purposes of producing a dataset that included characteristics of  immigration judges in order to describe the extent of variation in the  outcomes of completed asylum applications.
In 2008, we reported the asylum “grant rate” as the number of asylum  applications granted divided by the total number of all granted and denied  applications. In other words, the grant rate depended on the number of  completed asylum applications that ended in a decision by the  immigration judge of either grant or denial. In this report, to provide data  on all of the outcomes of completed asylum applications during our period  of analysis, we included the five possible outcomes in calculating the  asylum grant rate. These include applications that were granted and  denied, as well as those that had an outcome of withdrawn, abandoned,  or other. Thus, for the purposes of this report, we define the asylum grant  rate as the number of granted asylum applications divided by the total  number of completed applications (completions include applications that  are granted, denied, withdrawn, abandoned, or end in an outcome of  “other”) during the period of analysis. An expanded definition allows us to  describe the outcomes of completed asylum applications, including  outcomes other than granted or denied, and trends in those outcomes.  Although we analyzed data from fiscal year 1995 through fiscal year  2014, our reporting is focused on the time period from May 2007 through  fiscal year 2014 because this is the most recent period and includes  asylum completions since we last reported on asylum outcomes in 2008.
section_title: Factors Associated with Variability in Completed Asylum Applications
paragraphs: To discuss the factors associated with variability in the outcomes  completed asylum applications, we used multivariate statistical modeling  that held constant various other factors that could be associated with  asylum grant rates. Our analysis used multivariate statistical methods to  attribute the unique contribution of each factor to variation in asylum grant  rates. Our analysis updated our previous work on this issue, which took a  similar but not identical approach. Previously, we used multiple logistic  regression to estimate the associations between various case, judge, and  court factors and the probability that an applicant would receive asylum.  Our current analysis generally takes the same approach to update our  prior findings with more recent data on asylum applications completed  from May 2007 through fiscal year 2014. However, we revised aspects of  our data collection, preparation, and analysis to adjust for changes to the  available administrative data and to allow for more precise reporting on  the results. For a more detailed discussion of our multivariate analyses,  see Appendix III.
As in our 2008 report, data limitations prevented us from fully isolating  variation due to the unique judge or court. EOIR’s case management  system collects information to meet EOIR’s administrative needs, such as  the contact information of the applicant, the identity of the judge, and the  languages spoken by the applicant. We used data from this system for  statistical analysis because they are the primary data available on the  population of interest. However, EOIR’s system does not collect data on  all the details of individual proceedings that would be relevant to fully  isolate variation across judges and courts. In particular, we cannot hold  constant all relevant facts and circumstances of each case because  EOIR’s case management system does not collect that information. Such  facts and circumstances could be legally relevant and affect an  applicant’s chance of receiving asylum. As a result, our estimates of the  unique variation due to judges and courts may reflect circumstances of  the cases that we cannot measure. Nonetheless, the data available  allowed us to control for certain factors of each asylum application,  enabling us to compare outcomes across immigration courts and judges.
The data available allowed us to hold constant certain characteristics of  each case, such as applicant language, applicant country of nationality,  judge experience, and court district. Accounting for these characteristics  improves upon a simple comparison of asylum grant rates across judges  and courts, without any adjustment. Table 7 below lists the factors used in  our analysis and the source of the data. These characteristics may be  correlated with the underlying facts and circumstances of each case, and  therefore may indirectly explain factors we cannot measure. Data  limitations prevented us from controlling for factors other than those listed  in table 7 that could have contributed to variability in asylum application  outcomes.
section_title: EOIR Actions to Facilitate Access to Legal Resources
paragraphs: To examine the extent to which EOIR has taken action to facilitate access  to legal resources, we reviewed the Legal Orientation Program (LOP) and  the Legal Orientation Program for Custodians of Unaccompanied Alien  Children (LOPC) program documents and data, including the 2011  blanket purchase agreement and statements of work covering fiscal years  2013 through 2016; quarterly reports, annual reports; site visit reports,  and evaluation reports; and, data reported by the Vera Institute of Justice  (Vera) on the number of unique individuals served by LOP (fiscal years  2008 through 2015) and LOPC (fiscal years 2011 through 2015). We  analyzed these time periods because our 2008 report reviewed EOIR  data through 2007 (for LOP) and LOPC was established in 2010. We also  reviewed documentation on EOIR’s other legal access services, including  self-help materials and programs intended to increase pro bono  representation and serve vulnerable populations, the BIA Pro Bono  Project, the Model Hearing Program, National Qualified Representative  Program and the justice AmeriCorps Legal Service for Unaccompanied  Children program. To obtain additional context on the LOP and LOPC, we  reviewed DOJ’s Fiscal Year 2016 Congressional Budget Submission and  testimony statements about LOPC provided by EOIR’s director. We also  interviewed headquarters officials responsible for overseeing the LOP  and LOPC, as well as officials from Vera, who administer LOP and LOPC  through a contract with EOIR. We analyzed the documentation on LOP  and LOPC and information provided by EOIR officials on these programs  in light of principles outlined in the Government Performance and Results  Act (GPRA) of 1993, as updated by the GPRA Modernization Act  (GPRAMA) of 2010, to assess EOIR’s efforts to measure LOP and  LOPC progress and results against performance measures. We focused  our review on the LOP and LOPC because these are among EOIR’s  longest-standing legal access programs, they have received more funding  than EOIR’s Office of Legal Access Programs’ (OLAP) other programs,  and they have served tens of thousands of individuals since 2003 at  immigration courts throughout the United States.
We conducted this performance audit from January 2015 to October 2016  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
section_title: Appendix III: Multivariate Statistical Analysis of Outcomes of Completed Asylum Applications, Fiscal Years 1995 through 2014
paragraphs: To analyze how asylum grant rates varied across immigration judges and  courts, we used multivariate statistical modeling that held constant  various factors that could be associated with asylum grant rates. Below,  we describe the goals of this analysis and the design and modeling  methods we used. We describe the scope of our analysis and the data we  analyzed in appendix II.
subsections: section_title: Goals of the Analysis
paragraphs: The primary goal of our analysis was to isolate variation in asylum grant  rates due to the unique immigration judge or court assigned to each case.  Variation in asylum grant rates could arise from the facts and  circumstances of each case, and researchers have found that variation is  not unexpected given the institutional conditions under which judges  operate, including the particular requirements of the law, the standard  under which a judge may request corroborating evidence, and the  difﬁculty of assessing credibility. We sought to hold constant relevant  variables at the case, judge, and court levels, to estimate the amount of  residual variation that remained across immigration judges and courts.
Variation in asylum grant rates across immigration judges and courts  could reflect various other factors that are statistically correlated with  them. For example, if immigration courts tended to receive applications  from asylum-seekers of the same country of nationality, asylum grant  rates could potentially vary systematically based on nationality rather than  on the unique nature of the judge or court. Similarly, the quality and  accessibility of legal representation across the United States could  potentially affect the outcomes of asylum applications. Since immigration  judges’ decisions on asylum applications may incorporate many factors,  such as credibility and the facts underlying the claims, simple univariate  comparisons of asylum grant rates across judges and courts cannot  distinguish among these various potential sources of variation.
section_title: Design and Modeling Methods
paragraphs: The data available from the Executive Office for Immigration Review’s  (EOIR) case management system have a multilevel structure. Multiple  applications for asylum are clustered within the same judges, courts,  languages, and nationalities. These groups can be clustered among  themselves, with or without nesting. Our data processing, discussed  above, assigned one court to each judge, so that each judge was nested  within the court in which they adjudicated the most cases. The remaining  groups—courts, languages, and countries of nationality—were not  necessarily nested. Each language could have been associated with  multiple nationalities and vice versa. Each court could have been  associated with multiple languages and countries of nationality;  however,  in practice, courts could have heard cases from a predominate  combination of languages and nationalities, such as Chinese nationals  who speak Mandarin and apply for asylum in New York.
The scope of our analysis included all completed asylum applications that  met the screening criteria discussed in appendix II  and that ended in  outcomes of grant, deny, withdrawal, abandonment, or “other.” However,  we dichotomized the outcome of interest to reflect whether the applicant  was granted asylum or the application ended with any of the other  outcomes. Expanding the scope of our sample to include all completed  asylum applications rather than only applications that were granted or  denied allowed us to analyze the outcomes of all completed asylum  applications. Dichotomizing the outcome produced asylum grant rates as  a proportion of applications originally filed that meet our screening criteria,  which prevented bias from potentially  changing withdrawal practices over  time.
The data and outcome of interest suggested that a hierarchical or mixed  logistic regression model would adequately reflect the data generation  process. Accordingly, we developed a mixed model that represented the  grouping variables—judge, court, country of nationality, and language,  respectively—with random intercepts. Random effects allowed asylum  grant rates to be correlated within and across groups, which  parameterized our hypothesis that rates may systematically vary across  groups, all else being equal on the factors we analyzed. Random effects  made the model parsimonious because we could use the group variance  parameters to concisely describe variation across groups with many  levels. In contrast, modeling group variation with fixed effects would have  required estimating several hundred explicit parameters, one for each  group level, which would have consumed degrees of freedom and  complicated model interpretation.
Substantively, random effects accurately represented the random  assignment of judges to cases, which EOIR generally uses as an  administrative policy. Random assignment of countries of nationality,  languages, and courts has less of a direct substantive interpretation, but  would be consistent with an underlying data generation process where  the sample of asylum cases heard in any particular time period would be  drawn from an underlying population distribution of immigration courts  and applicant nationalities and languages.
We held constant case, judge, and court characteristics using covariates  with fixed parameters. The smaller number of parameters associated with  these covariates made a fixed effects approach easier to apply and  interpret. We assumed that the covariate effects did not vary across  groups, so that only the model’s intercept varies randomly. We had no  prior expectation that specific covariate effects should have varied across  groups. Moreover, increasing the number of random effects would have  increased the complexity of the model and could have made it hard to  estimate computationally.
We modeled the data separately by asylum type (affirmative and  defensive) and time period. This allowed all model parameters to vary  between asylum types and over time. The alternative approach, pooling  the data, potentially would have obscured important differences in the  affirmative and defensive asylum processes and important changes in the  processes over the 20 years that our data spanned. Specifically, we  modeled Yi, a dichotomous variable indicating whether applicant i in the  affirmative or defensive processes was granted asylum. Each applicant  was associated with a judge j, court c, nation n, and language l.  Consistent with a hierarchical logistic model, we assumed that  x denoted a vector of covariates with fixed coefficients β. α (.) denoted  the group random effects, each of which was normally distributed and  centered on zero. α was the population average intercept, with intercepts  for each group given by α + α (.). The variances of the random effects  around the population average were denoted by σ (.).
We used most of the covariates from our 2008 report on this issue.  However, we excluded several covariates that were unassociated with  outcome in that report and that were labor-intensive to measure. For  example, measuring a judge’s prior work experience for our prior report  involved biographical research on each judge. Specifically, the covariate  vector x included the following: whether the applicant had one or more dependents (indicator); whether the applicant sought asylum within one year of entry into the United States (indicator); whether an applicant was represented by counsel recognized to practice in immigration court (indicator); whether the applicant was ever detained (indicator for being currently detained or having been detained and released, compared to never having been detained; defensive cases only);
Presidential administration under which judges were appointed (vector of indicators, with the subsample mode set as the omitted level); judge years of experience (linear); judge gender (indicator); judge asylum caseload since October 1, 1994, the earliest date in our data, and in the 90 days prior to the completion of the asylum application, respectively (linear);
Circuit court (vector of indicators, with the subsample mode set as the omitted level); and
Court asylum caseload since October 1, 1994, the earliest date in our data, and in the 90 days prior to the decision, respectively (linear).
For each subsample defined by asylum type (affirmative or defensive)  and period, we rescaled the continuous covariates such that the vector of  sample means equaled 0 and the vector of variances equaled 1. We  rescaled each categorical covariate, including the groups used to define  random effects, such that the first level identified the mode and served as  the omitted level in a vector of indicator variables in xi for each variable.  After rescaling, α estimates the overall asylum grant rate for an applicant  having modal or mean values on x and α (.) . σ(.)
GAO,  U.S. Asylum System: Significant Variation Existed in Asylum Outcomes Across  Immigration Courts and Judges, GAO-08-940 (Washington, D.C: Sept. 25, 2008). of asylum grant rates across judges, courts, nations, and languages  around the population average for an applicant at the mode or mean of xi.
The center of the data at α does not necessarily correspond to a realistic  application for asylum. For example, all judges do not practice at the  modal court and all languages may not be spoken in the modal nation.  Nevertheless, rescaling facilitates estimation and interpretation of the  model. All inference can be done on α and α (.) directly, with no  transformation required. This allowed us to easily describe variation in  asylum grant rates for a hypothetical representative case in the  distribution of each covariate and group.
The key parameters of interest are the variances of the group random  , and implicitly, the fitted group-level distribution of asylum  effects, σ(.) grant rates for a modal or mean applicant:  To describe variation across groups on the probability scale, we  estimated the quantiles of this group distribution on the logit scale and  then transformed them using the inverse logistic function.
We estimated the model using maximum approximated likelihood  methods, as implemented in the lme4 package in R. We estimated the  confidence intervals of fixed parameters using Wald approximations and  the fitted covariance matrix. We estimated the confidence intervals of the  group random effect variances using profiled likelihood methods, as  implemented in lme4. We transformed the estimated confidence intervals  from the logit scale to the probability scale using the inverse logistic  calculation described above.
section_title: Results
paragraphs: Tables 8 through 14 report estimates of the fixed parameters on the  exponentiated logit scale and estimates of the random effect variances on  the logit scale, separately for affirmative and defensive applications. The  exponentiated scale allows us to interpret the results as odds-ratios or  semi-elasticities for categorical or continuous variables, respectively. We  omit estimates for the fixed effect parameters on the logit scale to prevent  the calculation of exact grant probabilities. Please see the body of our  report for further interpretation of the results.
section_title: Appendix IV: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Acknowledgments
paragraphs: In addition to the contact named above, Kathryn Bernet, Assistant  Director; Joel Aldape, David Alexander, Emily Christoff, Dominick Dale,  Jack Granberg, Grant Mallie,  Taylor Matheson, Jan Montgomery, Jon  Najmi, Mary Pitts, and Jeff Tessin made significant contributions to this  report.