Aviation Safety: Oversight of Foreign Code-Share Safety Program Should Be Strengthened
Published: 2005-08-05T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: U.S. airlines are increasingly relying on code-share partnerships with foreign carriers to provide additional sources of revenue. Code-sharing is a marketing arrangement in which an airline places its designator code on a flight operated by another airline and sells and issues tickets for that flight. To determine whether the foreign code-share partners of U.S. airlines meet an acceptable level of safety, in 2000, the Department of Transportation (DOT) established the Code-Share Safety Program, which requires U.S. airlines to conduct safety audits of their foreign code-share partners as a condition of code-share authorization. GAO's objective was to assess the federal government's efforts to provide reasonable assurance of safety and security on foreign code-share flights. GAO reviewed (1) the extent to which DOT's code-share authorization process is designed to consider safety and security, (2) the Federal Aviation Administration's (FAA) management of the Code-Share Safety Program, and (3) the implementation of the program by airlines and the results.
section_title: What GAO Found
paragraphs: In considering U.S. airlines' requests to establish code-share arrangements with foreign carriers, DOT's Office of International Aviation reviews, among other things, any safety and security objections from FAA and TSA. FAA assesses the safety of foreign civil aviation authorities and reviews reports of the safety audits that U.S. carriers have conducted of their foreign airline partners. From fiscal years 2000 through 2004, DOT (1) authorized U.S. airlines to establish or maintain code-share arrangements with foreign carriers 270 times and (2) did not suspend any arrangements because of known safety concerns. According to FAA, however, U.S. airlines occasionally have decided not to pursue code-share arrangements with foreign carriers because they expected FAA would object, and FAA sometimes puts its reviews of proposed code-share arrangements on hold if the agency has safety concerns. FAA and TSA did not object to any of the authorizations during that period for safety or security reasons. Although not involved in the code-share authorization process, the Department of Defense (DOD) reviews the safety of foreign airlines that transport DOD personnel. For their separate programs, FAA and DOD are reviewing many of the same safety audit reports on foreign carriers. The Code-Share Safety Program, which calls for U.S. airlines to conduct periodic safety audits of their foreign code-share partners, incorporates selected government auditing standards involving independence, professional judgment, and competence. However, FAA's reviews of the safety audit reports lacked management controls for reviewers' qualifications, documenting the closure of safety audit findings, verifying corrective actions taken in response to findings, and documenting reviews. Eight U.S. airlines with foreign code-share partners have implemented the DOT program by conducting safety audits of their foreign partners. According to our review of a random sample of audit reports that FAA reviewed from fiscal years 2000 through 2004, the largest numbers of safety findings identified were in the categories of (1) flight operations and (2) maintenance and engineering. GAO estimates that for 68 percent of the findings, the documentation was insufficient to demonstrate that the findings were closed or were resolved. Airlines are beginning to adopt a new safety audit program that requires the documentation of findings and corrective actions.

section_title: Letter
paragraphs: As the U.S. airline industry strives to improve its financial condition, many  carriers are increasing their focus on international service to provide  needed sources of revenue. To help expand their global market reach, U.S.  airlines have established an increasing number of code-share arrangements  with foreign airlines, and, as of May 2005, eight U.S. airlines had established  foreign code-share partnerships. Code-sharing is a marketing arrangement  in which an airline places its designator code on a flight operated by  another airline and sells and issues tickets for that flight. U.S. carriers must  obtain authorization for foreign code-share operations from the  Department of Transportation (DOT).
In 2000, DOT’s Office of the Secretary and Federal Aviation Administration  (FAA) established the Code-Share Safety Program to ensure that the  foreign code-share partners of U.S. airlines meet an acceptable level of  safety. The program was established in response to safety concerns that  arose after SwissAir Flight 111 crashed off the shores of Nova Scotia in  1998, killing 229 passengers and crewmembers, including 53 Americans. At  that time, SwissAir was a foreign partner of U.S.-based Delta Air Lines.  Before the Code-Share Safety Program was established, DOT reviewed the  potential competitive and economic impact of U.S. airlines’ code-share  arrangements with foreign airlines, but regarding safety considerations,  relied on the oversight by the home country. Now, under the Code-Share  Safety Program, U.S. airlines must conduct periodic safety audits of their  foreign code-share partners as a condition of code-share authorization.  DOT designed the Code-Share Safety Program to rely on U.S. airlines to  audit the safety of their foreign code-share partners because FAA does not  have the authority to inspect foreign aircraft that do not enter the United  States. The DOT program is similar to a safety audit program that the  Department of Defense (DOD) created in 1999 for foreign carriers that  transport DOD personnel.
You asked that we review the measures that the federal government is  taking to provide reasonable assurance of safety and security when  passengers travel on flights operated by foreign code-share partners of U.S.  airlines. To do so, we addressed the following questions:   To what extent is DOT’s authorization of U.S. airlines’ code-share  arrangements with foreign airlines designed to consider safety and  security?
How well has FAA managed the Code-Share Safety Program?
To what extent have U.S. airlines implemented the Code-Share Safety  Program, and what have been the results of their efforts?
To determine how safety and security are considered in DOT’s  authorization of U.S. airlines’ code-share arrangements with foreign  airlines, we reviewed the process that DOT follows in authorizing code- share arrangements, Code-Share Safety Program guidelines, applicable  aviation safety standards, and related legal authorities. Our review covered  safety audit reports on foreign carriers that FAA reviewed and U.S. airlines’  code-share partnerships with foreign carriers that DOT has authorized  since February 2000, when the Code-Share Safety Program was  established, through the end of fiscal year 2004. We also examined how  FAA’s assessments of foreign civil aviation authorities are used in the code- share approval process. During our review of the Code-Share Safety  Program, we found that DOD had a similar program designed to ensure the  safety of foreign airlines that transport DOD personnel. Because our  objective was to determine how the federal government is providing safety  oversight of foreign airlines, we included DOD in our review. However,  because DOD does not play a direct role in the authorization of foreign  code-share arrangements, this report focuses on the DOT Code-Share  Safety Program. Moreover, because security is a component of assessing  airline safety, we determined what the Transportation Security  Administration (TSA), the Department of Homeland Security agency  responsible for aviation security, has done to provide security information  to FAA and DOD for their safety reviews of foreign airlines.
Because the Code-Share Safety Program establishes an audit program, to  assess how well FAA has managed the program, we reviewed whether the  program’s design conformed with selected standards identified in  Government Auditing Standards. In addition, because we were  evaluating the management of a government program, we examined how  FAA applied certain management controls in its reviews of the safety audit  reports for the Code-Share Safety Program using Standards for Internal  Control in the Federal Government. We discussed with FAA officials how  they conducted the assessments of foreign civil aviation authorities and  with TSA officials how they assessed the security of certain foreign carriers  and airports.
Finally, to determine the extent to which U.S. airlines have implemented  the Code-Share Safety Program and the results, we interviewed safety  officials at the eight U.S. airlines that were participating in the Code-Share  Safety Program and reviewed a random sample of reports of safety audits  the airlines had conducted of their foreign code-share partners. We did not  determine whether the airlines complied with international aviation safety  standards but reviewed the safety audit reports to determine what types of  findings were identified and whether corrective actions were documented.  We also interviewed officials from the International Air Transport  Association (IATA), the trade association that represents air carriers  worldwide, to determine how IATA’s recent introduction of a new safety  audit program for its members may affect the Code-Share Safety Program.  We conducted our review in Arlington, Virginia; Atlanta; Denver; Eagan,  Minnesota; Ft. Worth; Houston; Phoenix; Pittsburgh; Seattle; Washington,  D.C.; and Montreal, Canada; from August 2004 through August 2005 in  accordance with generally accepted government auditing standards.  Appendix I provides additional information on our methodology.
subsections: section_title: Background
paragraphs: Each year, several million passengers travel on foreign airlines that have  established code-share arrangements with U.S. air carriers. Code-sharing  is a marketing arrangement in which an airline places its designator code  on a flight operated by another airline and sells and issues tickets for that  flight. On foreign code-share routes, U.S. airlines and their foreign partners  each place their respective designator code on flights operated by the other  airline. Passengers can purchase one ticket from a U.S. airline that can  include flight segments covered by one or more foreign partner airlines.  Air carriers throughout the world form code-share alliances to strengthen  or expand their market presence or ability to compete. Through code- sharing, U.S. airlines can offer seamless service to additional international  destinations without incurring the expense of establishing their own  operations to those locations. Moreover, airline officials said that code- share arrangements with foreign airlines have become important sources  of revenue. According to FAA, international markets are viewed as more  attractive growth markets by mainline carriers because of more limited  competition from low-cost carriers and greater profitability.
In recent years, U.S. airlines have established an increasing number of  code-share arrangements with foreign carriers to expand their service  markets. As of May 2005, eight U.S. airlines had established 108  arrangements to place their designator codes on 85 different foreign  carriers, up from six U.S. airlines that had established 39 arrangements to  place their designator codes on 38 different foreign carriers in fiscal year  2000. As shown in figure 1, the majority of U.S. airlines’ code-share  arrangements are with European airlines, representing over half, followed  by airlines from Asia and the Pacific, accounting for nearly a quarter of the  arrangements. Appendix II lists the U.S. carriers and their foreign code- share partners.
In 1998, SwissAir Flight 111, which was a code-share flight with U.S.-based  Delta Air Lines, crashed off the shores of Nova Scotia, killing 229  passengers, including 53 Americans. Following that accident, the DOT  Inspector General reviewed aviation safety under international code-share  agreements and issued a report in 1999 recommending, among other things,  that DOT develop and implement procedures requiring U.S. airlines to  conduct safety audits of foreign carriers as a condition of authorization of  code-share passenger services. Also in 1999, legislation was introduced in  the House of Representatives that would have statutorily required U.S.  airlines to audit the safety of their foreign code-share partners. Although  that legislation was not enacted, in 2000, DOT’s Office of the Secretary and  FAA established the Code-Share Safety Program, which included the  development of guidelines for U.S. carriers to follow in auditing the safety  of their foreign code-share partners as a condition of DOT’s authorization  of code-share passenger services.
DOD’s safety audit program, called the Commercial Air Transportation  Quality and Safety Review Program, expanded another program that DOD  established in 1986 to check the safety of charter aircraft transporting its  personnel, after an Arrow Air charter airplane transporting U.S. military  personnel crashed in 1985, killing 256 passengers and crew. In 1986,  Congress passed Public Law 99-661, which created a Commercial Airlift  Review Board and prohibits DOD from contracting with an air carrier  unless it meets certain safety standards and submits to a technical safety  evaluation. A 1999 memorandum of understanding between DOD and the  Air Transport Association, a U.S. airline industry association, allows DOD  to review the safety audits that U.S. airlines have conducted of their foreign  airline partners.
DOD is a major customer of airlines that have established code-share  arrangements through its participation in the General Services  Administration’s (GSA) city-pairs program, under which the government  negotiates service contracts for all federal government employees,  including military personnel, to save the government money on air travel.  The program requires federal employees and military personnel to fly with  carriers under such contracts when they travel on government business.  DOD is required to review the safety of all airlines that provide scheduled  service to its personnel under the GSA city-pairs program, which include  U.S. airlines’ foreign code-share partners. DOD’s program also has the  effect of having the airlines comply with DOD requirements if they want to  maintain the GSA contracts.
The safety of foreign carriers is also a concern because aviation accident  rates vary considerably from one region of the world to another. According  to data compiled by IATA, an international airline association, during 2004,  the North American region had the lowest aviation accident rate (0.29 hull  losses per million flight segments), while the Middle East had the highest  (5.32 hull losses per million flight segments). Africa had the second  highest rate, followed by South America, the Asia-Pacific region, and  Europe. These accident rates are shown in figure 2.
section_title: DOT’s Office of International Aviation Relies on FAA for Safety Assessments and TSA for Security Assessments to Authorize Code-Share Arrangements
paragraphs: DOT’s Office of International Aviation within the Office of the Secretary of  Transportation authorizes U.S. airlines’ code-share arrangements with  foreign airlines after considering, among other things, safety and security  information from FAA and TSA. FAA provides DOT’s Office of International  Aviation with a memorandum recording its “objection” or “no objection” to  the foreign code-share partners of U.S. airlines. This memorandum is based  on FAA’s assessments of foreign civil aviation authorities and reviews of  safety audits conducted by U.S. airlines of foreign carriers. TSA assesses  the security of foreign airlines that provide service to the United States and  its territories and certain foreign airports. DOT also considers the  competitive and antitrust implications of code-share arrangements. For its  program, DOD reviews many of the same safety audit reports on foreign  airlines that FAA reviews for the Code-Share Safety Program.
subsections: section_title: DOT Considers Information from Several Sources to Authorize Code-Share Arrangements
paragraphs: To authorize a code-share arrangement between a U.S. and a foreign  airline, DOT must find that the arrangement is in the public interest.  Under the DOT guidelines, this public interest finding includes a  determination of the foreign carrier’s level of safety and the economic  impact of the arrangement. Before authorizing a code-share arrangement,  DOT’s Office of International Aviation obtains (1) a memorandum of “no  objection” from FAA, based on its review of the safety audits and other  safety information available to FAA; (2) a clearance from DOT’s Office of  Policy on aspects of security involving the foreign carrier, including  information from TSA; and (3) a clearance from DOT’s Office of Aviation  Analysis and Office of the General Counsel concerning the code-share  arrangement’s competitive impact on the airline industry. The Office of  International Aviation also obtains advice from the Department of Justice  on potential antitrust issues. According to DOT officials, on 270  occasions, from February 2000 through the end of fiscal year 2004, DOT  authorized or reauthorized U.S. airlines to establish or maintain code-share  arrangements with foreign carriers and did not suspend any arrangements  during that time. However, FAA officials also said that U.S. airlines have  occasionally decided not to pursue code-share arrangements with foreign  airlines because they expected FAA would object. Code-share  arrangements may be periodically reauthorized based on the terms of the  initial authorization. Figure 3 shows the DOT code-share authorization  process.
section_title: FAA Response Reflects Assessments of Foreign Civil Aviation Authorities and Safety Audit Reviews
paragraphs: DOT’s Office of International Aviation solicits the views of FAA on the  safety aspect of its code-share authorization decision because of FAA’s  technical expertise in that area. FAA reviews reports of the safety audits  that U.S. carriers have conducted on the foreign carriers and other safety  information available to FAA, including its assessments of the capabilities  of the relevant foreign civil aviation authorities. FAA provided DOT’s Office  of International Aviation with memorandums of “no objection” on all  foreign airlines being considered for code-share authorization during fiscal  years 2000 through 2004. According to FAA officials, if FAA has safety  concerns, it puts a hold on its review of the proposed code-share  arrangement, allowing time for the safety issues to be resolved; and on four  occasions, from February 2000 through September 2004, U.S. airlines  suspended their code-share arrangements with foreign carriers because  FAA was questioning the capabilities of the civil aviation authorities under  which the foreign carriers were operating. Figure 4 shows FAA’s process  for providing information to DOT’s Office of International Aviation on U.S.  airlines’ applications to establish code-share arrangements with foreign  carriers.
Under the Code-Share Safety Program guidelines, DOT authorizes a U.S.  airline’s code-share arrangement with a foreign carrier only if the foreign  airline is from a country that is compliant with applicable international  aviation safety standards under FAA’s International Aviation Safety  Assessment (IASA) program. Under IASA, FAA reviews the capabilities of  foreign civil aviation authorities by checking their compliance with  standards established by the International Civil Aviation Organization  (ICAO), a United Nations aviation organization. Under IASA, FAA assigns  countries’ civil aviation authorities either a category 1 rating—meets ICAO  standards—or a category 2 rating—does not meet ICAO standards. During  the IASA process, FAA personnel, typically from various international field  offices, conduct on-site assessments of civil aviation authorities for  compliance with ICAO standards in eight areas: (1) primary aviation  legislation, (2) aviation regulations, (3) organization of the civil aviation  authority, (4) adequacy of the technical personnel, (5) technical guidance,  (6) licensing and certification, (7) records of continuing inspection and  surveillance, and (8) resolution of safety issues. Each country with carriers  serving, or wishing to serve, the United States in their own right or as part  of a code-share arrangement with a U.S. airline must first have an  assessment under the IASA program.
Although FAA’s plan is to reassess the category for each foreign civil  aviation authority every 2 years, FAA officials said that this activity  occurred less frequently because of a larger-than-anticipated number of  reassessments and constraints on the agency’s resources. FAA data  indicate that 67 of the 100 foreign civil aviation authorities in the IASA  program, or about two-thirds, have not been assessed within the last 4  years. According to FAA, some countries were not assessed within the last  4 years because available data indicated that their rating categorization  remained valid. FAA data also show that from January 1, 2000, through May  1, 2005, FAA assessed or reassessed—because of safety oversight  concerns—the capabilities of 33 foreign civil aviation authorities, 6 of  which were assessed more than once. Of the 42 countries’ civil aviation  authorities under which the foreign code-share partners of U.S. airlines are  operating, 16 have required an IASA assessment or reassessment since  2000 and 26 have not.
IASA results, along with the safety audits that U.S. airlines conduct of their  foreign code-share partners, are FAA’s principle measures of the level of  safety of the foreign carriers. According to the guidelines, the level of  oversight and regulation that an airline receives from its regulatory  authority is an important factor in assessing its safety. For this reason, DOT  authorizes U.S. airlines’ code-share arrangements only with foreign airlines  that are from IASA category 1 countries. As of May 2005, FAA had assigned  IASA category 1 ratings to 71 countries’ civil aviation authorities and IASA  category 2 ratings to 28; 94 other countries had not yet been categorized,  generally because no carriers from those countries had applied to provide  direct service to the United States.
DOT’s Office of International Aviation will not authorize a code-share  application, and FAA will not review the safety audit report if flights that  are intended to carry a U.S. carrier’s designator code would be operated by  a foreign carrier from a country with an IASA category 2 rating. If a U.S.  airline is seeking to establish a code-share arrangement with a foreign  carrier that is from a country that does not have an IASA rating, FAA  normally conducts the assessment before DOT’s Office of International  Aviation considers the application. When FAA lowers a country’s IASA  rating from category 1 to category 2, DOT’s Office of International Aviation  contacts any U.S. airline that has a code-share partnership with an airline  from that country to advise the U.S. airline of the lowered IASA rating so  that the U.S. carrier can promptly remove its code from any passenger  flights operated by that airline, according to agency officials. While DOT  indicated that it could, at its option, order the removal of U.S. airlines’  designator codes under these circumstances, in practice, DOT has not  needed to pursue that option because, when the airlines have learned about  an IASA category change affecting their service, they have removed their  operating codes from the foreign carrier. On four occasions since 2000, U.S.  airlines have suspended their code-share arrangements with foreign  airlines because FAA was questioning the capabilities of the civil aviation  authorities under which the foreign airlines were operating. The program  guidelines allow DOT to consider, on a case-by-case basis, continuing to  authorize a U.S. airline’s code-share arrangement with a foreign carrier that  is from a country with an IASA rating that has been lowered from category  1 to category 2. According to FAA, this case-by-case language was included  to enable DOT’s Office of International Aviation to accord U.S. airlines a  limited degree of flexibility needed to effectuate an orderly shutdown of  their code-share services. However, DOT officials told us that they will not  authorize the continuation of a code-share arrangement beyond the needs  of such an orderly shutdown.
section_title: FAA Reviews U.S. Airlines’ Methodologies for Auditing Their Foreign Code-Share Partners
paragraphs: FAA will not review a U.S. airline’s safety audit report on a foreign carrier  until FAA has reviewed and accepted the airline’s audit methodology.  According to the program guidelines, the U.S. airlines’ safety audit  methodologies should incorporate ICAO standards on personnel licensing,  aircraft operations, aircraft airworthiness, and security. The guidelines  also describe how the U.S. airlines should conduct their safety audits,  including what qualifications the auditors should possess, how the system  for reporting and correcting findings should be devised, what audit results  are satisfactory, how a safety monitoring system should be established, and  how frequently audits should be conducted. At the same time, FAA officials  said they provide the airlines with some flexibility in designing their audit  programs, as long as the programs address all of the relevant ICAO  standards. FAA reviewed and accepted an audit program for each of the  eight U.S. airlines to participate in the Code-Share Safety Program. In  designing their audit methodologies, some U.S. airlines include other  standards and best practices, such as ones developed by DOD, in addition  to the ICAO standards and recommended practices in the DOT program  guidelines. Moreover, to audit the safety of their foreign code-share  partners, six U.S. airlines have begun using standards from a new  international safety audit program developed by IATA called the IATA  Operational Safety Audit (IOSA), which incorporates the ICAO standards,  plus many additional industry best practices. IOSA was developed by IATA  to improve global airline safety and promote audit efficiency by reducing  redundant audits. In 2004, FAA accepted the IOSA program as a  methodology that would meet the Code-Share Safety Program guidelines.
Under the Code-Share Safety Program guidelines, after the U.S. airlines  have completed the audits and the foreign airlines have taken all corrective  actions, the U.S. airlines’ safety directors (or similar officials) should  provide written statements to FAA, known as compliance statements,  affirming that the audits were conducted in accordance with the guidelines  and that the foreign carriers meet the applicable ICAO standards.  According to an FAA official, U.S. airlines filed compliance statements for  all of the audit reports that FAA reviewed on foreign carriers. The  guidelines also indicate that to maintain their continued code-share  authorizations, U.S. airlines should audit the safety of their foreign code- share partners and submit compliance statements to FAA every 2 years. We  found that, for 12 out of 256 audit reports that FAA reviewed from February  2000 through the end of fiscal year 2004, FAA granted the U.S. airlines  extensions of time to submit compliance statements because delays had  resulted from the outbreak of Severe Acute Respiratory Syndrome (SARS),  the U.S. airline planned to cancel the code-share arrangement, or the  foreign carrier needed more time to implement corrective actions. FAA  generally granted the extensions for between 1 and 3 months, during which  time the code-share arrangements continued.
section_title: TSA Clearances Reflect Security Assessments of Foreign Airlines That Provide Direct Service to the United States, Its Territories, and Certain Foreign Airports
paragraphs: Since 2000, DOT’s Office of Intelligence and Security and Office of Policy  have provided security clearances to DOT’s Office of International Aviation  for all U.S. airlines’ proposed code-share arrangements with foreign  airlines. DOT’s Office of Policy receives security information on certain  foreign carriers and foreign airports from TSA, which assesses the  security of foreign airlines that provide direct service to the United States  and its territories, as well as to certain foreign airports. TSA provided  security clearances for all proposed code-share arrangements, from fiscal  years 2000 through 2004, for which it had information on the foreign  carriers. Because it lacks the authority, TSA does not assess the security  of other foreign carriers that do not provide direct service to the United  States and its territories. Twenty-nine, or about one-third, of the 85 foreign  code-share partners of U.S. airlines do not provide service to the United  States and its territories and therefore have not been assessed for security  by TSA.
DOT has also authorized U.S. airlines’ code-share arrangements with  foreign airlines that serve many foreign airports that TSA has not assessed  for security. As a result, passengers traveling on foreign code-share  partners of U.S. airlines may be traveling to certain foreign airports that  could have security risks. TSA has the authority to assess the security of a  foreign airport (1) served by U.S. airlines, (2) from which a foreign carrier  serves the United States and its territories, or (3) that “poses a high risk of  introducing danger to international air travel.” Also, TSA can assess “other  foreign airports the Secretary of Homeland Security considers  appropriate.” TSA has assessed the security of the foreign airports from  which domestic and foreign airlines provide direct service to the United  States and its territories. However, in addition to the foreign airports that  provide direct service to the United States and its territories, the foreign  code-share partners of U.S. airlines serve other foreign airports. TSA  officials indicated they have begun to assess the security of other foreign  airports. DOT has not always had comprehensive data on which foreign  airports are being served by the foreign code-share partners, so we were  unable to determine how many foreign airports have not undergone TSA  security assessments. For one U.S. airline for which we had complete  foreign code-share route information, we determined that the foreign  partners served 128 foreign airports that did not provide direct service to  the United States and its territories, and some of these 128 had yet to  undergo TSA security assessments.
In assessing the security of foreign airports, TSA rates them in categories  and assesses airports in those categories as appropriate. DOT’s Office of  International Aviation, which receives TSA’s security ratings through DOT’s  Office of Policy, authorizes code-share arrangements for U.S. airlines with  foreign carriers that serve foreign airports. According to DOT security  officials, it is not a problem to authorize code-share arrangements with  foreign airlines regardless of category because all airports must meet ICAO  security standards and are assessed appropriately. Moreover, officials  from TSA and DOT noted that both U.S. and foreign airlines can be required  to implement additional security measures at those airports. For example,  the TSA officials described an instance in which a bombing in a Middle  Eastern country resulted in the implementation of additional security  measures at an airport in that country. TSA officials said that because that  airport met ICAO security standards, TSA had to rely on increased security  measures voluntarily implemented by the carriers to help mitigate the  threat in that area.
section_title: DOD Provides Additional Safety Oversight of Foreign Airlines
paragraphs: While not involved in DOT’s code-share authorization process, DOD  reviews the safety of certain foreign airlines, thereby providing an  additional layer of federal oversight. The DOD Commercial Air  Transportation Quality and Safety Review Program is focused on ensuring  that the airlines DOD contracts with—to transport DOD personnel—meet  applicable safety standards. DOD requires U.S. airlines to audit the safety  of their foreign code-share partners every 2 years, on the basis of ICAO  standards, and monitor the safety of their foreign partners between safety  audits. In addition, DOD considers FAA’s IASA ratings of foreign civil  aviation authorities in determining whether to allow foreign carriers to fly  on GSA city-pair routes.
DOD requires that foreign airlines be assessed on the basis of standards  that DOD developed called Quality and Safety Requirements, which are  focused on system safety processes. According to a DOD official, these  DOD standards include safety processes that are not ICAO requirements,  which form the basis of the DOT program. A DOD official said, for  example, that DOD requires airlines to have a safety audit program that  analyzes and assesses trends of safety information, including feedback  from crew members, for the purpose of enhancing safety, which is not an  ICAO standard. Although DOT can suspend code-share authorizations for  safety reasons, DOD can cancel, at any time, contracts with airlines that  transport DOD personnel if it determines that they are not sufficiently safe.  Between audits, DOD takes certain steps to monitor the safety of foreign  carriers that FAA does not take, such as conducting semi-annual  evaluations that include requiring foreign carriers that DOD contracts with  to complete questionnaires about their safety. DOD does not consider TSA’s  security assessments of foreign airports in its review. DOD officials said  that they were unaware of TSA’s foreign airport assessments and would like  TSA to provide the information for DOD to consider as part of its reviews.
We found that DOD and FAA review many of the same safety audit reports  on foreign airlines. During fiscal years 2001 through 2004, DOD and FAA  reviewed 203 of the same reports of safety audits that U.S. airlines had  conducted of their foreign code-share partners. In reviewing these same  reports, DOD and FAA reached the same conclusions about the safety of  the foreign carriers involved. Because DOD and FAA are reviewing many of  the same audit reports, the DOT and DOD safety programs are duplicating  some efforts. In its 1999 report, the DOT Inspector General recommended  that, in establishing a safety program on foreign code-share partners of U.S.  airlines, FAA and DOT’s Office of the Secretary work closely with DOD to  maximize the use of limited resources, avoid duplication, and establish  protocols for exchanging information about the carriers’ safety  assessments. A DOD official said that he communicates frequently with  FAA Code-Share Safety Program officials, and that DOD has a full-time  liaison in FAA’s Flight Standards Service, who meets weekly with FAA  officials. However, FAA officials said that although DOD requests IASA  reports on certain countries, FAA does not routinely communicate with  DOD on its safety audit reviews of foreign carriers, and no set criteria spell  out the circumstances under which FAA and DOD should communicate  information on the safety of U.S. airlines’ foreign code-share partners.
When we discussed the possibility of reducing duplicative safety reviews  with FAA and DOD officials, an FAA official said he did not consider their  reviews to be duplicative because FAA and DOD have different objectives.  The FAA official said that FAA is reviewing the reports from the  perspective of a regulator, focusing on the carriers’ compliance with ICAO  standards. Furthermore, the FAA official questioned whether FAA or DOD  could assume each others’ responsibilities and report to different  departments. A DOD official also said the potential for duplication should  be considered from the perspective of DOD’s and FAA’s different objectives  in conducting their reviews. The DOD official said that DOD’s objective is  to ensure that its requirements for transporting DOD personnel are being  met. Another DOD official said that FAA and DOD are not duplicating their  efforts because neither agency has the expertise to conduct its reviews  from the other agency’s perspective.
section_title: Code-Share Safety Program Incorporates Auditing Standards, but FAA’s Oversight of the Program Lacks Certain Management Controls
paragraphs: The Code-Share Safety Program incorporates selected government auditing  standards involving independence, professional judgment, and  competence. According to FAA officials, FAA and DOT’s Office of the  Secretary worked with the airline industry to recommend that the Code- Share Safety Program guidelines incorporate these standards. Government  auditing standards provide an overall framework for ensuring that auditors  be independent and exercise judgment, competence, and quality control  and assurance in planning, conducting, and reporting on their work.  However, FAA’s management of the program did not incorporate certain  internal controls, which the Office of Management and Budget requires  federal managers to use in assessing the effectiveness and efficiency of  operations. These controls are related to establishing reviewers’  qualifications, documenting the closure of safety audit findings, verifying  corrective actions taken in response to the findings, and documenting  reviews.
subsections: section_title: Code-Share Safety Program Incorporates Selected Government Auditing Standards
paragraphs: The Code-Share Safety Program guidelines recommend that the airlines  incorporate certain government auditing standards in their safety audit  reviews. FAA has reviewed the methodologies that the U.S. airlines follow  in auditing the safety of their foreign code-share partners, which  incorporate these auditing standards. Ensuring independence is critical, for  example, because the U.S. airlines generally audit the safety of their foreign  code-share partners themselves. Although we did not assess the airlines’  compliance with the independence standard, U.S. airline officials told us  that they ensure independence by separating their safety and marketing  departments organizationally to prevent any possible influence from the  marketing staff on the safety audit results. In addition, safety officials at the  U.S. airlines participating in the Code-Share Safety Program indicated that  other airline departments do not have any input into their safety audit  results. Moreover, some airline safety officials said they were not aware of  the specific financial arrangements involved in their airlines’ code-share  partnerships. The program guidelines allow the U.S. airlines to employ  personnel or hire outside experts as consultants (contractors) to conduct  the safety audits. FAA officials said they are not concerned about allowing  the U.S. airlines to use their own employees to conduct the safety audits  because of the importance to the airlines of conducting sound safety audits  to limit the liability associated with establishing code-share arrangements  with foreign airlines. Table 1 lists the program guidelines that incorporate  the auditing standards.
section_title: FAA’s Reviews of the Safety Audit Reports Lacked Certain Management Controls
paragraphs: We found that FAA’s reviews of the safety audit reports lacked certain  management controls—including establishing reviewers’ qualifications,  verifying corrective actions, and documenting the reviews—but did employ  some management controls for monitoring and measuring performance.  Management controls are the continuous processes and sanctions that  federal agencies are required to use to provide reasonable assurance that  their goals, objectives, and missions are being met. These controls should  be an integral part of an agency’s operations and include a continuous  commitment to identifying and analyzing risks associated with achieving  the agency’s objectives, establishing program goals and evaluating  outcomes, and creating and maintaining related records.
Effective management controls require that personnel possess and  maintain a level of competence that allows them to accomplish their  assigned duties. In addition, management must identify the knowledge and  skills needed for various jobs, provide needed training, and obtain a  workforce that has the skills that match those necessary to achieve  organizational goals. However, we found that FAA has not established  competence criteria and qualifications for the personnel who review the  airlines’ safety audit reports. As a result, the FAA staff who are reviewing  the audit reports have different backgrounds and training, which may lead  to differing interpretations of the standards. The FAA headquarters official  who has reviewed a large number of the safety audit reports has aviation  experience as a military pilot and is trained as an ISO 9000 auditor but is  not trained as an FAA inspector and was hired in an administrative  capacity. Two other FAA headquarters staff who review the audit reports  have been trained as aviation safety inspectors. Furthermore, five FAA field  inspectors who are conducting many of the reviews have not had training  in IOSA, which six U.S. airlines in the Code-Share Safety Program are now  using as standards to audit the safety of their foreign code-share partners.  As a result of inspectors not having this training, this could impede FAA’s  review of the safety audits based on those standards. Moreover, the Code- Share Safety Program manager was transferred to a new position in  February 2005, leaving the position vacant since that time. As of June 2005,  FAA had not authorized this position to be filled and has denied a request  for another full-time staff position dedicated to the program. Since the  program manager’s departure, other staff in FAA’s International Programs  and Policy Office, which administers the Code-Share Safety Program, have  reviewed the safety audit reports in addition to performing their regular  duties. FAA program officials said that since the program manager was  transferred to another position, U.S. airlines must wait 3 to 4 weeks for  FAA to review their safety audits of foreign carriers, compared with waiting  1 day to 2 weeks before his transfer, and that U.S. airlines now must bring  all of their safety audit reports to FAA in Washington, D.C., for review—a  change that could hinder FAA’s review of documentation, such as safety  monitoring systems, that may be located at the airlines’ facilities. An FAA  management official said that because FAA’s Flight Standards Service, of  which the Code-Share Safety Program is a part, imposed a hiring freeze in  January 2005 for budgetary reasons, only critical positions are being  replaced. The official said that because the vacant position for the Code- Share Safety Program was not considered to be critical, it was not filled.
Effective management controls also require the establishment of policies  and procedures to verify that corrective actions have been taken in  response to identified problems. According to FAA and airline officials,  FAA staff review each audit report for about 2 to 4 hours, identifying any  areas that need further clarification or resolution. Although FAA staff  review the reports of all audits that U.S. airlines have conducted of their  foreign code-share partners, normally they only spot check whether  findings that were identified during the audit were resolved. According to  an FAA safety official, FAA relies on the U.S. airlines’ compliance  statements, signed by the airlines’ safety directors, which affirm that the  audits were conducted in accordance with the guidelines and that the  foreign carriers met the applicable ICAO standards, as proof that all  findings have been resolved. However, FAA’s reliance on the compliance  statements may not provide an effective management control to ensure  that corrective actions have been taken in response to audit findings. For  example, we found that FAA provided a memorandum of no objection to  DOT’s Office of International Aviation about a foreign code-share partner  that, according to an official from its U.S. partner, had not implemented all  of the corrective actions needed to resolve the findings. The safety audit  identified dozens of findings, many of which were also found in a second  audit 2 years later and, according to the airline, subsequently corrected.  Furthermore, because FAA has not provided its reviewers or the airlines  with a standard definition of “safety-critical” findings that must be  corrected before the audit can be closed, it is unknown whether these open  findings were safety critical. Moreover, the reasonableness of leaving open  dozens of safety audit findings is questionable, as is FAA’s reliance on the  airlines’ compliance statements as proof that all corrective actions have  been made. Although the U.S. carrier temporarily suspended the code- share arrangement with this foreign carrier, FAA officials said the  suspension occurred because of FAA’s concern about the safety oversight  of that foreign airline’s civil aviation authority, not because of the number  of audit findings or their lack of closure.
FAA uses compliance statements, which are based on the safety audit  results, as reasonable assurance that the foreign airlines meet ICAO safety  standards. However, FAA’s reliance on compliance statements may not  provide such assurance because FAA has accepted compliance statements  as proof that the carriers met ICAO safety standards, even in situations  when it questioned the audit results. For example, FAA provided  memorandums of no objection to DOT’s Office of International Aviation  that were based on safety audits conducted by one airline contractor over a  4-year period, many of which did not identify any findings, even though an  FAA official told us that he had discussed with the airline FAA’s concern  about the number of audits that did not identify any findings.
According to the Code-Share Safety Program guidelines, U.S. airlines  should not submit compliance statements to FAA until all corrective  actions have been completed; the statements should not be predicated on  future actions that are planned to be completed. However, FAA officials  said that they allow “nonsafety-critical” findings identified during the audit,  such as deficiencies in personnel training and omissions in manuals, to be  addressed later. Because FAA has not provided the airlines with a standard  definition of “safety-critical” findings that must be corrected before the  audit can be closed, airlines could interpret the term inconsistently in  documenting and resolving corrective actions. An FAA official indicated  that developing a definition of safety critical would be difficult and time  consuming. An aviation safety expert we consulted said that a definition of  safety critical would require considerable study and criteria development  because situations can be critical to safety in many ways. He added that a  well-trained and experienced aviation safety inspector could identify a  safety-critical situation. However, this same expert suggested that, as a  quality assurance measure, FAA select several audits each year and check  the underlying documentation in depth. Similarly, the DOT Inspector  General recommended in 1999 that FAA conduct comprehensive audits of a  sample of safety audits to confirm that carriers have applied agreed-upon  standards and procedures in conducting the audits. However, even if FAA  were to conduct such comprehensive audits, without a definition of safety- critical findings, the agency would still lack assurance that safety-critical  findings were identified and resolved.
FAA indicated that from August 2003 through July 2004, 18 of the 50 audit  reports on foreign airlines it reviewed were returned to U.S. carriers for  further action and 4 were placed on hold pending the outcome of IASA  reviews; the other 31 foreign carriers received memorandums of no  objection. Furthermore, FAA officials said that, according to anecdotal  information from some U.S. carriers, too many safety concerns were  identified during some safety audits for the carriers to proceed with  applications for code-share authorization. However, FAA officials said they  do not know how many times the safety audits have prevented airlines that  pose safety concerns from becoming code-share partners with U.S. airlines.
In addition, effective management controls require that documentation be  created and maintained to provide evidence of executing approvals,  authorizations, verifications, and performance reviews. FAA devised a  checklist for agency staff to complete while reviewing safety audit reports  to check for compliance with the program guidelines, record information  about findings, or report irregularities. FAA officials said that the checklist  was developed to establish and maintain consistency in reviewing the audit  reports. However, we found that the checklist did not consistently  document what actions FAA took when reviewing the airlines’ audit  reports, which findings it reviewed, and which corrective actions it verified  were implemented. For example, in some cases, the checklist provided  information about the closure of findings, but in other cases, no  information was recorded about closure. FAA officials said that portions of  the checklist may be left blank until the FAA reviewer has completed  discussions with the airline and answered all of the concerns to his or her  satisfaction, at which time the FAA reviewer will note that no irregularities  were found. Officials said that in such cases, the checklist would not  capture this process. However, not completing this information could  hinder future reviews of the same airline by impeding comparisons  between audits. Furthermore, because FAA often lacked documentation  that it had verified the closure of findings, we were unable to determine  how frequently FAA may have provided memorandums of no objection on  foreign carriers that had not implemented all corrective actions in response  to the findings, as occurred in the example discussed earlier.
Effective management controls also include monitoring to assess the  quality of performance over time. Management controls generally should  be designed to ensure ongoing monitoring during normal operations and  include regular management and supervisory activities, comparisons,  reconciliations, and other actions people take in performing their duties.  FAA officials said that the manager of the International Programs and  Policy Division, which is responsible for administering the Code-Share  Safety Program and is part of FAA’s Flight Standards Service, is briefed by  the Code-Share Safety Program staff on the results of their safety audit  reviews before a recommendation is made to the Director of Flight  Standards to sign the memorandums of no objection that are sent to DOT’s  Office of International Aviation. This procedure allows the International  Programs and Policy Division manager to monitor the results and the  decision-making processes involved. In addition to reviewing the audit  reports, FAA monitors the safety of foreign carriers through other sources  of information. FAA officials said they also review any accident and  incident information from aviation safety databases, company financial  histories, ICAO reports on the countries’ civil aviation authorities, media  reports, ramp inspection results, and information from FAA international  field offices about their inspections of foreign aircraft when these aircraft  enter the United States.
According to the Code-Share Safety Program guidelines, the U.S. airlines  participating in the program should have a process to monitor the safety of  their foreign code-share partners on an ongoing basis, and FAA should  review this monitoring process. FAA officials said they have reviewed the  monitoring systems at seven of the eight U.S. airlines participating in the  program. However, FAA had not documented its reviews of the  monitoring systems, so we were unable to verify that activity. Furthermore,  safety officials at three of the eight U.S. airlines said FAA had not reviewed  their monitoring systems. Without an FAA review, deficiencies in these  monitoring systems might not be identified.
FAA does not maintain information on the types and frequencies of audit  findings to provide a means of comparing the findings from initial and  recurrent audits of the same airline, or perform trend analysis that could  help identify problems across airlines or fleets. Trend analyses would be  useful for monitoring, on an ongoing basis, the effectiveness of FAA’s  internal quality control system. FAA officials said the checklists are not  used for tracking or trend analysis and that FAA does not formally examine  either the safety problems occurring most often or the geographic areas  where problems are occurring most frequently. However, the officials said  that the FAA program manager does want to have a general idea of the  types of problems being found, and the checklist provides this information  informally. According to one FAA official, the purpose of the checklist is to  ensure that the DOT guidelines are met, rather than to create a database of  findings. In our view, not maintaining such documentation could impede  analyses of trends and comparisons of findings, as well as limit  opportunities for assessing risks and prevents determining whether FAA  reviewed those findings.
Establishing performance measures is another component of effective  management controls. The Government Performance and Results Act of  1993 requires agencies to, among other things, set strategic and annual  performance goals, and measure and report on performance toward these  goals. Management controls play a significant role in helping managers  achieve those goals. FAA has established certain performance goals for the  Code-Share Safety Program, including reviewing at least 40 safety audit  reports during fiscal year 2004. FAA exceeded this goal by completing 57  reviews. In addition, FAA set a performance goal of meeting with major  U.S. air carriers to request feedback on the Code-Share Safety Program.  FAA met this goal in 2004.
section_title: U.S. Airlines Are Auditing Foreign Partners’ Safety, and Partners Are Taking Corrective Action, but Documentation of Corrective Actions Is Often Lacking
paragraphs: The eight U.S. airlines participating in the Code-Share Safety Program have  conducted the safety audits of their foreign code-share partners and have  monitored the safety of their code-share partners between audits, as  specified under the guidelines. Through those audits, the U.S. airlines have  identified numerous safety issues associated with their foreign partners’  operations. After completing the audits, the U.S. airlines have submitted  written statements to FAA affirming their foreign code-share partners’  compliance with ICAO standards, as specified under the guidelines.  However, the U.S. airlines have not always documented the  implementation of actions taken in response to the findings. Many airlines  are now moving to adopt the international safety audit program, IOSA,  which contains procedures that would help to ensure that corrective  actions implemented in response to audit findings are documented. Most  U.S. airline officials said they believe the Code-Share Safety Program  provides reasonable assurance of safety or is effective, but some officials  also suggested various changes in its administration.
subsections: section_title: U.S. Airlines’ Safety Audits Primarily Identified Findings in Foreign Partners’ Flight Operations and Maintenance and Engineering
paragraphs: The U.S. airlines participating in the Code-Share Safety Program have been  assessing the safety of their foreign code-share partners at least every 2  years, as the guidelines specify. We estimate, based on the results of our  sample of 149 randomly selected safety audit reports, that there are 2,047  findings among the audits that the eight U.S. airlines conducted of foreign  carriers, which FAA reviewed from February 2000 through September  2004. The program guidelines define a finding as an instance in which “the  performance of the standard does not meet the established criteria” under  ICAO standards. We estimate that 75 percent of the audits of foreign  carriers that the eight U.S. airlines conducted of foreign carriers and that  FAA reviewed from February 2000 through September 2004 contained at  least one finding. Airline officials told us that most findings related to a lack  of documentation. Documentation is important to ensure the  implementation of management controls, which should appear, for  example, in management directives and operating manuals. However, we  found that many of the safety audit findings were broader in scope than a  lack of documentation and extended to a lack of underlying policies and  procedures. We further estimate that findings related to deficiencies in  policies and procedures accounted for 23 percent of all findings.
The audits reviewed the carriers’ compliance in eight major categories  (organization, flight operations, flight dispatch, maintenance and  engineering, cabin operations, cargo and dangerous goods, ground  handling, and security). As shown in figure 5, the findings spanned all  eight categories, but the largest numbers were in two categories: (1) flight  operations, which govern the activities of the pilots, including training, and  (2) maintenance and engineering, which involves the oversight of activities  to maintain, repair, and overhaul aircraft, aircraft engines, and parts. In the  flight operations category, the findings included a lack of drug and alcohol  testing policies and a lack of documentation on flight time and rest  requirements for flight personnel. In the maintenance and engineering  category, one common type of finding related to the maintenance and  calibration of tools and supplies, which could affect safety.
section_title: Airlines Submitted Compliance Statements to FAA but Did Not Always Document the Closure of Findings
paragraphs: After U.S. airlines completed their audits, their safety directors submitted  statements to FAA affirming their foreign code-share partners’ compliance  with ICAO standards. FAA officials said they rely on these compliance  statements as the primary evidence that the foreign code-share partners of  U.S. airlines have resolved all safety-critical findings. However, on the basis  of our review of a sample of the audit reports, we estimate that, for 68  percent of the identified findings, the documentation was insufficient to  demonstrate that the findings had been closed or were resolved.  Specifically, the documentation either failed to indicate at least one of the  following three elements: (1) what corrective action was taken, (2) who  accepted the corrective action, and (3) when the corrective action was  accepted or the documentation was insufficient to determine whether the  findings were closed. An estimated 28 percent of the audit reports that  contained findings had at least one finding that lacked all three elements  documenting corrective actions. The Code-Share Safety Program  guidelines do not indicate that U.S. airlines should have documentation  available for FAA’s review to provide evidence of what corrective action  was taken, who accepted the action, and when the action occurred in  response to the findings identified in audits of their foreign code-share  partners.
We asked the eight U.S. airlines participating in the Code-Share Safety  Program what types of systems they were using to track any findings that  were not resolved when the safety audit was complete. We found that three  of the U.S. airlines were using computer systems to track the closure of  such open findings; three other airlines had computer systems that could  track the closure of findings, but their foreign partners had no open  findings; and two airlines indicated that they did not have systems to track  open findings because their foreign partners did not have any open  findings. At one U.S. airline that was using a computer system to track open  findings, officials said that a computer malfunction resulted in the loss of 6  months of data. An official from this airline said that before 2004, the  airline coordinated closure of any findings directly with the contractor.  When asked to produce this information, the airline did not have finding  closure documentation available for audits conducted before 2004. This  contractor said that although his firm was asked a few times by the U.S.  carrier to check on the closure of audit findings by its foreign partner, the  U.S. airline was responsible for tracking the closure of findings.
Airlines also lacked documentation on the closure of findings in part  because an unknown number of findings were closed on-site during the  audits and not documented. The FAA program manager said he  discouraged closing out findings on-site without documentation during the  audits because it does not leave an audit trail about what findings were  identified. Documentation provides a record of the execution of  management controls which, in this situation, relate to the implementation  of corrective actions.
We estimate that 25 percent of the audits were closed with no findings  identified. According to an FAA official, audits that identify no findings are  questionable because the airlines must comply with so many requirements  under either ICAO or IOSA standards. One U.S. airline used a contractor to  conduct 31 of the audits of foreign airlines in our sample from 1999 through  2003, over half of which identified no findings. As described earlier, an  FAA official told us that he had discussed with the airline FAA’s concern  about the number of audits conducted by the contractor that did not  identify any findings. The FAA official also said that he helped the airline  revise its approach to conducting the audits as a part of its internal  evaluation program. The contractor told us that it is common for the safety  audits not to identify findings because the airlines have prepared for the  audit, and the audit findings are sometimes resolved on the spot. The  contractor also said that his firm often recommended best practices that  the foreign carriers could implement, but these recommendations did not  relate to violations of ICAO standards and, thus, were not considered to be  findings. Furthermore, this contractor said that a representative from the  U.S. airline, who accompanied the contractor’s auditors on the audits, kept  the U.S. airline informed.
section_title: U.S. Airlines Also Monitor the Safety of Their Foreign Partners between Audits
paragraphs: The eight U.S. airlines participating in the Code-Share Safety Program have  processes to monitor the safety of their foreign code-share partners on an  ongoing basis, including their accident and incident rates, financial  condition, equipment age, labor issues, and other issues, as called for in the  program guidelines. Safety officials from the eight U.S. airlines said that, to  their knowledge, no fatal accidents had occurred on their foreign code- share routes since the Code-Share Safety Program began in 2000. We  observed the systems and information sources that each U.S. airline used  for monitoring. Airline officials showed us, for example, safety  questionnaires that they sent to their code-share partners between formal  safety audits, news subscription services, and aviation safety Web sites.  Some airline officials also said they occasionally made on-site visits to  monitor their partners’ safety. The airlines also indicated that they monitor  any accident and incident data for their code-share partners. According to a  safety official at one U.S. airline, a carrier’s past accident and incident  record does not conclusively prove that a safety problem exists, but it can  be an indicator of other deficiencies, such as gaps in training. Some  officials from airlines that are part of global alliances also said that they  share safety information about their mutual foreign code-share partners.  Four U.S. airlines had created computer databases to maintain this  monitoring information while the other four maintained paper files.
section_title: Use of International Safety Audit Program May Address Some Weaknesses in the U.S. Safety Audit Program
paragraphs: As U.S. airlines and their foreign code-share partners begin to use IOSA—a  new safety audit program developed by IATA—some of the weaknesses  that we observed in the Code-Share Safety Program may be addressed, and  U.S. airlines may receive other benefits. Increased use of IOSAs may help  to ensure that audit findings are resolved and corrective actions  implemented. IOSA requires that findings that are identified during the  audit be documented, excluding those that are corrected immediately on- site during an audit. In addition, IOSA requires documentation of closure  for findings, including the three elements we identified—(1) a description  of the corrective actions taken, (2) who accepted the corrective actions,  and (3) when the corrective action was accepted—as well as the reasoning  used by the auditing organization to clear the findings. As noted,  documentation of one or more of these elements was missing, or it could  not be determined if elements were missing for an estimated 68 percent of  the audit findings.
Six of the eight U.S. airlines use IOSA standards to audit the safety of their  foreign code-share partners, one may do so in the future, and one does not  plan to use the standards to audit the safety of its foreign code-share  partner. Moreover, according to some airline officials, U.S. airlines have a  financial incentive to encourage their foreign code-share partners to  undergo IOSAs because the auditing costs are shifted from the U.S. airline  to its foreign partner. However, not all U.S. airlines plan to require IOSAs of  their foreign code-share partners. For example, officials from one U.S.  airline said that IOSAs may be too expensive for some small foreign  carriers. Similarly, officials at another U.S. airline said that IOSAs are  applicable to airlines with large fleets and major processes but may not be  practical for smaller airlines. Officials at a third U.S. airline said they  preferred to continue conducting the safety audits themselves, rather than  using an auditing organization selected by IATA, because they wanted the  assurance of examining their partners’ operations in person, rather than  relying on an external organization.
Finally, increased use of IOSAs may help standardize aviation safety  auditing and streamline FAA’s review of audit reports. Under the IOSA  program, the airlines can obtain the audit results of their mutual code-share  partners. Of the eight U.S. airlines with foreign code-share partners, six  share 18 of the same foreign code-share partners. FAA recently allowed  U.S. airlines to submit for review audit reports that other U.S. airlines had  conducted on a shared foreign code-share partner. Some U.S. airlines, as  members of global airline alliances, plan to share their audit reports of  foreign partners and reduce duplicative audits. The IOSA program should  make it easier for airlines that are not in such alliances to share audit  reports. Increased sharing of the reports could reduce the number of  safety audits that the U.S. airlines would need to conduct of their foreign  partners and could thus reduce the number of reports that FAA would need  to review.
section_title: Most U.S. Airline Officials Believe the Code-Share Safety Program Provides a Reasonable Assurance of Safety
paragraphs: Officials at most U.S. airlines participating in the Code-Share Safety  Program told us they believe that the program provides reasonable  assurance of safety concerning their foreign code-share partners or is  effective. One airline official described the program as an “ingenious  technique” that has had the effect of raising aviation safety standards  worldwide by ensuring that safety issues will be resolved. This airline  official said that some foreign airlines, seeking to become code-share  partners of U.S. airlines, have restructured programs, rewritten manuals,  and instituted new management techniques—evidence, he said, of the  program’s effectiveness. Another U.S. airline official said that, without the  Code-Share Safety Program, U.S. airlines might not conduct safety audits of  their foreign code-share partners. An official at another U.S. airline said the  Code-Share Safety Program is a means to ensure that a carrier meets  minimum ICAO-based international aviation safety standards and that the  IOSA program creates a baseline of auditing standards to be followed  worldwide. However, the official said that a safety audit, whether  conducted by an auditing organization selected by IOSA or a U.S. airline, is  only a snapshot of the carrier for the period in which the audit is  conducted. The airline official said that the carrier’s actions before the  audit or after the audit may differ and cannot be adequately evaluated until  additional safety information is collected from the carrier between safety  audits or until the next safety audit.
An official at another U.S. airline participating in the Code-Share Safety  Program said that although a safety audit provides a very good assessment  of an airline’s compliance with aviation safety standards, it does not  guarantee the safety of the carrier’s operations. This official added that  even if a safety audit were conducted on a carrier monthly, it would not  guarantee that the carrier would never have an accident. Furthermore, an  official at another U.S. airline said that the Code-Share Safety Program is  not necessarily required to provide reasonable assurance of safety  concerning the foreign code-share partners of U.S. airlines and that the  airline does not necessarily believe that formal, FAA-approved safety audits  are the only way to gain such assurance. This airline official said that U.S.  airlines should not be required to conduct safety audits of foreign airlines  that are operating out of countries that FAA rated as IASA category 1 and  that U.S. airlines should be able to choose whether to conduct safety audits  in countries that FAA has rated as IASA category 2 or has not rated. This  airline official added that while the U.S. airline may continue to audit its  partners on its own, it does not believe that FAA should oversee this  process. However, an FAA IASA program official told us that the IASA  program focuses on the capabilities of the foreign civil aviation authorities  and does not ensure the safety of any carriers operating in IASA category 1  countries. This FAA official also said that inconsistencies in aviation safety  oversight can exist throughout the world, even in countries with “higher”  standards, and that some countries exceed ICAO standards, while others  do not.
A safety official at one U.S. airline said he believed that the Code-Share  Safety Program guidelines should be made regulations. Although officials  from DOT’s Office of International Aviation and FAA said that making the  program regulatory is not needed because it is working well, this airline  safety official said that making the program regulatory would allow  requirements to be applied more evenly to all airlines participating in the  program. This airline official added that DOT is requiring the guidelines to  be followed and therefore they are regulations in practice. A safety official  at another U.S. airline questioned why DOT requires “guidelines” to be  followed. He said that if DOT wants “rigid compliance” with the guidelines,  it should make the program regulatory. A safety official at a third U.S.  airline said the program’s requirements should be standardized, noting that,  for example, FAA was inconsistent about its requirements for reviewing  auditors’ qualifications. An aviation safety expert we consulted also said  that the program should be made regulatory, observing that both the Code- Share Safety Program and IASA suffer from a “lack of regulatory teeth” and  that making them regulatory would provide clarity to the DOT  requirements, which he said are “mere policies.” At the same time, this  expert said that although the program is not regulatory, the Code-Share  Safety Program guidelines clearly lay out what is expected of the airlines  and set the standards that must be met. He added that under the guidelines,  U.S. airlines are held accountable for the safety of their foreign code-share  partners.
Finally, officials at two airlines said that they would like FAA to provide a  definition of safety critical or to define when an audit is considered to be  closed so that it would be clear which findings must be resolved before  closing an audit and submitting a compliance statement. As noted, FAA  officials said that they allow nonsafety-critical findings identified during  the audits to be addressed after the code-share arrangement is authorized.
section_title: Conclusions
paragraphs: The safety of foreign code-share partners of U.S. airlines is important  because several million people fly on those foreign carriers using tickets  purchased from U.S. airlines each year. Under the Code-Share Safety  Program, the U.S. airlines are auditing the safety of their foreign code-share  partners and identifying safety concerns, which the foreign carriers are  addressing. However, FAA’s reviews of the safety audit reports lack  management controls for establishing reviewers’ qualifications, verifying  corrective actions, and documenting the reviews. FAA, for example, has  not established the qualifications needed to review safety audit reports,  and FAA field inspectors, who are reviewing many of the safety audit  reports, have not been trained in the IOSA program—potentially impeding  FAA’s review of audits that were conducted using those standards. In  addition, the program guidelines do not provide clear direction to the U.S.  airlines and FAA reviewers on which concerns are critical to safety and  must be addressed before DOT’s Office of International Aviation will  authorize or reauthorize a code-share arrangement. Without a definition of  safety-critical concerns and complete documentation of the closure of  findings, FAA lacks clear criteria for responding to requests from DOT’s  Office of International Aviation about the safety of foreign carriers and  lacks assurance that safety-critical concerns have been addressed.  Furthermore, FAA is not using effective management controls when it fails  to document its reviews of the airlines’ safety audit reports. Without  complete documentation, a determination cannot be made of what actions  FAA took when reviewing the reports, which findings it reviewed, and  which corrective actions it verified were implemented. Because  documentation on FAA’s verification of the closure of findings was often  lacking, we were unable to determine how frequently FAA may have failed  to object to the authorization of code-share arrangements with foreign  carriers that had not implemented all corrective actions in response to the  findings. FAA also has not implemented a DOT Inspector General’s  recommendation that it conduct a comprehensive examination of a sample  of audit reports to verify the underlying documentation. Furthermore,  FAA’s not collecting and tracking safety audit findings is an obstacle to  conducting trend analysis or spotting anomalies.
The airlines’ increasing adoption of the IOSA program as a worldwide  safety auditing standard is likely to change how FAA conducts its safety  reviews of foreign code-share partners of U.S. airlines. Moreover, IOSA  requires that actions to correct all findings, except those that are corrected  during an audit, be documented—a requirement that is lacking in FAA’s  program. However, the adoption of the IOSA program is likely to be  gradual, given that, as of June 2005, 66 of IATA’s 265 members had  completed the program.
Finally, although DOD and FAA officials said they have different program  objectives, the two federal agencies are nevertheless duplicating efforts by  reviewing many of the same audit reports. In addition, DOD is not receiving  the foreign airport security assessment information from TSA that DOT is  receiving. TSA’s information would provide DOD with more complete data  for its safety reviews.
section_title: Recommendations
paragraphs: To improve the safety oversight of foreign code-share operations, we  recommend that the Secretary of Transportation direct the FAA  Administrator to implement the following three recommendations: 1. Revise the Code-Share Safety Program guidelines to improve the  effectiveness of the program and the clarity of the procedures that the  airlines should follow in documenting and closing out safety audit  findings. Because the audit guidelines indicate that the airlines should  not submit compliance statements until all corrective actions have  been completed, but FAA is allowing the airlines to resolve “nonsafety- critical” findings later, FAA should consider either following that  guideline or defining “safety-critical” audit findings, so that the airlines  and FAA reviewers know which types of findings must be corrected  before submitting the compliance statements. 2. Develop mechanisms to enhance FAA’s management controls over its  reviews of the safety audit reports. In developing the mechanisms, FAA  should consider standardizing the qualifications and training needed  for agency staff to review the airlines’ safety audit reports; identifying  ways to document its reviews of the airlines’ safety audit reports;  increasing the scrutiny of audit reports that have an unusually high or  low number of findings, periodically selecting a sample of safety audits  to conduct a comprehensive review of the underlying documentation  collected; and collecting and analyzing information on the audit  findings for the foreign code-share partners of U.S. airlines so that the  data can be more easily quantified and analyzed to spot possible trends  and anomalies, should FAA decide such analyses are needed. 3. Finally, explore with DOD potential opportunities to reduce duplication  of efforts in reviewing the same safety audit reports.
Because security is an important component of assessing airline safety, to  improve DOD’s oversight of foreign carriers that transport DOD personnel,  we also recommend that the Secretary of Homeland Security direct the  Assistant Secretary of Homeland Security for TSA to develop a process of  routinely coordinating with DOD regarding information on the security of  foreign airports for DOD to consider in reviewing the safety of foreign  airlines. Such a process could be documented in a memorandum of  understanding or other written procedures to ensure such coordination.
section_title: Agency Comments
paragraphs: We provided drafts of this report to the Department of Homeland Security,  (DHS), DOD, and DOT. DHS provided written comments, agreeing with our  recommendation regarding TSA. DHS’s comments are reprinted in  appendix III. DOD provided no comments on our findings or  recommendations. DOD and DOT provided some technical clarifications,  which we incorporated into this report as appropriate. We received  comments from DOT officials, including FAA’s Deputy Associate  Administrator for Aviation Safety. FAA generally agreed with the report and  agreed to consider our recommendations. In addition, FAA provided  comments on the Code-Share Safety Program, emphasizing that it is a  collaborative effort between DOT’s Office of the Secretary, FAA, and the air  carriers. FAA officials also said that the program established guidelines for  approving international code-share operations, with the intent of  encouraging the highest possible levels of safety for international code- share operations. According to FAA, the program outlines the necessary  steps that U.S. air carriers must follow in seeking approval from DOT to  conduct code-share operations with foreign air carriers. The officials added  that the Code-Share Safety Program charges U.S. air carriers with the  primary responsibility for ensuring that their foreign code-share partners  comply with applicable international aviation standards.
As agreed with your office, unless you announce the contents of this report  earlier, we plan no further distribution until 30 days from the date of this  letter. At that time, we will send copies of this report to interested  congressional committees; the Secretary of Transportation; the  Administrator of FAA; the Secretary of Defense; the Secretary of Homeland  Security; and the Assistant Secretary of Homeland Security for the  Transportation Security Administration. Copies will also be available to  others upon request and at no cost on GAO’s Web site at www.gao.gov.
If you or your staff have any questions about this report, please call me at  (202) 512-2834 or dillinghamg@gao.gov. Contact points for our Offices of  Congressional Relations and Public Affairs may be found on the last page  of this report. GAO staff who made major contributions to this report are  listed in appendix IV.
section_title: Objectives, Scope, and Methodology
paragraphs: Our objective was to review the measures that the federal government is  taking to provide reasonable assurance of safety and security when  passengers travel on flights operated by the foreign code-share partners of  U.S. airlines. To accomplish this, we reviewed (1) the extent to which the  Department of Transportation’s (DOT) authorization of U.S. airlines’ code- share arrangements with foreign airlines is designed to consider safety and  security, (2) how well FAA has managed the Code-Share Safety Program,  and (3) the extent to which U.S. airlines have implemented the Code-Share  Safety Program, and the results of their efforts.
To determine how safety and security are considered in DOT’s  authorization of U.S. airlines’ code-share arrangements with foreign  airlines, we interviewed officials at DOT’s Office of International Aviation,  Federal Aviation Administration (FAA), Transportation Security  Administration (TSA), and the Department of Justice (DOJ) and reviewed  the Code-Share Safety Program guidelines and related program  documentation, applicable international aviation safety standards, and  relevant legal authorities. Our review covered the U.S. airlines’ code-share  partnerships with foreign carriers that DOT authorized from February  2000, when the Code-Share Safety Program began, through fiscal year  2004. At DOT, we interviewed the officials who decide whether to  authorize such partnerships about the authorization process, their sources  of information, and how often they authorize the partnerships. To gain a  better understanding of the authorization process and the information  considered, we also reviewed a sample of code-share applications that U.S.  airlines had filed to establish code-share partnerships with foreign carriers.  Our sample consisted of one randomly selected application filed by each of  the eight U.S. airlines participating in the Code-Share Safety Program.
We also interviewed DOT security officials about how they provide security  clearances for foreign carriers and how often they have provided those  clearances for code-share authorization. Because TSA was the source of  aviation security information for DOT, we interviewed TSA officials about  how they assess the security of foreign airlines and airports. We also  reviewed data from TSA about the results and frequency of its security  assessments of foreign airports and related legal authorities. Based on our  understanding of the data through interviews with TSA officials, we  determined that the data were sufficiently reliable for our purposes. In  addition, we interviewed DOT officials who review the competitive aspects  of the code-share arrangements about how they conduct their reviews and  how often they have provided those clearances for code-share  authorization. Because these DOT officials received advice from DOJ on  potential antitrust issues involving the code-share partnerships, we also  interviewed DOJ officials who provided that advice about their process and  sources of information.
At FAA, we interviewed officials about how they assess the capabilities of  foreign civil aviation authorities through the International Aviation Safety  Assessment (IASA) program and how those assessments relate to the  Code-Share Safety Program. We also analyzed data on the results and  frequency of IASA reviews since the Code-Share Safety Program was  initiated. Based on our understanding of the data through interviews with  FAA officials, we determined that the data were sufficiently reliable for our  purposes. We reviewed documentation that FAA staff had prepared when  they reviewed the airlines’ safety audit reports to determine how they  documented their reviews. We also discussed with FAA officials how often  FAA provided memorandums of no objection to DOT’s Office of  International Aviation to support U.S. airlines’ applications for code-share  arrangements with foreign carriers.
Because the Code-Share Safety Program was designed to assess foreign  airlines’ compliance with aviation safety standards established by  International Civil Aviation Organization (ICAO), we interviewed ICAO  officials about the standards, related international aviation safety issues,  and the ICAO Universal Safety Oversight Audit Program, which assesses  the capabilities of countries’ civil aviation authorities. In addition, because  many airlines are planning to use a new international safety audit  program—the International Air Transport Association’s (IATA) Operational  Safety Assessment (IOSA) program—to assess the safety of their foreign  partners, we interviewed IATA officials about how the program was  developed, how airlines plan to implement it, and how it could affect the  Code-Share Safety Program. For background information on how aviation  safety varies internationally, we obtained data from IATA on aviation  accident rates for different world regions. We did not review the reliability  of IATA’s aviation accident data because we used this information only for  background purposes. We also interviewed officials from the Air Transport  Association—a U.S. airline association—about its involvement in  establishing the DOD safety audit program and its views on the Code-Share  Safety Program and FAA’s IASA program. Finally, because we found during  our review that DOD had also established a program for reviewing the  safety of foreign carriers, we interviewed DOD officials about the design  and implementation of its program. In addition, we obtained information  about the safety audit reports that DOD had reviewed from fiscal year 2001  through fiscal year 2004 and the results, which we compared with the  results of those that FAA reviewed. We also discussed with FAA and DOD  officials the extent to which they have coordinated their efforts.
To determine how well FAA has managed the Code-Share Safety Program,  we evaluated whether DOT’s Office of the Secretary and FAA incorporated  selected government auditing standards in the program’s design and  whether FAA effectively used management controls in reviewing the safety  audit reports. Because the Code-Share Safety Program establishes an audit  program, we reviewed whether the program’s design, as reflected in the  program guidelines, conforms to certain standards identified in  Government Auditing Standards. We reviewed selected general  standards that are contained in Government Auditing Standards  (independence, professional judgment, and competence) to assess the  program’s design. Although we examined the audit methodologies that the  U.S. airlines had developed and submitted to FAA for review, we did not  review them for conformance with government auditing standards because  FAA had already conducted this review as a condition of accepting the U.S.  airlines’ participation in the program. In addition, because we were  evaluating the management of a government program, we examined FAA’s  application of management controls, which is synonymous with the term  “internal controls,” in its reviews of the safety audit reports using  Standards for Internal Control in the Federal Government. We selected  the management controls that were applicable to FAA’s review of the audit  reports for establishing reviewers’ qualifications, verifying corrective  actions, documenting the reviews, and monitoring and measuring  performance. We also reviewed the recommendations contained in a 1999  DOT Office of the Inspector General report on aviation safety under  international code-share agreements to determine whether and to what  extent the report’s recommendations—about how a code-share safety audit  program should be designed—were implemented.
To determine the extent to which U.S. airlines have implemented the Code- Share Safety Program and the results, we interviewed officials at the eight  U.S. airlines that were participating in the program about how they were  assessing the safety of their foreign partners and reviewed a sample of the  reports. We drew a stratified random probability sample of 153 reports of  audits conducted by U.S. airlines of their foreign code-share partners. This  sample was drawn from a population of documentation maintained by FAA  for the 242 audit reports that the agency had reviewed from February 2000  through September 2004. Of these 153 sampled audits, 2 were out of scope  because the airlines withdrew them from consideration and 2 were in  scope, but we did not complete our reviews of these reports. We ultimately  collected information for 149 in-scope audits. With this probability sample,  each audit report in the study population had a positive probability of being  selected, and that probability could be computed for any audit. We  stratified the population into nine groups on the basis of the U.S. airline  conducting the audit, and further, for some of those airlines, whether the  foreign airlines being audited were code-share partners with more than one  U.S. airline or whether FAA’s records of its reviews of the audit reports  contained comments about the findings. Each sampled audit was  subsequently weighted in the analysis to statistically account for all of the  audits in the study population, including those that were not selected.  During our audit work, three airlines provided information about a total of  14 additional audit reports that, according to the airlines, FAA had  reviewed. These 14 audits were not included in the population from which  we drew our sample because FAA’s files did not contain information about  them. Estimates generated in this report pertain only to the 242 audit  reports that, according to FAA’s files, the agency reviewed.
Because we followed a probability procedure based on random selections,  our sample is only one of a large number of samples that we might have  drawn. Since each sample could have provided different estimates, we  express our confidence in the precision of our particular sample’s results in  95-percent confidence intervals. These are intervals that would contain the  actual population values for 95 percent of the samples we could have  drawn. As a result, we are 95-percent confident that each of the confidence  intervals in this report will contain the true values in the study population.  All percentage estimates from the sample of audits have sampling margins  of error of plus or minus 10 percentage points or less unless otherwise  noted. All numerical estimates other than percentages have margins of  error of plus or minus 10 percent of the value of those estimates or less  unless otherwise noted.
We did not determine whether the airlines complied with international  aviation safety standards. However, we performed a content analysis of the  audit reports in our sample to determine what types of safety findings were  identified regarding the foreign carriers. We recorded the findings and  grouped them into eight categories: (1) organization, (2) flight operations,  (3) flight dispatch, (4) maintenance and engineering, (5) cabin operations,  (6) cargo and dangerous goods, (7) ground handling, and (8) security— because the reports were generally organized into those categories. We  then further divided those eight categories into at least six issue  subcategories. Two coders independently categorized each finding, and any  coding disagreements were resolved between the coders or by a third  reviewer.
During our review of the audit reports, we also attempted to determine  whether corrective actions taken in response to the findings were  documented. To accomplish this, we looked for evidence of (1) what  corrective action was taken, (2) who accepted the corrective action, and  (3) when the corrective action was accepted. We considered these three  elements to be sufficient evidence of documentation after observing how  some airlines had documented the closure of findings and by reviewing  Government Auditing Standards, which indicate that auditors should  examine whether recommendations from previous audits have been  implemented, and from Standards for Internal Control in the Federal  Government, which require management to determine whether proper  actions have been taken in response to findings and audit  recommendations.
In addition to reviewing the audit reports at the airlines, we interviewed  safety officials (typically the safety directors) at all eight U.S. airlines  participating in the Code-Share Safety Program about how they assess the  safety of their foreign code-share partners, including how they plan, carry  out, and close the audits, as well as monitor the safety of their foreign  partners between audits. We also observed the monitoring systems that  they had implemented, as the program guidelines require, and sources of  information that they used to monitor the safety of their foreign code-share  partners. In addition, we asked the U.S. airline safety officials about their  program-related interactions with FAA and DOT’s Office of International  Aviation, whether and how they believe the program could be improved,  and what they thought about the implications of the airlines’ increased  adoption of IOSA by as an international aviation safety audit program. We  also obtained the views of an aviation safety expert about the Code-Share  Safety Program. We selected this expert because of his experience in  aviation safety, which included helping to design FAA’s IASA program.  Because some airlines had used contractors to conduct safety audits of  their foreign code-share partners, we interviewed one contractor who said  that he had conducted or helped to conduct safety audits for five of the  eight U.S. airlines in the Code-Share Safety Program about how his firm  conducted the audits and the qualifications of his staff.
Finally, for background information on the extent to which passengers are  traveling on foreign code-share partners of U.S. airlines, we asked the eight  U.S. airlines to provide such data from 2000 through 2004 using the same  methodology, which was based on the number of tickets that the U.S.  airlines sold for travel on their foreign code-share partners. For example, if  a U.S. airline sold a single ticket for travel that included one or more  foreign code-share partner flight segments, this ticket was counted once. If  a U.S. airline sold separate tickets for travel that included more than one  foreign code-share partner flight segment, each flight segment was counted  as a separate ticket. Some airlines could not provide data for all 4 years, but  all eight U.S. airlines were able to provide data for 2004, which we reported.  We did not independently verify this information provided by the airlines  because it was used only for background purposes.
section_title: U.S. Carriers and Their Foreign Code-Share Partners
paragraphs: Alaska (1 partner)
America West (1 partner)
American (24 partners)
Contintental (18 partners)
AeroLitoral Aeromexico Air Europa Air France Alitalia Brit Air  COPA CSA Czech Emirates EVA Airways flybe.British European KLM Cityhopper KLM Exel Airlines KLM Royal Dutch Airlines Korean Airlines Maersk Air TAP Air Portugal Virgin Atlantic (Continued From Previous Page)
Delta (21 partners)
Northwest (11 partners)
Aeromexico Air Alps  Air France Alitalia CSA Czech KLM Cityhopper  KLM Exel  KLM Royal Dutch Airlines Korean Airlines Malev Express Malev Hungarian Airlines (Continued From Previous Page)
United (23 partners)
US Airways (9 partners)
section_title: Comments from DHS
section_title: GAO Contacts and Staff Acknowledgments
subsections: section_title: GAO Contacts
section_title: Staff Acknowledgments
paragraphs: In addition to the above individuals, Elizabeth Eisenstadt, Jessica A. Evans,  Brandon Haller, Bob Homan, David Hooper, Casey Keplinger, Elizabeth A.  Marchak, Sara Ann Moessbauer, Mark Ramage, and Sidney Schwartz made  key contributions to this report.