Airline Consumer Protections: Additional Actions Could Enhance DOT's Compliance and Education Efforts
Released: 2018-11-20T00:00:00


Highlights: section_title: Why GAO Did This Study
paragraphs: Airlines recently came under scrutiny for their treatment of passengers—including a high-profile incident in which a passenger was forcibly removed from an overbooked flight. However, airlines maintain that service has improved, citing better on-time performance and lower airfares. DOT has the authority to issue and enforce certain consumer protection requirements. DOT also educates passengers about their rights.

GAO was asked to examine airline consumer protection issues. This report examines, among other issues, (1) trends in DOT's data on airline service; (2) the effectiveness of DOT's compliance efforts; and (3) the extent to which DOT's passenger education efforts align with key practices for consumer outreach. GAO reviewed DOT data on airline service and analyzed passenger complaint data for the 12 largest domestic airlines from 2008 through 2017; reviewed relevant documents and data on DOT's compliance program; assessed DOT's educational efforts against key practices for successful consumer outreach; and interviewed DOT officials. GAO interviewed or obtained written information from 11 of the 12 airlines.

section_title: What GAO Found
paragraphs: The Department of Transportation's (DOT) data offered mixed information on whether airlines' service improved from 2008 through 2017. While DOT's operational data on rates of late flights, denied boardings, and mishandled baggage generally suggested improvement, the rate of passenger complaints received by DOT increased about 10 percent—from about 1.1 complaints per 100,000 passengers to 1.2 complaints per 100,000 passengers.

DOT conducts five key activities to ensure airlines' compliance with consumer protection requirements (see table). However, GAO found that DOT lacked performance measures to help it evaluate some of these activities and that it could improve its procedures (i.e., guidance documents and training materials), that analysts use to code passenger complaints.

Performance measures : DOT has established objectives for each of its five key compliance activities that state what it seeks to achieve; however, DOT lacks performance measures for three objectives. For example, DOT lacks a performance measure for conducting inspections of airlines' compliance with consumer protection requirements at airlines' headquarters and at airports. As a result, DOT is missing opportunities to capture critical information about airlines' compliance with consumer protection requirements.

Procedures : DOT has procedures to help analysts code passenger complaints and identify potential consumer protection violations. GAO found that DOT's guidance for coding passenger complaints did not consistently include definitions or examples that illustrate appropriate use or help analysts select among the various complaint categories. Additional procedures would help DOT ensure that complaints are consistently coded and that potential violations are properly identified.

GAO found that while DOT has taken steps to educate passengers on their rights, its efforts did not fully align with four of nine key practices GAO previously identified for conducting consumer education. For example, while DOT has defined the goals and objectives of its outreach efforts, it has not used budget information to prioritize efforts or established performance measures to assess the results. DOT has also not solicited input directly from passengers to understand what they know about their rights. Taking such actions would provide DOT with greater assurance that its efforts are meeting passengers' needs.

section_title: What GAO Recommends
paragraphs: GAO is making six recommendations, including that DOT: develop performance measures for compliance activities, improve its procedures for coding airline passengers' complaints, and improve how passenger education aligns with GAO's key practices. DOT concurred with our recommendations and provided technical comments, which we incorporated as appropriate.

section_title: Letter
paragraphs: Each year hundreds of millions of passengers rely on airlines to get them  to their destination without incident. However, if a service incident does  occur—such as a delayed or cancelled flight, mishandled or lost baggage,  an unsatisfactory customer-service experience, or a denied boarding—a  passenger’s experience can quickly deteriorate. For example, video of a  passenger being forcibly removed from an overbooked United Express  flight awaiting departure at Chicago’s O’Hare International Airport on April  9, 2017, went viral on social media, focusing national attention on the  common airline business practice of overbooking flights—which can lead  to denied boardings—and what is allowed under federal regulation.  Despite the public attention, airlines have maintained that these high- profile incidents are isolated and that service has improved in recent  years, citing better on-time performance, lower airfares, and newer  aircraft. More recently, some airlines’ services and treatment of  passengers, as well as their traveling pets, have come under additional  public scrutiny, as have related federal requirements. For example,  airlines have called for additional regulation of service animals after  instances in which passengers tried to misrepresent pets as emotional  support animals to avoid certain fees and requirements.

While U.S. airlines’ business practices were largely deregulated following  the Airline Deregulation Act of 1978, a number of consumer protections  for airline passengers (i.e. “consumer protections”) are in place at the  federal level. For example, some consumer protections are in federal  statute, such as the Air Carrier Access Act of 1986, as amended, which  prohibits airlines from discriminating against individuals based on a  disability. Federal statutes have also authorized the Department of  Transportation (DOT) to regulate certain areas affecting passengers. For  example, DOT has the authority to stop airlines from engaging in unfair or  deceptive practices, or unfair methods of competition, and may  promulgate consumer protection regulations under that authority. In  particular, under that authority, among others, DOT has issued three final  rules on Enhancing Airline Passenger Protections since 2009, with the  most recent rule issued in November 2016. These rules have, among  other things, restricted long tarmac delays, significantly increased  compensation amounts for passengers who are involuntarily denied  boarding, and required certain airlines to post information about their fees  and on-time performance on their websites. Despite these protections,  some congressional members have questioned if additional consumer  protections are needed and whether passengers know about their  existing rights. More recently, in October 2018, the FAA Reauthorization  Act of 2018 was enacted and includes a number of consumer protection  provisions.

You asked us to examine airline consumer protection issues. This report: describes trends in DOT’s data on airline service from 2008 through  2017 and airlines’ actions to improve service, assesses how effectively DOT ensures airlines’ compliance with  consumer protection requirements, and assesses the extent to which DOT’s airline passenger education  efforts align with key practices for consumer outreach.

At your request, we also examined the relationship between airline  competition and airline customer service (see app. I).

The scope of this report focused on airline consumer protection issues  overseen by DOT. To describe trends in airline service, we analyzed  DOT’s operational data—including data on late flights and cancellations,  denied boardings, and mishandled baggage—and passenger complaints  submitted to DOT from 2008 through 2017. We limited our analysis of  passenger complaints to “selected” airlines that were required to report  operational data to DOT in 2017—the most recent year of available data  when we started our review—because they were the largest U.S.  domestic passenger airlines in 2016. To assess the reliability of DOT’s  data, we reviewed DOT reports, analyzed data to identify any outliers,  and interviewed DOT program officials about how the data were collected  and used. We determined that the data were sufficiently reliable for our  purposes, including to present high-level trends in passenger service over  time. To understand airlines’ actions to enhance service from 2013  through 2017, we conducted a literature search of popular press articles  and interviewed or obtained written responses from representatives from  11 of the 12 selected airlines. We requested interviews with all selected  12 airlines; 11 airlines agreed to be interviewed or provided written  responses (Alaska Air Group representatives provided written responses  on behalf of both Alaska and Virgin America as the airlines have merged  since 2016), and 1 airline declined to be interviewed. Given the dynamic  nature of the industry, we asked about business practices and reviewed  passenger satisfaction data for the most recent 5-year period.

To determine how effectively DOT ensures airlines’ compliance with  consumer protection requirements, we conducted multiple interviews with  DOT officials and reviewed DOT’s data and documents on its compliance  activities from 2008 through 2017. We then evaluated selected DOT  compliance activities against principles of Standards for Internal Control  in the Federal Government related to DOT’s policies and procedures for  coding passenger complaints, case management system, and  performance measures. We also compared DOT’s use of performance  measures against other leading practices for successful performance  measures.

To understand the extent to which DOT’s passenger education efforts  align with key practices for consumer outreach, we assessed DOT’s  efforts to disseminate educational materials against key practices we  identified in prior work. We believe the key practices we identified in  2007 remain relevant today since the practices are not time-sensitive.  DOT officials agreed that these key practices were relevant to conducting  consumer outreach. See appendix II for more information on our scope  and methodology.

We conducted this performance audit from September 2017 to November  2018 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
subsections: section_title: Background
subsections: section_title: State of the Airline Industry
paragraphs: In the U.S. commercial airline industry, passengers travel by air on  network, low-cost, and regional airlines. With thousands of employees  and hundreds of aircraft, network airlines support large, complex hub- and-spoke operations, which provide service at various fare levels to  many destinations. Low-cost airlines generally operate less costly point- to-point service using fewer types of aircraft. Regional airlines typically  operate small aircraft—turboprops or regional jets with up to 100 seats—and generally provide service to smaller communities on behalf of  network airlines.

The U.S. airline industry’s financial health has improved greatly in recent  years due in part to increased demand for air travel as a result of the  improved economy, industry reorganization, and changes in business  practices. Starting in 2007, airlines faced a number of major challenges,  including volatile fuel prices, the financial crisis, and the ensuing  recession of 2007–2009. These events led to a wave of domestic airline  bankruptcies, five airline mergers, and changes in airlines’ business  practices. In all, these circumstances—such as the improved economy  and new airline business practices—contributed to record level profits for  airlines. For example, in 2017, U.S. airlines reported an after-tax net profit  of $13.4 billion for domestic operations, according to DOT data.

As the industry recovered from the recession and passenger traffic began  to rebound, airlines began to exercise “capacity restraint” by carefully  controlling the number of seats on flights to achieve higher load factors in  order to control costs and improve profitability. Because capacity  restraint may result in fewer empty seats on many flights, this practice  also limits airlines’ ability to rebook passengers if a flight is delayed or  cancelled. Airlines have also made changes in their ticket pricing. For  example, airlines now generally “unbundle” optional services from the  base ticket price and charge ancillary fees for those services.  Unbundling may result in passengers paying for services that were  previously included in the price of the ticket. Additionally, certain aspects  of customer service quality are tied to the class of ticket passengers  purchase. For example, purchasing a “basic economy” ticket may  include restrictions, such as not allowing passengers to select seats or  charging for carry-on bags, that would not apply to a higher priced ticket  class. Similarly, the quality of seating varies based on the ticket class  purchased—even within the main cabin of the aircraft. Moreover, while  the recent airline mergers have resulted in some new service options for  passengers in certain markets, they have also reduced consumers’  choice of airlines on some routes and can result in higher ticket prices. At  the same time, low-cost airlines provide greater competition in the  markets they serve, which may help to keep prices in check.

section_title: Factors That Affect Passengers’ Satisfaction with Service
paragraphs: Many factors—from booking a flight through collecting checked  baggage—may contribute to passengers’ level of satisfaction with an  airline’s service, according to an airline industry association and market  research organizations (see fig.1). For example, one industry survey  found that passengers most valued affordable airfare, convenient flight  schedules, and reliable on-time departures and arrivals.

section_title: DOT’s Regulatory, Compliance, and Education Efforts
paragraphs: DOT’s regulatory activities include issuing consumer protection  regulations. Specifically, DOT may issue or amend consumer protection  regulations under its statutory authority to prohibit unfair or deceptive  practices, or unfair methods of competition by airlines, among others.  As mentioned previously, under this authority DOT has promulgated  various regulations to enhance airline consumer protections since 2009  (see table 1).

When regulations are promulgated, agency officials must determine how  to promote compliance and deter noncompliance. Agencies charged  with promoting regulatory compliance, including DOT, usually adopt a  program that consists of two types of activities: those that encourage  compliance and those that enforce the regulations. Compliance  assistance helps regulated entities, such as U.S. airlines, understand and  meet regulatory requirements, whereas activities such as monitoring,  enforcement, and data reporting deter noncompliance and ensure that  entities follow requirements. Agencies choose a mix of compliance  activities that will achieve their desired regulatory outcome.

DOT promotes airlines’ compliance with consumer protection  requirements through a number of activities, and it educates passengers  on their rights. For example, DOT has the authority to investigate whether  an airline has been, or is engaged, in an unfair or deceptive practice or an  unfair method of competition in air transportation or the sale of air  transportation. If DOT finds that an airline has violated consumer  protection requirements, DOT may take enforcement action against the  airline by, for example, assessing civil penalties. In addition to  promoting airlines’ compliance with consumer protection requirements,  DOT also conducts activities aimed at educating passengers about their  rights and the services provided by airlines. For example, DOT has an aviation consumer protection website where it highlights passengers’  rights and describes how to file complaints with DOT, in addition to other  consumer resources. Within DOT’s Office of the Secretary (OST), the  Office of the Assistant General Counsel for Aviation Enforcement and  Proceedings and its Aviation Consumer Protection Division are  responsible for these efforts. According to DOT officials, the annual  appropriation to OST’s Office of the General Counsel provides funding for  DOT’s consumer protection activities, among other things. At the end of  fiscal year 2017, DOT employed 38 staff—including 18 attorneys and 15  analysts—to conduct these activities, according to DOT officials.

section_title: DOT’s Data Provide Mixed Information on Improvement in the Quality of Airline Service; Selected Airlines Indicate They Are Taking Steps Intended to Enhance Service DOT’s Data Provide Mixed Information on the Quality of Airline Services
paragraphs: DOT’s data, which include both operational measures of airline service,  as well as passenger complaints received by DOT, provide mixed  information on whether service improved from 2008 through 2017. DOT  requires reporting airlines to provide operational data, including  information on late, cancelled, or diverted flights; mishandled baggage;  and denied boardings. These data showed some general improvement  in the quality of airline service from 2008 through 2017. However, during  the same time period, the total number of passenger complaints filed with  DOT increased for “selected” airlines. Moreover, while these data may  be imperfect measures of service quality, they do provide some indication  of the passenger experience. DOT publishes data on both operational  performance and passengers’ complaints in its monthly Air Travel  Consumer Report to inform the public about the quality of services  provided by airlines.
subsections: section_title: Selected Airlines Indicate They Have Taken a Variety of Actions to Enhance Passenger Service
paragraphs: Representatives from all 11 selected airlines highlighted actions they took  to enhance passenger service since 2013, including in some of the areas  discussed above. While customer service is important for airlines, these  actions can also be motivated in part by other factors—including  compliance with certain consumer protection requirements or DOT  consent orders, or competition with other airlines. For example, one  airline developed a wheelchair tracking system in response to DOT  enforcement, which also contributed to the airline’s goal to improve its  services to passengers with disabilities. Additional examples of service  improvements are listed below.

On-time performance. Representatives we interviewed from almost  all selected airlines (10 of 11) reported taking actions intended to  improve on-time performance or mitigate challenges associated with  flight delays and cancellations. These actions varied across airlines  from those intended to improve operational performance to those  intended to improve the comfort of passengers. For example, one  airline began tracking flights that were “at-risk” of meeting DOT’s  definition of a chronically delayed flight, so it could, among other  things, swap crews or substitute aircraft and avoid these types of  delays. According to DOT regulations, airlines with a chronically  delayed flight for more than four consecutive one-month periods are  engaging in a form of unrealistic scheduling, which is an unfair or  deceptive practice and an unfair method of competition. Airlines  have also used technology, such as text-messaging updates, to  communicate with passengers during delays and cancellations (8 of  9); increased the number of situations where passengers are  compensated during delays and cancellations (5 of 9); and  empowered customer service agents to provide food, beverages, and  entertainment to passengers during flight delays (1 of 9). For  example, one airline e-mails all passengers that experience long  delays with an apology and voucher for future travel, regardless of  whether the delay was within the airline’s control. While DOT has  some requirements for airlines on delays and cancellations, such as  on tarmac delays and chronically delayed flights, it generally does not  require airlines to compensate passengers for delays.

Baggage handling. Representatives we interviewed from almost all  network and low-cost airlines (8 of 9) reported investing resources in  order to improve baggage-handling efforts and minimize the effects to  passengers whose bags are lost or delayed. Among other things,  airlines upgraded baggage technology (5 of 9); modernized the claims  process, so passengers could complete forms on-line (3 of 9); and  instituted replacement baggage programs, where passengers get a  replacement bag at the airport (2 of 9). For example, one airline  invested several million dollars to use radio frequency identification  technology (RFID) to track bags, as well as allowing passengers to  track their baggage via an application on their smartphone. Another  airline introduced a policy to use FedEx to deliver delayed bags if the  airline cannot return them within 24 hours. Since 2011, DOT has  required certain airlines to make every reasonable effort to return  mishandled baggage within 24 hours.

Quality of interaction with airline staff. Representatives we  interviewed from almost all selected airlines (10 of 11) reported  improving training programs in an attempt to enhance interactions  between airline staff and passengers. For example, one airline worked  with the Disney Institute to provide training to staff on relating to  guests during travel disruptions and de-escalating conflict. While  airlines have increased customer service training, representatives  from one industry association said that the training would be more  beneficial if it was provided on a more regular basis. Two airlines also  expanded their customer service departments’ hours to better match  when passengers travel. According to DOT officials, airlines are not  required to provide customer service training to staff.

Passengers with disabilities. Representatives we interviewed from  almost all network and low-cost airlines (8 of 9) reported taking  actions intended to improve services for passengers with disabilities.  These actions included programs to replace damaged or misplaced  wheelchairs or other assistive devices (3 of 9); improving seating and  access to lavatories in the aircraft (1 of 9); and using RFID technology  to track wheelchairs (1 of 9). For example, representatives from one  airline told us they have retrofitted their larger single aisle aircraft  lavatories to be wheelchair accessible. Two airlines also reported  changing policies pertaining to emotional support animals. For  example, one airline has an online registration for emotional support  animals where passengers must submit all documentation at least 48  hours in advance of the flight; according to representatives, the  process allows the airline to validate the required paperwork, while  providing relevant information to passengers with emotional support  animals and ensuring the safety of everyone onboard the aircraft.

Involuntary denied boardings. Representatives we interviewed from  network and low-cost airlines (9) reported taking steps to reduce or  eliminate involuntary denied boardings. Representatives from three  airlines said they have reduced or stopped overbooking flights, and  other representatives (5 of 9) said their airlines have begun soliciting  volunteers to be “bumped” off a flight (i.e., give up their seat) earlier in  the process. Two conduct reverse auctions where they ask  passengers what compensation they would accept to take an  alternative flight. Airlines are also offering additional incentives to  encourage passengers to voluntarily switch to flights with available  seats (5 of 9)—including travel vouchers with fewer restrictions or that  cover ancillary fees, gift cards for Amazon and other retailers, or large  travel credits of up to $10,000.

section_title: DOT Conducts Multiple Activities to Monitor Airline Compliance, but Opportunities Exist to Improve These Efforts’ Effectiveness
paragraphs: DOT promotes and monitors airlines’ compliance with consumer  protection requirements and deters noncompliance in five key ways, such  as by reviewing passenger complaint data and taking enforcement action  where it identifies violations. However, we found that DOT could improve  its procedures to provide additional assurances that analysts consistently  code passengers’ complaints and properly identify potential consumer  protection violations, in addition to more fully utilizing data from DOT’s  information systems to inform its compliance program. Further, while DOT  has objectives for each of its five key compliance activities, it lacks  performance measures for three of these objectives. As a result, DOT is  limited in its ability to assess progress toward achieving its goal of  promoting airlines’ compliance with consumer protection requirements or  to identify and make any needed improvements.
subsections: section_title: DOT Conducts Five Key Compliance Activities
paragraphs: DOT conducts five key activities to help airlines understand and comply  with consumer protection requirements: (1) providing compliance  assistance to airlines, (2) processing complaints from passengers, (3)  conducting compliance inspections of airlines at headquarters and  airports, (4) conducting airline investigations, and (5) enforcing airlines’  compliance with consumer protection requirements. Collectively, these  key compliance activities are intended to help airlines understand and  meet consumer protection requirements and deter noncompliance.

Providing compliance information to airlines. DOT attorneys assist  airlines in meeting consumer protection requirements by developing  guidance materials and responding to questions. DOT publishes  these materials—such as topic-specific webpages and frequently  asked questions—on its website. Attorneys and analysts also  informally respond to questions or requests for information from airline  representatives.

Processing complaints from passengers. As previously stated,  passengers may file complaints with DOT via its website, by mail, or  through DOT’s telephone hotline. DOT analysts use a web  application—the Consumer Complaints Application system—to  receive, code, and track passenger complaints. In 2017, DOT’s 15  analysts processed about 18,000 air travel-related complaints. Initial  processing involves reviewing the information in the complaint,  notifying complainants that their complaint was received, and  transmitting the complaint to the relevant airline for action. Analysts  assign one of 15 high-level complaint category codes (e.g.,  “advertising” or “discrimination”) to each complaint as well as more  specific lower-level complaint codes and codes indicating a potential  violation of consumer protection requirements as necessary. Analysts  initially code a complaint based on the passenger’s perception of  events and not on an assessment of whether the complaint is a  potential violation of consumer protections. According to DOT officials,  when initially coding passenger complaints, analysts generally use  their judgment to code each passenger’s complaint based on the  primary issue. While analysts handle a variety of complaints, DOT  may designate specific analysts to handle more complex complaint  codes, such as disability complaints. On a monthly basis, DOT  provides airlines the opportunity to review the complaints received  and the agency’s categorization of each complaint. At that time,  airlines have an opportunity to challenge DOT’s categorizations.  According to DOT officials, a limited number of complaints are  recoded as a result of this process.

Conducting compliance inspections of airlines at headquarters  and airports. DOT analysts and attorneys inspect airlines at airline  headquarters and airports to assess their compliance with consumer  protection requirements. From 2008 through 2016, analysts and  attorneys conducted compliance inspections of airlines at the airlines’  headquarters, but DOT has not conducted any such inspections since  September 2016. Beginning in 2015, DOT initiated compliance  inspections of airlines at airports, and DOT continued to conduct  these inspections through 2018. According to DOT officials, they have  exclusively conducted on-site inspections of airlines at airports in  recent years due, in part, to limited resources and budget  unpredictability. However, officials stated that they would consider  conducting more inspections of airlines at airline headquarters in the  future.

Inspections of airlines at airlines’ headquarters examine customer  service policies and passenger complaints received directly by  airlines, among other things. According to DOT officials, these  inspections represent a “deep dive” into an airline’s relevant  policies and involve collecting and analyzing data prior to and after  their weeklong visit, as well as interviewing corporate personnel.  DOT analysts and attorneys use the agency’s inspection checklist  to assess compliance with a variety of regulated areas such as the  inclusion of certain information on the airline’s website and the  proper reporting of data to DOT (e.g., mishandled baggage and  on-time performance data). According to DOT data, between 2008  and 2016 DOT completed inspections at 33 U.S. airlines’  headquarters. These 33 inspections identified 23 systemic  violations, resulting in consent orders. Two inspections resulted  in warning letters, and eight did not identify any systemic  violations. The assessed penalty amounts for these inspections  ranged from $40,000 to $1,200,000.

Inspections of airlines at airports examine staff’s knowledge of  certain consumer protection requirements and the availability and  accuracy of signage and documentation. Such inspections provide  DOT the opportunity to examine multiple airlines in one visit.  According to DOT officials, during these unannounced  inspections, attorneys and analysts focus on assessing  compliance through observation and interviews with randomly  selected airline employees. For example, analysts and attorneys  may confirm the availability of information on compensation for  denied boarding from an airline gate agent or review an airline’s  required signage on compensation for mishandled baggage to  determine whether the information is accurate. According to DOT  data, DOT inspected 12 to 14 U.S. airlines annually—most  multiple times—at 51 domestic airports from 2015 through 2017.  In 2017, DOT conducted inspections at 18 domestic airports that  included inspecting 12 U.S. airlines multiple times. In total, from  2015 through 2017, DOT found violations of various consumer  protection requirements for 13 airlines that DOT addressed  through warning letters. In addition, DOT found violations related  to incorrect (e.g., out-of-date) or missing notices regarding  baggage liability limits or oversales compensation for 8 airlines  that were settled by consent orders with penalties between  $35,000 and $50,000.

Conducting airline investigations. According to DOT officials,  attorneys determine whether to open an investigation by weighing  numerous factors, including whether they believe an airline is  systematically violating consumer protection requirements. Attorneys  may initiate an investigation based on findings from trends in  passenger complaints, compliance inspections, monitoring of airline  websites and news media, or information supplied by other entities,  including other DOT offices or governmental agencies. According to  DOT officials, after gathering preliminary information, an attorney may  notify the airline of his or her investigation, request information for  further analysis, and then determine whether a violation has occurred  and which enforcement action, if any, is appropriate. Attorneys  document these investigations using DOT’s case management  system. From 2008 to 2017, DOT initiated almost 2,500  investigations as shown in table 2 below.

Enforcing airlines’ compliance with consumer protection  requirements. When investigations result in a determination that a  violation occurred, DOT may pursue enforcement action against the  airline by, for example; (1) seeking corrective actions through warning  letters; (2) consent orders (which may include fines); or (3)  commencement of a legal action (see table 2). According to DOT  officials, attorneys consider a number of factors in determining the  appropriate enforcement action, including whether there is evidence  of recidivism or systemic misconduct, and the number of passengers  affected. According to DOT data, most investigations result in  administrative closures and findings of no violation.

According to DOT officials, when attorneys decide to issue a consent  order, they work with their managers to arrive at an initial civil penalty  level and then negotiate with the airline to arrive at a final settlement  agreement and civil penalty amount if applicable. DOT has criteria for  setting civil penalties, but officials describe the process as “more art than  science” because facts and circumstances always vary. Civil penalties  assessed in consent orders often include three parts: mandatory  penalties, credits, and potential future penalties (see table 3). A  mandatory penalty is the portion of the assessed penalty that must be  paid immediately or in installments over a specified period of time. A  credit is the portion of the assessed penalty that DOT allows an airline to  not pay in order to give credit to the airline for spending funds on  passenger compensation or toward specific service improvements, both  of which must be above and beyond what is required by existing  requirements. A potential future penalty is the portion of the assessed  penalty that the airline will pay if DOT determines that the airline violated  certain requirements during a specified period of time.

Our review of 76 consent orders for our 12 selected airlines where a  penalty was assessed found that DOT issued penalties totaling  $17,967,000 from 2008 through 2017. Of this, 47 percent ($8,437,700)  comprised mandatory penalties paid by the airline. The remaining  amounts were either credits or potential future penalties. According to  DOT officials, credits are a better way to effect positive change than  merely assessing a mandatory penalty. For example, one recent consent  order included violations of regulations regarding assistance for  passengers with disabilities, among other things. The airline and DOT  agreed to an assessed civil penalty amount of $400,000, $75,000 of  which was credited to the airline for compensation to customers filing disability-related complaints in certain years and for implementation of an  application to provide real-time information and response capabilities to a  wheelchair dispatch and tracking system, among other things. However,  our review found that consent orders do not always ensure future  compliance. Specifically, we found 14 instances where an airline received  multiple consent orders for the same regulatory violation. Three of these  instances—each for different airlines—related to violations of the “full fare  rule,” and two—also for different airlines—related to airlines’ failure to  adhere to customer service plans.

section_title: Improvements to DOT’s Procedures Could Provide Greater Assurance That Passengers’ Complaints Are Consistently Coded and that Consumer Protection Violations are Properly Identified
paragraphs: We found that while DOT has some procedures (i.e., guidance  documents and on-the-job training) in place for coding passenger  complaints, it lacks others that could help ensure that analysts  consistently code complaints and that potential consumer protection  violations are properly identified. Federal internal control standards state  that agencies should design control activities to achieve objectives and  establish and operate monitoring activities to evaluate results. By  designing and assessing control activities, such as procedures and  training, agencies are able to provide management with assurance that  the program achieves its objectives, which in this case involve identifying  instances of airline noncompliance.

DOT has taken some steps to help analysts code passenger complaints  and properly identify potential violations of consumer protection  requirements:

Guidance documents. DOT developed two documents to guide  complaint processing and evaluation—a coding sheet that helps  analysts determine how to code complaints and identify potential  consumer protection violations, and a user guide that describes how  analysts should enter complaint information into the web application.  However, we found that these documents may not be clear or specific  enough to ensure that analysts consistently coded complaints or  properly identified potential consumer protection violations. For  example, while the coding sheet includes explanatory notes in 9 of the  15 complaint categories, it does not include definitions and examples  for each of DOT’s 15 complaint categories that would illustrate  appropriate use of a complaint code, a gap that could result in  inconsistent coding.

On-the-job training. DOT supplements its guidance documents with  on-the-job training, which officials told us helps analysts consistently  code complaints and identify potential consumer protection violations;  however, DOT has not established formal training materials to ensure  all new analysts get the same information. DOT pairs each newly- hired analyst with a senior analyst to be their coach and instruct them  on how to code complaints. According to DOT officials, senior and  supervisory analysts determine when new analysts are able to code  and work independently but continue to monitor their work as needed  and determined by the senior analyst. DOT officials stated that while  the agency does not regularly check the extent to which complaints  are consistently coded, supervisory analysts check analysts’  complaint coding on an as-needed basis throughout the year, as well  as during semi-annual performance reviews. However, DOT does not  provide formal training materials or other guidance to ensure that  senior analysts are conveying the same information during these  informal, on-the-job training sessions.

DOT officials stated that the combination of the existing guidance,  procedures, and hands-on training provides adequate assurance that  analysts share a common understanding of the complaint categories  resulting in complaints being consistently coded. As a result, DOT officials  have not developed additional guidance documents or established formal  training materials.

While DOT officials said they believe their procedures and on-the-job  training are sufficient to ensure that complaints are consistently coded  and that potential consumer protection violations are properly identified, a  recent DOT Office of Inspector General (OIG) report found that DOT  analysts did not identify when to code complaints as potential consumer  protection violations for a sample of frequent flyer complaints the agency  reviewed. As a result, in 2016, the DOT OIG recommended that DOT  provide additional training on what constitutes an unfair or deceptive  practice to strengthen oversight of airlines’ frequent flyer programs. In  response, DOT created a special team to process frequent flyer  complaints and developed and provided review team analysts and other  members with training on how to review complaints and identify potential  violations related to airlines’ frequent flyer programs.

Improving DOT’s procedures that analysts use to code complaints and  identify potential consumer protection violations could provide DOT with  additional assurances that analysts: share a common understanding of  the definitions of all the complaint codes, are coding complaints in each  category consistently, and are identifying potential consumer protection  violations. Consistent coding among analysts is important for a number of  reasons. First, according to DOT officials, passengers use complaint  data—which are publicly reported in DOT’s Air Travel Consumer  Report—to make decisions about air travel, including which airlines to fly.  Second, DOT analysts and attorneys use complaint data to guide their  compliance activities (e.g., selecting airlines for inspections and  investigations, and determining proper enforcement actions).

section_title: DOT Is Missing Opportunities to Use Its Case Management System to Help Inform Its Compliance Program
paragraphs: We found that while DOT’s case management system allows attorneys to  track investigations, it lacks functionality that would allow DOT officials to  more efficiently use data from the system to inform other key activities,  such as making compliance and enforcement decisions. Federal  internal control principles state that agencies should design an entity’s  information system and related control activities to achieve objectives and  respond to risks, which in this case involve using data from DOT’s case  management system to inform its compliance activities.

Our review of DOT’s case management system identified the following  limitations that affect DOT’s ability to use data from its case management  system to target resources and accurately monitor trends in violations,  compliance activities, and the results of its enforcement actions:

Key data are optional. Attorneys are not required to complete certain  key data fields in the case management system. For example,  attorneys are not required to document the outcome of an  investigation in the “enforcement action” field. According to officials,  while attorneys do not always complete this field, they often choose to  document the outcome of investigations in the case notes. Even if that  information is captured in the case notes section, attorneys can only  access that information by individually reviewing each case file.

Data entries are limited. Attorneys cannot record multiple consumer  protection violations for a single investigation in the case management  system. As a result, when multiple violations occur, attorneys must  use their professional judgement to select the primary violation to  record. Our review of the 76 consent orders against selected airlines  resulting from airline investigations identified 24 instances—or more  than 30 percent—where an airline violated multiple consumer  protection regulations. While this is a small subset of all investigations  (2,464) DOT completed across our timeframe, it suggests  investigations could include violations of multiple consumer protection  regulations.

Data entries do not reflect DOT’s compliance activities. While the  case management system includes a field for attorneys to document  the source of investigations, the field’s response options do not fully  correspond to DOT’s key compliance activities or align to DOT’s  documentation listing the sources of investigations. For example, the  field that tracks the source of an investigation includes an option to  identify passenger complaints as the source but not an inspection of  an airline. Officials told us that, like the outcomes of investigations,  attorneys often document the source of an investigation in the case  notes. However, as mentioned previously, information captured in the  case notes section can only be accessed by individually reviewing  each case file.

Limited reporting capabilities exist. Attorneys are limited in their  ability to run reports to understand trends across multiple  investigations, according to DOT officials. For example, the case  management system lacks a function to run reports by certain data  fields. Specifically, according to DOT officials, attorneys cannot run  reports by the airline name data field and must instead type in the  airline name to create a report, a process that could produce  unreliable results if an airline’s name is inconsistently entered into the  database.

According to DOT officials, the case management system’s capabilities  are limited largely because the database was designed as a mechanism  for attorneys to manage ongoing investigations. DOT officials told us that,  while the database has successfully fulfilled that role, officials have  increasingly used data from the case management system to make  enforcement decisions. For example, DOT attorneys use information from  the case management system to inform civil penalty amounts. In addition,  DOT uses data from the case management system to analyze the results  of investigations and inspections, as well as the details of consent orders  in order to target future compliance activities. However, because of  limited reporting capabilities, attorneys and managers must manually  create summary documents from the case management system’s data,  work that could be time consuming and subject to manual errors, and that  does not address the issue that some data are not entered into various  data fields in the first place.

Recognizing limitations with the case management system, DOT has  taken steps to improve the system. Specifically, starting in June 2018,  DOT began working with a contractor to update the case management  system’s functionality. Among other things, the updates are intended to  improve the system’s ability to run reports, which could enhance DOT’s  ability to examine trends in enforcement actions and penalty amounts,  and allow the system to track investigation milestones. While DOT’s  planned updates may help DOT officials better examine trends in  enforcement actions, the planned updates do not fully address the issues  we identified above, particularly related to collecting complete data.  Collecting complete and comprehensive data in the case management  system could allow DOT to better track trends in its investigations,  inspections, and enforcement actions and to use that information to make  data-driven decisions about future compliance activities and enforcement  actions.

section_title: DOT Lacks Performance Measures for Three of Five of Its Compliance Program Objectives
paragraphs: While DOT has five objectives for its key compliance program activities, it  has not established performance measures for three of these objectives.  Objectives communicate what results the agency seeks to achieve, and  performance measures show the progress the agency is making toward  achieving those objectives. Federal internal control standards state that  agencies should define objectives clearly to enable the identification of  risks and define risk tolerances. They further state that management  defines objectives in measurable terms, so that performance toward  those objectives can be assessed. Additionally, the Government  Performance and Results Act of 1993 (GPRA), as enhanced by the  GPRA Modernization Act of 2010, requires agencies to develop objective,  measurable, and quantifiable performance goals and related measures  and to report progress in performance reports in order to promote public  and congressional oversight, as well as to improve agency program  performance.

In fiscal years 2017 and 2018, DOT developed objectives for each of its  five key compliance activities; however, as illustrated in table 4 below,  DOT does not have performance measures for three of its objectives.

For the three objectives for which DOT has not established performance  measures, it has documented qualitative measures in internal agency  documents. For example, while DOT has not developed a performance  measure related to enforcing airlines’ compliance with consumer  protection requirements, it summarized enforcement cases in fiscal year  2017 that illustrated actions the agency had taken to achieve this  objective. For instance, one enforcement action included a consent order  against an airline with an assessed penalty of $1.6 million for violating  DOT’s tarmac delay rule. DOT highlighted similar accomplishments for  educating airlines and conducting inspections. For example, DOT issued  guidance to help airlines understand their legal obligations to not  discriminate against passengers in air travel on the basis of race, color,  national origin, religion, sex or ancestry, and the agency highlighted  identifying unlawful practices by multiple airlines during an inspection of  airlines at an airport. While the actions described may provide DOT with  some information on whether it is achieving its objectives, they fall short  of internal control standards that call for federal agencies to define  objectives in measureable terms to assess performance.

DOT officials stated that they have not developed performance measures  to monitor progress toward achievement of some objectives because it is  difficult to develop quantifiable performance measures. We have  previously reported that officials from other enforcement agencies with  similar objectives found it challenging to develop performance measures  in part due to the reactive nature of enforcement as well as the difficultly  of quantifying deterrence, but were ultimately able to do so. Developing  performance measures for all objectives would allow DOT to more fully  assess the effectiveness of its efforts at promoting airlines’ compliance  with consumer protection requirements. Specifically:

Providing compliance information to airlines. DOT has not  developed quantifiable performance measures to assess how well  DOT educates airlines about consumer protection requirements. For  example, DOT does not have a performance measure for developing  and disseminating guidance for specific rules or to issue information  on new rules within a certain time frame. Rather, officials told us that  they proactively e-mail stakeholders new consumer protection rules— rather than relying on stakeholders having to find them on DOT’s  website or Regulations.gov—and if officials receive the same question  repeatedly, about the same requirement they might issue guidance on  the topic. According to DOT officials, these activities help ensure that  stakeholders are complying with relevant consumer protection  requirements. DOT officials did not provide a specific reason for why  they do not have a performance measure related to this objective.  However, without such a measure, DOT cannot be sure that it is  providing timely educational materials to clarify new consumer  protection requirements and assist airlines in complying with these  requirements.

Conducting compliance inspections of airlines at headquarters  and airports. DOT lacks quantifiable performance measures related  to conducting inspections of airlines at airlines’ headquarters and at  airports. Having such a measure could help ensure that DOT  conducts these activities. Specifically, we found that while DOT  continues to conduct inspections of airlines at airports, it has not  conducted inspections at airlines’ headquarters since 2016, despite  having identified this compliance activity as a key priority in planning  documents. According to DOT officials, they have not conducted  inspections at airlines’ headquarters for two primary reasons. First,  DOT officials said inspections at airlines’ headquarters require  significant staff resources, which DOT has allocated to other  compliance activities in recent years. Second, officials said that no  airline was an obvious choice for an inspection at its headquarters  because DOT had not received a disproportionate number of  complaints against a specific airline to suggest an inspection was  warranted. However, the DOT OIG previously directed the agency to  make these inspections a priority and to allocate resources  accordingly, and DOT officials themselves have said that these  inspections provide incentives for airlines’ continued compliance  regardless of whether one airline has an obvious problem.  Establishing performance measures for conducting both types of  inspections would provide greater assurance that DOT conducts  these activities on a regular basis.

Moreover, officials told us that inspections at airlines’ headquarters  examine specific consumer protection requirements that are not  examined during inspections at airports, and that inspections at  headquarters help promote compliance. Among other things,  inspections at airlines headquarters allow DOT officials to: (1) review  training manuals and training records; (2) examine a sample of  passengers’ complaint data received directly by the airlines, including  disability and discrimination complaints; and (3) verify that airlines are  current on reporting data such as on mishandled baggage and denied  boardings to DOT. Performance measures related to how often and  under what circumstances compliance inspections should take place  could provide assurance that DOT conducts these activities, and is  not missing opportunities to monitor airlines’ compliance with  consumer protection requirements.

Enforcing airlines’ compliance with consumer protections. DOT  officials told us that they have not developed performance measures  for enforcement actions because they would not want to have  performance measures that were punitive or reactive by, for example,  requiring the agency to collect a certain penalty amount from airlines.  While we acknowledge the complexity and risks involved in setting  these types of performance measures, as mentioned previously, other  agencies have done so. For example, one of the Federal Trade  Commission’s performance measures is to focus 80 percent of  enforcement actions on consumer complaints. Without a performance  measure for enforcement activities, DOT is missing opportunities to  assess the effectiveness of these activities and make any needed  changes. We have previously reported that performance  measurement gives managers crucial information to identify gaps in  program performance and plan any needed improvements.

section_title: DOT Has Made Recent Improvements, but Its Passenger Education Efforts Do Not Fully Align with Key Consumer Outreach Practices
subsections: section_title: DOT Updated Its Website to More Effectively Educate Passengers on their Rights
paragraphs: DOT’s primary vehicle for educating passengers is its aviation consumer  protection website, which it relaunched in November 2017 (see fig. 3).  According to DOT officials, as part of the relaunch, DOT improved the  navigability and accessibility of the website by, among other things,  arranging material by topic, adding icons for various subjects, and  including a link for the website on DOT’s aviation homepage. The  website now includes summaries of passengers’ rights on a number of  issues including tarmac delays, overbookings, mishandled baggage, and  disability issues, as well as DOT’s rules, guidance issued to airlines and  others, and enforcement orders on key consumer protection issues.  Moreover, the website is now accessible to people with disabilities.  Moving forward, DOT has a number of additional updates planned  through fiscal year 2019. For example, DOT plans to update its website  with information on frequent flyer issues, optional services and fees, and  codeshare agreements by the end of calendar year 2018. According to  DOT officials, while not statutorily required to conduct these education  activities, passenger education is a key effort to ensuring airlines’  compliance.

DOT also has numerous other efforts to educate passengers on their  rights. For example:

Establishing resources for passengers. DOT developed Fly  Rights—an online brochure that details how passengers can avoid  common travel problems—in addition to material on unaccompanied  minors, family seating, and a glossary of common air travel terms.  DOT also developed training tools (e.g., brochures, digital content,  and videos) on the rights of passengers with disabilities under the Air  Carrier Access Act of 1986 and its implementing regulations, including  wheelchair assistance at airports and onboard aircraft, traveling with a  service animal, and traveling with assistive devices. While some of  these materials were developed primarily for airline employees and  contractor staff, others were developed to directly assist passengers  with disabilities by providing helpful tips on airlines’ responsibilities,  according to DOT officials.

Building consumer education information into existing  regulations. Passenger education is built into certain consumer  protection requirements, according to DOT officials. For example,  when an airline involuntarily denies a passenger boarding,  immediately after the denied boarding occurs the airline must provide  a written statement explaining the terms, conditions, and limitations of  denied boarding compensation, and describing the airline’s boarding  priority rules and criteria.

Responding to complaints. DOT officials said they include  information on an airline’s responsibilities when responding to  passenger complaints. For example, if a passenger submits a  complaint to DOT about not receiving compensation for a delayed or  cancelled flight, the DOT analyst may inform the passenger that  airlines are generally not required to compensate passengers in these  instances.

section_title: DOT’s Educational Efforts Fully Align with Five of Nine Key Practices
paragraphs: We compared DOT’s efforts to educate airline passengers about their  rights against key practices for consumer outreach GAO identified in prior  work and found that DOT’s efforts fully align with five of the nine key  practices (see fig. 4). For example, we found that DOT has successfully  identified the goals and objectives of its passenger education program  and identified the appropriate media mix for disseminating its materials.  Similarly, we found that DOT had identified and engaged stakeholders, a  step that, according to DOT officials, allowed them to better tailor  materials.

However, as summarized in the figure below, we found that DOT only  partially met or did not meet the remaining four key practices.

For example, DOT’s actions do not align with the key practice to “identify  resources” and only partially align with the key practice to “develop  consistent, clear messages” based on the established budget. According  to a senior DOT official, DOT has not identified budgetary resources  because, while important, DOT’s educational efforts are secondary to the  office’s other efforts. Further, officials said that it has been difficult for the  agency to develop a budget when it has been operating under a  continuing resolution for some part of the fiscal year for the last decade.  However, without identifying short- and long-term budgetary resources  and planning activities accordingly, DOT is missing an opportunity to plan  educational efforts or prioritize needs based on available resources.

In addition, we found DOT’s efforts only partially align with the key  practice that calls for an agency to research its target audience. While  DOT has solicited some input from stakeholder groups such as those  representing passengers with disabilities, DOT has not solicited feedback  directly from passengers to understand what they know about their rights.  DOT officials said they have not sought such feedback because they  have not identified a method for doing so that would be statistically  generalizable and not cost prohibitive. While costs are always an issue  when considering budget priorities, we have previously reported on other  agencies’ direct consumer outreach efforts that while not statistically  generalizable were nonetheless useful for understanding the effect of the  agencies’ efforts. For example, the Bureau of Consumer Financial  Protection has used focus groups to understand its outreach efforts.  Bureau of Consumer Financial Protection officials previously told GAO  that while obtaining information through such efforts was resource  intensive, it allowed them to assess the performance of their outreach  activities. In another case, an agency surveyed users that access its  website to help it understand whether its outreach efforts were effective.  Obtaining input from passengers directly on what information they want or  what they know about their rights would provide DOT with greater  assurance that educational materials are appropriately tailored to meet a  wide range of passengers’ needs.

Finally, DOT has not established performance measures to understand  the quality of its passenger education materials (i.e., process measures)  or the effectiveness of its efforts (i.e., outcome measures). DOT officials  said that they receive informal input from stakeholders on the quality of  the materials and track website traffic to understand whether materials  are reaching passengers. Officials said they believe that these  mechanisms provide them with some assurance that the materials are  meeting passengers’ needs and that passengers are accessing and using  the materials. While these mechanisms may provide DOT with some  information on how often materials are accessed online, they do not help  it understand the quality of the materials and measure the success of its  passenger education efforts. For example, while DOT officials track  website traffic, they have not established a related performance measure.  A number of different measures could be used to track processes and  outcomes related to the use of its website, including the time consumers  spend on the website, number of website pages viewed, bounce rate (i.e.,  percentage of visitors who looked at only one page and immediately left  the site), or user’s perception of the experience of their visit.  Establishing such measures would provide DOT with greater assurances  that its educational efforts are appropriately tailored to passengers and  leading to improved understanding of passengers’ rights, including  whether any adjustments are needed.

section_title: Conclusions
paragraphs: To enforce consumer protection requirements, such as those preventing  unfair or deceptive practices or unfair methods of competition by airlines,  DOT has conducted almost 2,500 investigations and issued about 400  consent orders over the last decade. However, DOT lacks reasonable  assurance that its approach is achieving the highest level of airlines’  compliance, given its available resources. For example, DOT has not  assessed whether its procedures and training materials help analysts  consistently code passengers’ complaints and identify potential consumer  protection violations. Additionally, DOT has not fully used data from its  case management system to inform its compliance program. Moreover, in  the absence of comprehensive performance measures, DOT lacks a full  understanding of the extent to which it is achieving its goal of airlines’  compliance with consumer protection requirements and whether any  programmatic changes may be warranted. Improvements in these areas  would provide DOT with additional information to target its resources and  improve compliance.

DOT has taken positive steps to educate passengers about their rights— through its revamped website and other educational resources.  Nevertheless, DOT could improve its efforts by more fully following key  practices GAO previously identified for conducting consumer education,  such as by:  seeking feedback directly from consumers;  identifying short- and long-term budget resources; and establishing performance measures.

Taking such actions would provide DOT with greater assurance that its  efforts are meeting passengers’ needs.

section_title: Recommendations for Executive Action
paragraphs: We are making the following six recommendations to DOT:

The Office of the Secretary should assess its procedures and training  materials for coding airline passengers’ complaints, as appropriate, to  help ensure that passengers’ complaints are consistently coded and  that potential consumer protection violations are properly identified.  (Recommendation 1)

The Office of the Secretary should assess the feasibility and cost of  updating its airline case management system to address data and  reporting limitations, and to undertake those updates that are cost  effective and feasible. (Recommendation 2)

The Office of the Secretary should establish performance measures  for each of its objectives for its five key airline-compliance activities.  (Recommendation 3)

The Office of the Secretary should capture feedback directly from  airline passengers or identify other mechanisms to capture  passengers’ perspectives to inform DOT’s education efforts.  (Recommendation 4)

The Office of the Secretary should identify available short- and long- term budgetary resources for DOT’s airline-passenger education  efforts. (Recommendation 5)

The Office of the Secretary should develop performance measures for  DOT’s efforts to educate airline passengers. (Recommendation 6)

section_title: Agency Comments
paragraphs: We provided a draft of this report to DOT for review and comment.  DOT  provided written comments, which are reprinted in appendix IV, and  technical comments, which we incorporated as appropriate. DOT  concurred with our recommendations and officials said that they had  begun taking steps to address the recommendations.

We are sending copies of this report to the appropriate congressional  committees, DOT, and other interested parties. In addition, the report is  available at no charge on the GAO website at http://www.gao.gov.

If you or your staff members have any questions about this report, please  contact me at 202-512-2834 or vonaha@gao.gov. Contact points for our  Office of Congressional Relations and Public Affairs may be found on the  last page of this report. GAO staff who made major contributions to this  report are listed in appendix V.

section_title: Appendix I: Studies on the Effect of Market Structure on Elements of Airlines’ Customer Service
paragraphs: Since its deregulation in 1978, numerous studies have examined the  effects of competition in the airline industry. Most have examined the link  between competition and pricing on specific airline routes—i.e., airline  service between two airports or cities. These routes are viewed as the  relevant markets for competitive analysis because they reflect the  products that consumers purchase and for which airlines set prices.  These studies have examined the pricing effect: (1) of route competition,  (2) of the extent of an airline’s presence at airports, and (3) of mergers in  the evolving airline industry. Studies have generally shown (1) that prices  tend to be higher when fewer airlines serve a city-pair market and (2) that  airline dominance at airports can be associated with higher market  prices. Other studies have also shown that the presence of a low-cost  airline on a route—or even the threat of entry by a low-cost airline—is  associated with lower fares.

In addition, some studies have examined whether there is a link between  the level of competition in city-pair markets and certain elements of  customer service quality, such as the incidence and length of delays,  cancellations, lost baggage, flight frequency, and denied boarding. While  competition generally lowers prices, the effect of competition on the  quality of service is more ambiguous. On the one hand, firms may  compete on quality of service; in this instance, competition leads to higher  service, but it is also possible that a firm facing less competition may  invest in quality of service to more fully differentiate among passengers. A  variety of factors could influence the association between competition and  customer service. These factors include, for example:  the cost of providing higher levels of quality,  the extent to which consumers have full knowledge of quality,  the extent to which consumers change future purchasing decisions  based on quality, and  the value consumers place on product quality relative to product  price.

In the context of the airline industry, airline investments that underlie the  provision of consumer services are not necessarily route-specific as they  more likely relate to investments airlines make at airports, or at the overall  airline level. For example, airlines make decisions about the extent to  which resources—such as the number of aircraft and customer service  personnel—are available at a given airport. Moreover, policies regarding  training of gate and customer service personnel likely take place at the  corporate level as do decisions about the configuration of aircraft, which  may have related quality of service factors. Also, because airlines provide  a service that involves a large network, some elements of quality may  relate to the broad decisions regarding the management of that network.  For example, if a flight is delayed on one route, it may affect the  timeliness of several downstream flights due to the late arrival of the  aircraft, pilots, and flight attendants, and airlines may take these  networked effects into consideration in ways that could affect customer  service. Still, some decisions that airlines make do have route-specific  consequences that could influence customer service, such as decisions  on flight scheduling, and which flights to cancel or delay in the face of  operational disruptions.

Some empirical airline literature on the impact of competition on certain  quality factors predates several airline mergers, and some was conducted  more recently. In the earlier literature, several studies found a linkage  between the competitiveness of airline markets and customer service  outcomes such as on-time performance, cancellations, mishandled  baggage and flight frequency. These studies generally found that more  competitive markets are associated with an improvement in one or more  of these aspects of customer service. For example, one study found a  small increase in the number of cancelled flights when a route was served  by only one airline, and another found that such routes had, on average,  slightly longer delays. However, the extent of these improvements has  typically been small, such as an association with a small reduction in  cancellations or a reduced average delay of just a few minutes. On the  other hand, some studies found that delays and cancellations are less  common when they involve airlines’ hub airports—especially when a flight  is destined for an airline’s hub airport.

In order to look more closely at the relationship between market  competition and airline customer service in recent years, we reviewed  several more current studies. Specifically, because the nature of the  airline industry—particularly its competitive landscape—transformed after the 2007–2009 recession, we selected studies that included at least some  of the study period post-recession. We identified six studies that met our  criteria for inclusion, each of which examined some aspect of the link  between airline market competition and one or more element of customer  service.

As with the earlier studies, these more recent studies generally found  greater competition was associated with some improved customer  service. Specifically, some studies found that flight delays were, on  average, a little longer, and flight cancellations more likely when markets  were more highly concentrated or in the aftermath of an airline merger.  For example, one study found that a particular level of increased route  concentration was associated with about a 4-minute average increase in  flight delay. Another study found a similar effect on delay and also found  a slightly higher incidence of cancellations on more concentrated routes.  These increases in delays and cancellations were generally small. In the  case of mergers, the findings are somewhat mixed. One study we  reviewed found increased cancellations and more delays after mergers,  but the effects tended to diminish over time, while another study did not  find an effect of mergers on these measures of customer service. Another  study found that the effect of mergers on consumer welfare—as  measured by both price and flight frequency—may be idiosyncratic to the  specific airlines involved in the merger and the state of competition in the  broader market at the time of the merger. Finally, a GAO study that  examined the effect of the tarmac delay rule on flight cancellations found  that flights on routes where either the originating or destination airport  was a hub airport for the airline had a lower likelihood of cancellation,  possibly indicating a focus by airlines on maintaining smooth operations  as much as possible. Generally, the differing findings on the extent or  existence of quality impacts could be the result of varied methodologies in  these analyses, including differing model specifications, variable  measurements, and analysis time frames.

Finally, while these studies provide insight into the link between  competition and certain aspects of service quality, some elements of  airline’s service quality are harder to explore in this way. For example,  there are no data that would be readily usable in empirical analyses on  the effect of competition on certain quality measures such as the extent  airline websites are user-friendly, the ability to be rebooked on a different  flight when a flight is missed or was cancelled, the helpfulness of airline  staff, and consumer satisfaction with airline cabin amenities, such as seat  comfort and availability and quality of food for sale. Moreover, while  studies examine effects of competition at the route level, the national  airline industry has become more concentrated in the past decade due to  a series of bankruptcies and mergers. The reduced competition at this  broad level may also have implications for customer service, such as the  level of service provided at airports and policies on flight cancellations  and rebooking.

section_title: Appendix II: Objectives, Scope, and Methodology
paragraphs: Our objectives for this report were to: (1) describe trends in DOT data on  airline service from 2008 through 2017 and airlines’ actions to improve  service; (2) assess how effectively DOT ensures airlines’ compliance with  consumer protection requirements; and (3) assess the extent to which  DOT’s airline passenger education efforts align with key practices for  consumer outreach. We also examined the relationship between airline  competition and customer service (app. I).

The scope of this report focused on issues regarding consumer  protections for airline passengers (i.e., “consumer protections”) overseen  by DOT. We focused our analysis on the time period 2008 through 2017  unless otherwise noted because it encompassed key additions or  amendments to consumer protection regulations, including Enhancing  Airline Passenger Protections I, II, and III. For each of our objectives, we  reviewed documents and data from DOT and airlines, to the extent  possible. We also conducted multiple interviews with officials from DOT’s  Office of the Assistant General Counsel for Aviation Enforcement and  Proceedings and its Aviation Consumer Protection Division, in addition to  a non-generalizable sample of 25 stakeholders—including  representatives from 11 airlines, 3 market research organizations, 3  aviation academics, and 8 industry associations representing airlines,  airline staff, and airline passengers.

To describe trends in airline service, we analyzed DOT operational data  and passenger complaints submitted to DOT from 2008 through 2017.  Specifically, we analyzed DOT’s data on late flights; cancellations;  diverted flights (i.e., flights operated from the scheduled origin point to a  point other than the scheduled destination point in the airline’s published  schedule); voluntary and involuntary denied boardings; and mishandled  baggage to describe airlines’ operational performance. From 2008  through 2017, DOT required airlines with at least one percent of domestic  scheduled-passenger revenues in the most recently reported 12-month  period to report this data for reportable flights—we refer to these airlines  as “reporting airlines” throughout our report. We also obtained data for  passenger complaints submitted to DOT and analyzed the data to identify  the frequency, types, and changes in complaints over time. We limited  our analysis of passenger complaint data to “selected” airlines that were  required to report operational data to DOT in 2017— the most recent year  of available data when we started our review—because they were the 12  largest U.S. domestic passenger airlines in 2016. To assess the  reliability of the operational data and complaints, we conducted electronic  testing of the data to identify any outliers, compared our results to DOT  published data, and interviewed DOT officials about how the data were  collected and used. Because our interviews with DOT officials indicated  that no changes had been made to the processes used to collect and  maintain both data sources, we also relied on our past data reliability  assessments from recently issued GAO reports, assessments that found  that both data sources are sufficiently reliable for providing information on  trends over time. Therefore, we determined that the data were  sufficiently reliable for our purposes, including to present high-level trends  in service over time. Moreover, we also reviewed analyses from three  market research organizations that we identified during the course of our  work— J.D. Power and Associates, the American Customer Satisfaction  Index, and the Airline Quality Rankings—to provide additional information  on airline service quality. We interviewed the authors to understand how  they conducted the analyses; however, we did not evaluate the  underlying methodologies. We determined that the results were reliable  enough to report their high-level trends on passenger satisfaction.

To understand airlines’ actions to enhance service, we interviewed or  received written responses from 11 of 12 selected airlines. We  conducted interviews with airline representatives using a semi-structured  interview instrument, which included questions pertaining to business  practices aimed at improving service from 2013 through 2017, among  other things. We conducted three pretests with one airline and two  industry groups. Representatives from each group provided technical  comments, which we incorporated, as appropriate. We limited our  timeframe to the most recent 5 years because business practices in the  industry evolve quickly and we wanted to highlight the most relevant and  recent practices. During interviews, we asked selected airline  representatives whether these practices were documented in contracts of  carriage or other customer commitment documents and reviewed those  documents as appropriate. During these interviews, we also asked  selected airline representatives if they considered certain aspects of their  passenger complaint data they receive directly from passengers to be  proprietary, and all airline representatives said the data were proprietary.  To inform interviews with selected airlines representatives and to  understand recent airlines business practices aimed at improving service  for passengers, we also conducted a literature search of trade  publications and industry reports from 2013 through 2017. Where  relevant, we used information from this literature search as additional  context and as a basis for our questions to airline representatives  regarding specific business practices.

To describe how DOT ensures airlines’ compliance with consumer  protection requirements, we reviewed DOT’s documentation of the  policies, procedures, and guidance that describe its five key compliance  activities. In addition, we conducted multiple interviews with staff from  DOT’s Office of the Assistant General Counsel for Aviation Enforcement  and Proceedings and its Aviation Consumer Protection Division. To  identify trends in DOT’s key compliance activities from 2008 through  2017, we analyzed reports and data DOT provided on the number and  results of its airline inspections, investigations, enforcement actions, and  civil penalties—including data from DOT’s case management system. To  assess the reliability of the data, we interviewed DOT officials to  understand how the data are collected and used and the steps DOT takes  to ensure the data are accurate, complete, and reliable. We determined  that the data were reliable enough to summarize trends in DOT’s  investigation and enforcement actions from 2008 through 2017.

To determine how effectively DOT implements its compliance program,  we assessed selected key compliance activities—i.e., coding passenger  complaints, using the case management system to inform compliance  activities, and developing objectives and related performance  measures—against selected principles of Standards of Internal Control in  the Federal Government related to control activities. We also  summarized other leading practices for developing performance  measures, in addition to our past work, which has identified other  agencies with successful performance measures.

To understand the extent to which passenger education materials  developed by DOT align with key practices for consumer outreach, we  reviewed DOT’s educational materials and assessed them against nine  key practices we previously developed for consumer education  planning. In that prior work, GAO convened an expert panel of 14 senior  management-level experts in strategic communications to identify the key  practices of a consumer education campaign. We believe the key  practices the expert panel identified in 2007 remain relevant today since  the practices are not time-sensitive. In addition to reviewing relevant  materials, we also conducted interviews with DOT officials to understand  their outreach efforts. During these interviews, DOT officials agreed that  these criteria were relevant to conducting consumer outreach. For a  complete list of the criteria and corresponding definitions, see appendix  III.

To understand the impact of airline competition on customer service  provided to passengers we conducted a literature search of pertinent  studies in scholarly, peer-reviewed journals, conference papers, and  government publications. We restricted our review to results published  between January 1, 2012, and December 31, 2017, and our search  yielded 57 academic results and 10 government studies. Of these results,  we reviewed each abstract to determine whether it was relevant to our  objective based on criteria we established. For example, we limited  results to those looking at the U.S. airline system and eliminated results  that focused solely on airfares. In total, we found that 5 academic studies  and 1 government study were ultimately relevant and sufficiently reliable  for our report. Moreover, we also summarized 6 additional studies that we  identified by reviewing the bibliographies of our selected studies or that  were identified as key pieces of research in the field to summarize prior  work in this area.

We conducted this performance audit from September 2017 to November  2018 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.

section_title: Appendix III: Key Practices for Conducting Consumer Outreach
paragraphs: GAO previously identified nine key practices that are important to  conducting a consumer education campaign (see table 5).

section_title: Appendix IV: Comments from the Department of Transportation

section_title: Appendix V: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
paragraphs: Andrew Von Ah, (202) 512-2834 or vonaha@gao.gov.

section_title: Staff Acknowledgments
paragraphs: In addition to the individual named above, other key contributors to this  report were Jonathan Carver, Assistant Director; Melissa Swearingen,  Analyst-in-Charge; Amy Abramowitz; Lacey Coppage; Caitlin Cusati;  Delwen Jones; Kelsey Kreider; Ethan Levy; Gail Marnik; SaraAnn  Moessbauer; Malika Rice; Minette Richardson; Pamela Snedden; and  Laurel Voloder.

Fast facts: Remember when a passenger was forcibly removed from an overbooked flight? Recent customer service issues like this have prompted Congress to question if the Department of Transportation is doing enough to protect passengers.

We summarized airline consumer protections and trends in DOT service data (like on-time flights) and passenger complaints. We found that while service generally improved in recent years, complaints increased.

We also found DOT could improve its airline oversight and passenger education efforts. We made 6 recommendations, including that DOT capture feedback from passengers directly to inform its education efforts.

Question-summary annotations: questions: question: To what extent does DOT employ performance measures?
answer: DOT has established objectives for each of its five key compliance activities that state what it seeks to achieve; however, DOT lacks performance measures for three objectives.
child_questions: question: What is an example of a measure that the DOT has not implemented?
answer: For example, DOT lacks a performance measure for conducting inspections of airlines' compliance with consumer protection requirements at airlines' headquarters and at airports.
child_questions: question: What is the result of the lack of implementation?
answer: As a result, DOT is missing opportunities to capture critical information about airlines' compliance with consumer protection requirements.

questions: question: How successful was DOT in implementing GAO's recommendations on passenger education?
answer: GAO found that while DOT has taken steps to educate passengers on their rights, its efforts did not fully align with four of nine key practices GAO previously identified for conducting consumer education.
child_questions: question: What is one example of DOT's shortcoming?
answer: For example, while DOT has defined the goals and objectives of its outreach efforts, it has not used budget information to prioritize efforts or established performance measures to assess the results.

question: What are other ongoing imperatives for DOT?
answer: DOT has also not solicited input directly from passengers to understand what they know about their rights.
child_questions: question: How would taking this action help the DOT?
answer: Taking such actions would provide DOT with greater assurance that its efforts are meeting passengers' needs.

questions: question: What was GAO asked to review?
answer: GAO was asked to examine airline consumer protection issues.
child_questions: question: What does thie sreport examine?
answer: This report examines, among other issues, (1) trends in DOT's data on airline service; (2) the effectiveness of DOT's compliance efforts; and (3) the extent to which DOT's passenger education efforts align with key practices for consumer outreach.

question: How did GAO perform its analysis?
answer: GAO reviewed DOT data on airline service and analyzed passenger complaint data for the 12 largest domestic airlines from 2008 through 2017; reviewed relevant documents and data on DOT's compliance program; assessed DOT's educational efforts against key practices for successful consumer outreach; and interviewed DOT officials. GAO interviewed or obtained written information from 11 of the 12 airlines.