Airline Consumer Protections: Additional Actions Could Enhance DOT's Compliance and Education Efforts
Released: 2018-11-20T00:00:00

Letter

Background

State of the Airline Industry

In the U.S. commercial airline industry, passengers travel by air on network, low-cost, and regional airlines. With thousands of employees and hundreds of aircraft, network airlines support large, complex hub- and-spoke operations, which provide service at various fare levels to many destinations. Low-cost airlines generally operate less costly point- to-point service using fewer types of aircraft. Regional airlines typically operate small aircraft—turboprops or regional jets with up to 100 seats—and generally provide service to smaller communities on behalf of network airlines.

The U.S. airline industry’s financial health has improved greatly in recent years due in part to increased demand for air travel as a result of the improved economy, industry reorganization, and changes in business practices. Starting in 2007, airlines faced a number of major challenges, including volatile fuel prices, the financial crisis, and the ensuing recession of 2007–2009. These events led to a wave of domestic airline bankruptcies, five airline mergers, and changes in airlines’ business practices. In all, these circumstances—such as the improved economy and new airline business practices—contributed to record level profits for airlines. For example, in 2017, U.S. airlines reported an after-tax net profit of $13.4 billion for domestic operations, according to DOT data.

As the industry recovered from the recession and passenger traffic began to rebound, airlines began to exercise “capacity restraint” by carefully controlling the number of seats on flights to achieve higher load factors in order to control costs and improve profitability. Because capacity restraint may result in fewer empty seats on many flights, this practice also limits airlines’ ability to rebook passengers if a flight is delayed or cancelled. Airlines have also made changes in their ticket pricing. For example, airlines now generally “unbundle” optional services from the base ticket price and charge ancillary fees for those services. Unbundling may result in passengers paying for services that were previously included in the price of the ticket. Additionally, certain aspects of customer service quality are tied to the class of ticket passengers purchase. For example, purchasing a “basic economy” ticket may include restrictions, such as not allowing passengers to select seats or charging for carry-on bags, that would not apply to a higher priced ticket class. Similarly, the quality of seating varies based on the ticket class purchased—even within the main cabin of the aircraft. Moreover, while the recent airline mergers have resulted in some new service options for passengers in certain markets, they have also reduced consumers’ choice of airlines on some routes and can result in higher ticket prices. At the same time, low-cost airlines provide greater competition in the markets they serve, which may help to keep prices in check.

Factors That Affect Passengers’ Satisfaction with Service

Many factors—from booking a flight through collecting checked baggage—may contribute to passengers’ level of satisfaction with an airline’s service, according to an airline industry association and market research organizations (see fig.1). For example, one industry survey found that passengers most valued affordable airfare, convenient flight schedules, and reliable on-time departures and arrivals.

DOT’s Regulatory, Compliance, and Education Efforts

DOT’s regulatory activities include issuing consumer protection regulations. Specifically, DOT may issue or amend consumer protection regulations under its statutory authority to prohibit unfair or deceptive practices, or unfair methods of competition by airlines, among others. As mentioned previously, under this authority DOT has promulgated various regulations to enhance airline consumer protections since 2009 (see table 1).

When regulations are promulgated, agency officials must determine how to promote compliance and deter noncompliance. Agencies charged with promoting regulatory compliance, including DOT, usually adopt a program that consists of two types of activities: those that encourage compliance and those that enforce the regulations. Compliance assistance helps regulated entities, such as U.S. airlines, understand and meet regulatory requirements, whereas activities such as monitoring, enforcement, and data reporting deter noncompliance and ensure that entities follow requirements. Agencies choose a mix of compliance activities that will achieve their desired regulatory outcome.

DOT promotes airlines’ compliance with consumer protection requirements through a number of activities, and it educates passengers on their rights. For example, DOT has the authority to investigate whether an airline has been, or is engaged, in an unfair or deceptive practice or an unfair method of competition in air transportation or the sale of air transportation. If DOT finds that an airline has violated consumer protection requirements, DOT may take enforcement action against the airline by, for example, assessing civil penalties. In addition to promoting airlines’ compliance with consumer protection requirements, DOT also conducts activities aimed at educating passengers about their rights and the services provided by airlines. For example, DOT has an aviation consumer protection website where it highlights passengers’ rights and describes how to file complaints with DOT, in addition to other consumer resources. Within DOT’s Office of the Secretary (OST), the Office of the Assistant General Counsel for Aviation Enforcement and Proceedings and its Aviation Consumer Protection Division are responsible for these efforts. According to DOT officials, the annual appropriation to OST’s Office of the General Counsel provides funding for DOT’s consumer protection activities, among other things. At the end of fiscal year 2017, DOT employed 38 staff—including 18 attorneys and 15 analysts—to conduct these activities, according to DOT officials.

DOT’s Data Provide Mixed Information on Improvement in the Quality of Airline Service; Selected Airlines Indicate They Are Taking Steps Intended to Enhance Service DOT’s Data Provide Mixed Information on the Quality of Airline Services

DOT’s data, which include both operational measures of airline service, as well as passenger complaints received by DOT, provide mixed information on whether service improved from 2008 through 2017. DOT requires reporting airlines to provide operational data, including information on late, cancelled, or diverted flights; mishandled baggage; and denied boardings. These data showed some general improvement in the quality of airline service from 2008 through 2017. However, during the same time period, the total number of passenger complaints filed with DOT increased for “selected” airlines. Moreover, while these data may be imperfect measures of service quality, they do provide some indication of the passenger experience. DOT publishes data on both operational performance and passengers’ complaints in its monthly Air Travel Consumer Report to inform the public about the quality of services provided by airlines.

Selected Airlines Indicate They Have Taken a Variety of Actions to Enhance Passenger Service

Representatives from all 11 selected airlines highlighted actions they took to enhance passenger service since 2013, including in some of the areas discussed above. While customer service is important for airlines, these actions can also be motivated in part by other factors—including compliance with certain consumer protection requirements or DOT consent orders, or competition with other airlines. For example, one airline developed a wheelchair tracking system in response to DOT enforcement, which also contributed to the airline’s goal to improve its services to passengers with disabilities. Additional examples of service improvements are listed below.

On-time performance. Representatives we interviewed from almost all selected airlines (10 of 11) reported taking actions intended to improve on-time performance or mitigate challenges associated with flight delays and cancellations. These actions varied across airlines from those intended to improve operational performance to those intended to improve the comfort of passengers. For example, one airline began tracking flights that were “at-risk” of meeting DOT’s definition of a chronically delayed flight, so it could, among other things, swap crews or substitute aircraft and avoid these types of delays. According to DOT regulations, airlines with a chronically delayed flight for more than four consecutive one-month periods are engaging in a form of unrealistic scheduling, which is an unfair or deceptive practice and an unfair method of competition. Airlines have also used technology, such as text-messaging updates, to communicate with passengers during delays and cancellations (8 of 9); increased the number of situations where passengers are compensated during delays and cancellations (5 of 9); and empowered customer service agents to provide food, beverages, and entertainment to passengers during flight delays (1 of 9). For example, one airline e-mails all passengers that experience long delays with an apology and voucher for future travel, regardless of whether the delay was within the airline’s control. While DOT has some requirements for airlines on delays and cancellations, such as on tarmac delays and chronically delayed flights, it generally does not require airlines to compensate passengers for delays.

Baggage handling. Representatives we interviewed from almost all network and low-cost airlines (8 of 9) reported investing resources in order to improve baggage-handling efforts and minimize the effects to passengers whose bags are lost or delayed. Among other things, airlines upgraded baggage technology (5 of 9); modernized the claims process, so passengers could complete forms on-line (3 of 9); and instituted replacement baggage programs, where passengers get a replacement bag at the airport (2 of 9). For example, one airline invested several million dollars to use radio frequency identification technology (RFID) to track bags, as well as allowing passengers to track their baggage via an application on their smartphone. Another airline introduced a policy to use FedEx to deliver delayed bags if the airline cannot return them within 24 hours. Since 2011, DOT has required certain airlines to make every reasonable effort to return mishandled baggage within 24 hours.

Quality of interaction with airline staff. Representatives we interviewed from almost all selected airlines (10 of 11) reported improving training programs in an attempt to enhance interactions between airline staff and passengers. For example, one airline worked with the Disney Institute to provide training to staff on relating to guests during travel disruptions and de-escalating conflict. While airlines have increased customer service training, representatives from one industry association said that the training would be more beneficial if it was provided on a more regular basis. Two airlines also expanded their customer service departments’ hours to better match when passengers travel. According to DOT officials, airlines are not required to provide customer service training to staff.

Passengers with disabilities. Representatives we interviewed from almost all network and low-cost airlines (8 of 9) reported taking actions intended to improve services for passengers with disabilities. These actions included programs to replace damaged or misplaced wheelchairs or other assistive devices (3 of 9); improving seating and access to lavatories in the aircraft (1 of 9); and using RFID technology to track wheelchairs (1 of 9). For example, representatives from one airline told us they have retrofitted their larger single aisle aircraft lavatories to be wheelchair accessible. Two airlines also reported changing policies pertaining to emotional support animals. For example, one airline has an online registration for emotional support animals where passengers must submit all documentation at least 48 hours in advance of the flight; according to representatives, the process allows the airline to validate the required paperwork, while providing relevant information to passengers with emotional support animals and ensuring the safety of everyone onboard the aircraft.

Involuntary denied boardings. Representatives we interviewed from network and low-cost airlines (9) reported taking steps to reduce or eliminate involuntary denied boardings. Representatives from three airlines said they have reduced or stopped overbooking flights, and other representatives (5 of 9) said their airlines have begun soliciting volunteers to be “bumped” off a flight (i.e., give up their seat) earlier in the process. Two conduct reverse auctions where they ask passengers what compensation they would accept to take an alternative flight. Airlines are also offering additional incentives to encourage passengers to voluntarily switch to flights with available seats (5 of 9)—including travel vouchers with fewer restrictions or that cover ancillary fees, gift cards for Amazon and other retailers, or large travel credits of up to $10,000.

DOT Conducts Multiple Activities to Monitor Airline Compliance, but Opportunities Exist to Improve These Efforts’ Effectiveness

DOT promotes and monitors airlines’ compliance with consumer protection requirements and deters noncompliance in five key ways, such as by reviewing passenger complaint data and taking enforcement action where it identifies violations. However, we found that DOT could improve its procedures to provide additional assurances that analysts consistently code passengers’ complaints and properly identify potential consumer protection violations, in addition to more fully utilizing data from DOT’s information systems to inform its compliance program. Further, while DOT has objectives for each of its five key compliance activities, it lacks performance measures for three of these objectives. As a result, DOT is limited in its ability to assess progress toward achieving its goal of promoting airlines’ compliance with consumer protection requirements or to identify and make any needed improvements.

DOT Conducts Five Key Compliance Activities

DOT conducts five key activities to help airlines understand and comply with consumer protection requirements: (1) providing compliance assistance to airlines, (2) processing complaints from passengers, (3) conducting compliance inspections of airlines at headquarters and airports, (4) conducting airline investigations, and (5) enforcing airlines’ compliance with consumer protection requirements. Collectively, these key compliance activities are intended to help airlines understand and meet consumer protection requirements and deter noncompliance.

Providing compliance information to airlines. DOT attorneys assist airlines in meeting consumer protection requirements by developing guidance materials and responding to questions. DOT publishes these materials—such as topic-specific webpages and frequently asked questions—on its website. Attorneys and analysts also informally respond to questions or requests for information from airline representatives.

Processing complaints from passengers. As previously stated, passengers may file complaints with DOT via its website, by mail, or through DOT’s telephone hotline. DOT analysts use a web application—the Consumer Complaints Application system—to receive, code, and track passenger complaints. In 2017, DOT’s 15 analysts processed about 18,000 air travel-related complaints. Initial processing involves reviewing the information in the complaint, notifying complainants that their complaint was received, and transmitting the complaint to the relevant airline for action. Analysts assign one of 15 high-level complaint category codes (e.g., “advertising” or “discrimination”) to each complaint as well as more specific lower-level complaint codes and codes indicating a potential violation of consumer protection requirements as necessary. Analysts initially code a complaint based on the passenger’s perception of events and not on an assessment of whether the complaint is a potential violation of consumer protections. According to DOT officials, when initially coding passenger complaints, analysts generally use their judgment to code each passenger’s complaint based on the primary issue. While analysts handle a variety of complaints, DOT may designate specific analysts to handle more complex complaint codes, such as disability complaints. On a monthly basis, DOT provides airlines the opportunity to review the complaints received and the agency’s categorization of each complaint. At that time, airlines have an opportunity to challenge DOT’s categorizations. According to DOT officials, a limited number of complaints are recoded as a result of this process.

Conducting compliance inspections of airlines at headquarters and airports. DOT analysts and attorneys inspect airlines at airline headquarters and airports to assess their compliance with consumer protection requirements. From 2008 through 2016, analysts and attorneys conducted compliance inspections of airlines at the airlines’ headquarters, but DOT has not conducted any such inspections since September 2016. Beginning in 2015, DOT initiated compliance inspections of airlines at airports, and DOT continued to conduct these inspections through 2018. According to DOT officials, they have exclusively conducted on-site inspections of airlines at airports in recent years due, in part, to limited resources and budget unpredictability. However, officials stated that they would consider conducting more inspections of airlines at airline headquarters in the future.

Inspections of airlines at airlines’ headquarters examine customer service policies and passenger complaints received directly by airlines, among other things. According to DOT officials, these inspections represent a “deep dive” into an airline’s relevant policies and involve collecting and analyzing data prior to and after their weeklong visit, as well as interviewing corporate personnel. DOT analysts and attorneys use the agency’s inspection checklist to assess compliance with a variety of regulated areas such as the inclusion of certain information on the airline’s website and the proper reporting of data to DOT (e.g., mishandled baggage and on-time performance data). According to DOT data, between 2008 and 2016 DOT completed inspections at 33 U.S. airlines’ headquarters. These 33 inspections identified 23 systemic violations, resulting in consent orders. Two inspections resulted in warning letters, and eight did not identify any systemic violations. The assessed penalty amounts for these inspections ranged from $40,000 to $1,200,000.

Inspections of airlines at airports examine staff’s knowledge of certain consumer protection requirements and the availability and accuracy of signage and documentation. Such inspections provide DOT the opportunity to examine multiple airlines in one visit. According to DOT officials, during these unannounced inspections, attorneys and analysts focus on assessing compliance through observation and interviews with randomly selected airline employees. For example, analysts and attorneys may confirm the availability of information on compensation for denied boarding from an airline gate agent or review an airline’s required signage on compensation for mishandled baggage to determine whether the information is accurate. According to DOT data, DOT inspected 12 to 14 U.S. airlines annually—most multiple times—at 51 domestic airports from 2015 through 2017. In 2017, DOT conducted inspections at 18 domestic airports that included inspecting 12 U.S. airlines multiple times. In total, from 2015 through 2017, DOT found violations of various consumer protection requirements for 13 airlines that DOT addressed through warning letters. In addition, DOT found violations related to incorrect (e.g., out-of-date) or missing notices regarding baggage liability limits or oversales compensation for 8 airlines that were settled by consent orders with penalties between $35,000 and $50,000.

Conducting airline investigations. According to DOT officials, attorneys determine whether to open an investigation by weighing numerous factors, including whether they believe an airline is systematically violating consumer protection requirements. Attorneys may initiate an investigation based on findings from trends in passenger complaints, compliance inspections, monitoring of airline websites and news media, or information supplied by other entities, including other DOT offices or governmental agencies. According to DOT officials, after gathering preliminary information, an attorney may notify the airline of his or her investigation, request information for further analysis, and then determine whether a violation has occurred and which enforcement action, if any, is appropriate. Attorneys document these investigations using DOT’s case management system. From 2008 to 2017, DOT initiated almost 2,500 investigations as shown in table 2 below.

Enforcing airlines’ compliance with consumer protection requirements. When investigations result in a determination that a violation occurred, DOT may pursue enforcement action against the airline by, for example; (1) seeking corrective actions through warning letters; (2) consent orders (which may include fines); or (3) commencement of a legal action (see table 2). According to DOT officials, attorneys consider a number of factors in determining the appropriate enforcement action, including whether there is evidence of recidivism or systemic misconduct, and the number of passengers affected. According to DOT data, most investigations result in administrative closures and findings of no violation.

According to DOT officials, when attorneys decide to issue a consent order, they work with their managers to arrive at an initial civil penalty level and then negotiate with the airline to arrive at a final settlement agreement and civil penalty amount if applicable. DOT has criteria for setting civil penalties, but officials describe the process as “more art than science” because facts and circumstances always vary. Civil penalties assessed in consent orders often include three parts: mandatory penalties, credits, and potential future penalties (see table 3). A mandatory penalty is the portion of the assessed penalty that must be paid immediately or in installments over a specified period of time. A credit is the portion of the assessed penalty that DOT allows an airline to not pay in order to give credit to the airline for spending funds on passenger compensation or toward specific service improvements, both of which must be above and beyond what is required by existing requirements. A potential future penalty is the portion of the assessed penalty that the airline will pay if DOT determines that the airline violated certain requirements during a specified period of time.

Our review of 76 consent orders for our 12 selected airlines where a penalty was assessed found that DOT issued penalties totaling $17,967,000 from 2008 through 2017. Of this, 47 percent ($8,437,700) comprised mandatory penalties paid by the airline. The remaining amounts were either credits or potential future penalties. According to DOT officials, credits are a better way to effect positive change than merely assessing a mandatory penalty. For example, one recent consent order included violations of regulations regarding assistance for passengers with disabilities, among other things. The airline and DOT agreed to an assessed civil penalty amount of $400,000, $75,000 of which was credited to the airline for compensation to customers filing disability-related complaints in certain years and for implementation of an application to provide real-time information and response capabilities to a wheelchair dispatch and tracking system, among other things. However, our review found that consent orders do not always ensure future compliance. Specifically, we found 14 instances where an airline received multiple consent orders for the same regulatory violation. Three of these instances—each for different airlines—related to violations of the “full fare rule,” and two—also for different airlines—related to airlines’ failure to adhere to customer service plans.

Improvements to DOT’s Procedures Could Provide Greater Assurance That Passengers’ Complaints Are Consistently Coded and that Consumer Protection Violations are Properly Identified

We found that while DOT has some procedures (i.e., guidance documents and on-the-job training) in place for coding passenger complaints, it lacks others that could help ensure that analysts consistently code complaints and that potential consumer protection violations are properly identified. Federal internal control standards state that agencies should design control activities to achieve objectives and establish and operate monitoring activities to evaluate results. By designing and assessing control activities, such as procedures and training, agencies are able to provide management with assurance that the program achieves its objectives, which in this case involve identifying instances of airline noncompliance.

DOT has taken some steps to help analysts code passenger complaints and properly identify potential violations of consumer protection requirements:

Guidance documents. DOT developed two documents to guide complaint processing and evaluation—a coding sheet that helps analysts determine how to code complaints and identify potential consumer protection violations, and a user guide that describes how analysts should enter complaint information into the web application. However, we found that these documents may not be clear or specific enough to ensure that analysts consistently coded complaints or properly identified potential consumer protection violations. For example, while the coding sheet includes explanatory notes in 9 of the 15 complaint categories, it does not include definitions and examples for each of DOT’s 15 complaint categories that would illustrate appropriate use of a complaint code, a gap that could result in inconsistent coding.

On-the-job training. DOT supplements its guidance documents with on-the-job training, which officials told us helps analysts consistently code complaints and identify potential consumer protection violations; however, DOT has not established formal training materials to ensure all new analysts get the same information. DOT pairs each newly- hired analyst with a senior analyst to be their coach and instruct them on how to code complaints. According to DOT officials, senior and supervisory analysts determine when new analysts are able to code and work independently but continue to monitor their work as needed and determined by the senior analyst. DOT officials stated that while the agency does not regularly check the extent to which complaints are consistently coded, supervisory analysts check analysts’ complaint coding on an as-needed basis throughout the year, as well as during semi-annual performance reviews. However, DOT does not provide formal training materials or other guidance to ensure that senior analysts are conveying the same information during these informal, on-the-job training sessions.

DOT officials stated that the combination of the existing guidance, procedures, and hands-on training provides adequate assurance that analysts share a common understanding of the complaint categories resulting in complaints being consistently coded. As a result, DOT officials have not developed additional guidance documents or established formal training materials.

While DOT officials said they believe their procedures and on-the-job training are sufficient to ensure that complaints are consistently coded and that potential consumer protection violations are properly identified, a recent DOT Office of Inspector General (OIG) report found that DOT analysts did not identify when to code complaints as potential consumer protection violations for a sample of frequent flyer complaints the agency reviewed. As a result, in 2016, the DOT OIG recommended that DOT provide additional training on what constitutes an unfair or deceptive practice to strengthen oversight of airlines’ frequent flyer programs. In response, DOT created a special team to process frequent flyer complaints and developed and provided review team analysts and other members with training on how to review complaints and identify potential violations related to airlines’ frequent flyer programs.

Improving DOT’s procedures that analysts use to code complaints and identify potential consumer protection violations could provide DOT with additional assurances that analysts: share a common understanding of the definitions of all the complaint codes, are coding complaints in each category consistently, and are identifying potential consumer protection violations. Consistent coding among analysts is important for a number of reasons. First, according to DOT officials, passengers use complaint data—which are publicly reported in DOT’s Air Travel Consumer Report—to make decisions about air travel, including which airlines to fly. Second, DOT analysts and attorneys use complaint data to guide their compliance activities (e.g., selecting airlines for inspections and investigations, and determining proper enforcement actions).

DOT Is Missing Opportunities to Use Its Case Management System to Help Inform Its Compliance Program

We found that while DOT’s case management system allows attorneys to track investigations, it lacks functionality that would allow DOT officials to more efficiently use data from the system to inform other key activities, such as making compliance and enforcement decisions. Federal internal control principles state that agencies should design an entity’s information system and related control activities to achieve objectives and respond to risks, which in this case involve using data from DOT’s case management system to inform its compliance activities.

Our review of DOT’s case management system identified the following limitations that affect DOT’s ability to use data from its case management system to target resources and accurately monitor trends in violations, compliance activities, and the results of its enforcement actions:

Key data are optional. Attorneys are not required to complete certain key data fields in the case management system. For example, attorneys are not required to document the outcome of an investigation in the “enforcement action” field. According to officials, while attorneys do not always complete this field, they often choose to document the outcome of investigations in the case notes. Even if that information is captured in the case notes section, attorneys can only access that information by individually reviewing each case file.

Data entries are limited. Attorneys cannot record multiple consumer protection violations for a single investigation in the case management system. As a result, when multiple violations occur, attorneys must use their professional judgement to select the primary violation to record. Our review of the 76 consent orders against selected airlines resulting from airline investigations identified 24 instances—or more than 30 percent—where an airline violated multiple consumer protection regulations. While this is a small subset of all investigations (2,464) DOT completed across our timeframe, it suggests investigations could include violations of multiple consumer protection regulations.

Data entries do not reflect DOT’s compliance activities. While the case management system includes a field for attorneys to document the source of investigations, the field’s response options do not fully correspond to DOT’s key compliance activities or align to DOT’s documentation listing the sources of investigations. For example, the field that tracks the source of an investigation includes an option to identify passenger complaints as the source but not an inspection of an airline. Officials told us that, like the outcomes of investigations, attorneys often document the source of an investigation in the case notes. However, as mentioned previously, information captured in the case notes section can only be accessed by individually reviewing each case file.

Limited reporting capabilities exist. Attorneys are limited in their ability to run reports to understand trends across multiple investigations, according to DOT officials. For example, the case management system lacks a function to run reports by certain data fields. Specifically, according to DOT officials, attorneys cannot run reports by the airline name data field and must instead type in the airline name to create a report, a process that could produce unreliable results if an airline’s name is inconsistently entered into the database.

According to DOT officials, the case management system’s capabilities are limited largely because the database was designed as a mechanism for attorneys to manage ongoing investigations. DOT officials told us that, while the database has successfully fulfilled that role, officials have increasingly used data from the case management system to make enforcement decisions. For example, DOT attorneys use information from the case management system to inform civil penalty amounts. In addition, DOT uses data from the case management system to analyze the results of investigations and inspections, as well as the details of consent orders in order to target future compliance activities. However, because of limited reporting capabilities, attorneys and managers must manually create summary documents from the case management system’s data, work that could be time consuming and subject to manual errors, and that does not address the issue that some data are not entered into various data fields in the first place.

Recognizing limitations with the case management system, DOT has taken steps to improve the system. Specifically, starting in June 2018, DOT began working with a contractor to update the case management system’s functionality. Among other things, the updates are intended to improve the system’s ability to run reports, which could enhance DOT’s ability to examine trends in enforcement actions and penalty amounts, and allow the system to track investigation milestones. While DOT’s planned updates may help DOT officials better examine trends in enforcement actions, the planned updates do not fully address the issues we identified above, particularly related to collecting complete data. Collecting complete and comprehensive data in the case management system could allow DOT to better track trends in its investigations, inspections, and enforcement actions and to use that information to make data-driven decisions about future compliance activities and enforcement actions.

DOT Lacks Performance Measures for Three of Five of Its Compliance Program Objectives

While DOT has five objectives for its key compliance program activities, it has not established performance measures for three of these objectives. Objectives communicate what results the agency seeks to achieve, and performance measures show the progress the agency is making toward achieving those objectives. Federal internal control standards state that agencies should define objectives clearly to enable the identification of risks and define risk tolerances. They further state that management defines objectives in measurable terms, so that performance toward those objectives can be assessed. Additionally, the Government Performance and Results Act of 1993 (GPRA), as enhanced by the GPRA Modernization Act of 2010, requires agencies to develop objective, measurable, and quantifiable performance goals and related measures and to report progress in performance reports in order to promote public and congressional oversight, as well as to improve agency program performance.

In fiscal years 2017 and 2018, DOT developed objectives for each of its five key compliance activities; however, as illustrated in table 4 below, DOT does not have performance measures for three of its objectives.

For the three objectives for which DOT has not established performance measures, it has documented qualitative measures in internal agency documents. For example, while DOT has not developed a performance measure related to enforcing airlines’ compliance with consumer protection requirements, it summarized enforcement cases in fiscal year 2017 that illustrated actions the agency had taken to achieve this objective. For instance, one enforcement action included a consent order against an airline with an assessed penalty of $1.6 million for violating DOT’s tarmac delay rule. DOT highlighted similar accomplishments for educating airlines and conducting inspections. For example, DOT issued guidance to help airlines understand their legal obligations to not discriminate against passengers in air travel on the basis of race, color, national origin, religion, sex or ancestry, and the agency highlighted identifying unlawful practices by multiple airlines during an inspection of airlines at an airport. While the actions described may provide DOT with some information on whether it is achieving its objectives, they fall short of internal control standards that call for federal agencies to define objectives in measureable terms to assess performance.

DOT officials stated that they have not developed performance measures to monitor progress toward achievement of some objectives because it is difficult to develop quantifiable performance measures. We have previously reported that officials from other enforcement agencies with similar objectives found it challenging to develop performance measures in part due to the reactive nature of enforcement as well as the difficultly of quantifying deterrence, but were ultimately able to do so. Developing performance measures for all objectives would allow DOT to more fully assess the effectiveness of its efforts at promoting airlines’ compliance with consumer protection requirements. Specifically:

Providing compliance information to airlines. DOT has not developed quantifiable performance measures to assess how well DOT educates airlines about consumer protection requirements. For example, DOT does not have a performance measure for developing and disseminating guidance for specific rules or to issue information on new rules within a certain time frame. Rather, officials told us that they proactively e-mail stakeholders new consumer protection rules— rather than relying on stakeholders having to find them on DOT’s website or Regulations.gov—and if officials receive the same question repeatedly, about the same requirement they might issue guidance on the topic. According to DOT officials, these activities help ensure that stakeholders are complying with relevant consumer protection requirements. DOT officials did not provide a specific reason for why they do not have a performance measure related to this objective. However, without such a measure, DOT cannot be sure that it is providing timely educational materials to clarify new consumer protection requirements and assist airlines in complying with these requirements.

Conducting compliance inspections of airlines at headquarters and airports. DOT lacks quantifiable performance measures related to conducting inspections of airlines at airlines’ headquarters and at airports. Having such a measure could help ensure that DOT conducts these activities. Specifically, we found that while DOT continues to conduct inspections of airlines at airports, it has not conducted inspections at airlines’ headquarters since 2016, despite having identified this compliance activity as a key priority in planning documents. According to DOT officials, they have not conducted inspections at airlines’ headquarters for two primary reasons. First, DOT officials said inspections at airlines’ headquarters require significant staff resources, which DOT has allocated to other compliance activities in recent years. Second, officials said that no airline was an obvious choice for an inspection at its headquarters because DOT had not received a disproportionate number of complaints against a specific airline to suggest an inspection was warranted. However, the DOT OIG previously directed the agency to make these inspections a priority and to allocate resources accordingly, and DOT officials themselves have said that these inspections provide incentives for airlines’ continued compliance regardless of whether one airline has an obvious problem. Establishing performance measures for conducting both types of inspections would provide greater assurance that DOT conducts these activities on a regular basis.

Moreover, officials told us that inspections at airlines’ headquarters examine specific consumer protection requirements that are not examined during inspections at airports, and that inspections at headquarters help promote compliance. Among other things, inspections at airlines headquarters allow DOT officials to: (1) review training manuals and training records; (2) examine a sample of passengers’ complaint data received directly by the airlines, including disability and discrimination complaints; and (3) verify that airlines are current on reporting data such as on mishandled baggage and denied boardings to DOT. Performance measures related to how often and under what circumstances compliance inspections should take place could provide assurance that DOT conducts these activities, and is not missing opportunities to monitor airlines’ compliance with consumer protection requirements.

Enforcing airlines’ compliance with consumer protections. DOT officials told us that they have not developed performance measures for enforcement actions because they would not want to have performance measures that were punitive or reactive by, for example, requiring the agency to collect a certain penalty amount from airlines. While we acknowledge the complexity and risks involved in setting these types of performance measures, as mentioned previously, other agencies have done so. For example, one of the Federal Trade Commission’s performance measures is to focus 80 percent of enforcement actions on consumer complaints. Without a performance measure for enforcement activities, DOT is missing opportunities to assess the effectiveness of these activities and make any needed changes. We have previously reported that performance measurement gives managers crucial information to identify gaps in program performance and plan any needed improvements.

DOT Has Made Recent Improvements, but Its Passenger Education Efforts Do Not Fully Align with Key Consumer Outreach Practices

DOT Updated Its Website to More Effectively Educate Passengers on their Rights

DOT’s primary vehicle for educating passengers is its aviation consumer protection website, which it relaunched in November 2017 (see fig. 3). According to DOT officials, as part of the relaunch, DOT improved the navigability and accessibility of the website by, among other things, arranging material by topic, adding icons for various subjects, and including a link for the website on DOT’s aviation homepage. The website now includes summaries of passengers’ rights on a number of issues including tarmac delays, overbookings, mishandled baggage, and disability issues, as well as DOT’s rules, guidance issued to airlines and others, and enforcement orders on key consumer protection issues. Moreover, the website is now accessible to people with disabilities. Moving forward, DOT has a number of additional updates planned through fiscal year 2019. For example, DOT plans to update its website with information on frequent flyer issues, optional services and fees, and codeshare agreements by the end of calendar year 2018. According to DOT officials, while not statutorily required to conduct these education activities, passenger education is a key effort to ensuring airlines’ compliance.

DOT also has numerous other efforts to educate passengers on their rights. For example:

Establishing resources for passengers. DOT developed Fly Rights—an online brochure that details how passengers can avoid common travel problems—in addition to material on unaccompanied minors, family seating, and a glossary of common air travel terms. DOT also developed training tools (e.g., brochures, digital content, and videos) on the rights of passengers with disabilities under the Air Carrier Access Act of 1986 and its implementing regulations, including wheelchair assistance at airports and onboard aircraft, traveling with a service animal, and traveling with assistive devices. While some of these materials were developed primarily for airline employees and contractor staff, others were developed to directly assist passengers with disabilities by providing helpful tips on airlines’ responsibilities, according to DOT officials.

Building consumer education information into existing regulations. Passenger education is built into certain consumer protection requirements, according to DOT officials. For example, when an airline involuntarily denies a passenger boarding, immediately after the denied boarding occurs the airline must provide a written statement explaining the terms, conditions, and limitations of denied boarding compensation, and describing the airline’s boarding priority rules and criteria.

Responding to complaints. DOT officials said they include information on an airline’s responsibilities when responding to passenger complaints. For example, if a passenger submits a complaint to DOT about not receiving compensation for a delayed or cancelled flight, the DOT analyst may inform the passenger that airlines are generally not required to compensate passengers in these instances.

DOT’s Educational Efforts Fully Align with Five of Nine Key Practices

We compared DOT’s efforts to educate airline passengers about their rights against key practices for consumer outreach GAO identified in prior work and found that DOT’s efforts fully align with five of the nine key practices (see fig. 4). For example, we found that DOT has successfully identified the goals and objectives of its passenger education program and identified the appropriate media mix for disseminating its materials. Similarly, we found that DOT had identified and engaged stakeholders, a step that, according to DOT officials, allowed them to better tailor materials.

However, as summarized in the figure below, we found that DOT only partially met or did not meet the remaining four key practices.

For example, DOT’s actions do not align with the key practice to “identify resources” and only partially align with the key practice to “develop consistent, clear messages” based on the established budget. According to a senior DOT official, DOT has not identified budgetary resources because, while important, DOT’s educational efforts are secondary to the office’s other efforts. Further, officials said that it has been difficult for the agency to develop a budget when it has been operating under a continuing resolution for some part of the fiscal year for the last decade. However, without identifying short- and long-term budgetary resources and planning activities accordingly, DOT is missing an opportunity to plan educational efforts or prioritize needs based on available resources.

In addition, we found DOT’s efforts only partially align with the key practice that calls for an agency to research its target audience. While DOT has solicited some input from stakeholder groups such as those representing passengers with disabilities, DOT has not solicited feedback directly from passengers to understand what they know about their rights. DOT officials said they have not sought such feedback because they have not identified a method for doing so that would be statistically generalizable and not cost prohibitive. While costs are always an issue when considering budget priorities, we have previously reported on other agencies’ direct consumer outreach efforts that while not statistically generalizable were nonetheless useful for understanding the effect of the agencies’ efforts. For example, the Bureau of Consumer Financial Protection has used focus groups to understand its outreach efforts. Bureau of Consumer Financial Protection officials previously told GAO that while obtaining information through such efforts was resource intensive, it allowed them to assess the performance of their outreach activities. In another case, an agency surveyed users that access its website to help it understand whether its outreach efforts were effective. Obtaining input from passengers directly on what information they want or what they know about their rights would provide DOT with greater assurance that educational materials are appropriately tailored to meet a wide range of passengers’ needs.

Finally, DOT has not established performance measures to understand the quality of its passenger education materials (i.e., process measures) or the effectiveness of its efforts (i.e., outcome measures). DOT officials said that they receive informal input from stakeholders on the quality of the materials and track website traffic to understand whether materials are reaching passengers. Officials said they believe that these mechanisms provide them with some assurance that the materials are meeting passengers’ needs and that passengers are accessing and using the materials. While these mechanisms may provide DOT with some information on how often materials are accessed online, they do not help it understand the quality of the materials and measure the success of its passenger education efforts. For example, while DOT officials track website traffic, they have not established a related performance measure. A number of different measures could be used to track processes and outcomes related to the use of its website, including the time consumers spend on the website, number of website pages viewed, bounce rate (i.e., percentage of visitors who looked at only one page and immediately left the site), or user’s perception of the experience of their visit. Establishing such measures would provide DOT with greater assurances that its educational efforts are appropriately tailored to passengers and leading to improved understanding of passengers’ rights, including whether any adjustments are needed.

Conclusions

To enforce consumer protection requirements, such as those preventing unfair or deceptive practices or unfair methods of competition by airlines, DOT has conducted almost 2,500 investigations and issued about 400 consent orders over the last decade. However, DOT lacks reasonable assurance that its approach is achieving the highest level of airlines’ compliance, given its available resources. For example, DOT has not assessed whether its procedures and training materials help analysts consistently code passengers’ complaints and identify potential consumer protection violations. Additionally, DOT has not fully used data from its case management system to inform its compliance program. Moreover, in the absence of comprehensive performance measures, DOT lacks a full understanding of the extent to which it is achieving its goal of airlines’ compliance with consumer protection requirements and whether any programmatic changes may be warranted. Improvements in these areas would provide DOT with additional information to target its resources and improve compliance.

DOT has taken positive steps to educate passengers about their rights— through its revamped website and other educational resources. Nevertheless, DOT could improve its efforts by more fully following key practices GAO previously identified for conducting consumer education, such as by: seeking feedback directly from consumers; identifying short- and long-term budget resources; and establishing performance measures.

Taking such actions would provide DOT with greater assurance that its efforts are meeting passengers’ needs.

Recommendations for Executive Action

We are making the following six recommendations to DOT:

The Office of the Secretary should assess its procedures and training materials for coding airline passengers’ complaints, as appropriate, to help ensure that passengers’ complaints are consistently coded and that potential consumer protection violations are properly identified. (Recommendation 1)

The Office of the Secretary should assess the feasibility and cost of updating its airline case management system to address data and reporting limitations, and to undertake those updates that are cost effective and feasible. (Recommendation 2)

The Office of the Secretary should establish performance measures for each of its objectives for its five key airline-compliance activities. (Recommendation 3)

The Office of the Secretary should capture feedback directly from airline passengers or identify other mechanisms to capture passengers’ perspectives to inform DOT’s education efforts. (Recommendation 4)

The Office of the Secretary should identify available short- and long- term budgetary resources for DOT’s airline-passenger education efforts. (Recommendation 5)

The Office of the Secretary should develop performance measures for DOT’s efforts to educate airline passengers. (Recommendation 6)

Agency Comments

We provided a draft of this report to DOT for review and comment. DOT provided written comments, which are reprinted in appendix IV, and technical comments, which we incorporated as appropriate. DOT concurred with our recommendations and officials said that they had begun taking steps to address the recommendations.

We are sending copies of this report to the appropriate congressional committees, DOT, and other interested parties. In addition, the report is available at no charge on the GAO website at http://www.gao.gov.

If you or your staff members have any questions about this report, please contact me at 202-512-2834 or vonaha@gao.gov. Contact points for our Office of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix V.

Appendix I: Studies on the Effect of Market Structure on Elements of Airlines’ Customer Service

Since its deregulation in 1978, numerous studies have examined the effects of competition in the airline industry. Most have examined the link between competition and pricing on specific airline routes—i.e., airline service between two airports or cities. These routes are viewed as the relevant markets for competitive analysis because they reflect the products that consumers purchase and for which airlines set prices. These studies have examined the pricing effect: (1) of route competition, (2) of the extent of an airline’s presence at airports, and (3) of mergers in the evolving airline industry. Studies have generally shown (1) that prices tend to be higher when fewer airlines serve a city-pair market and (2) that airline dominance at airports can be associated with higher market prices. Other studies have also shown that the presence of a low-cost airline on a route—or even the threat of entry by a low-cost airline—is associated with lower fares.

In addition, some studies have examined whether there is a link between the level of competition in city-pair markets and certain elements of customer service quality, such as the incidence and length of delays, cancellations, lost baggage, flight frequency, and denied boarding. While competition generally lowers prices, the effect of competition on the quality of service is more ambiguous. On the one hand, firms may compete on quality of service; in this instance, competition leads to higher service, but it is also possible that a firm facing less competition may invest in quality of service to more fully differentiate among passengers. A variety of factors could influence the association between competition and customer service. These factors include, for example: the cost of providing higher levels of quality, the extent to which consumers have full knowledge of quality, the extent to which consumers change future purchasing decisions based on quality, and the value consumers place on product quality relative to product price.

In the context of the airline industry, airline investments that underlie the provision of consumer services are not necessarily route-specific as they more likely relate to investments airlines make at airports, or at the overall airline level. For example, airlines make decisions about the extent to which resources—such as the number of aircraft and customer service personnel—are available at a given airport. Moreover, policies regarding training of gate and customer service personnel likely take place at the corporate level as do decisions about the configuration of aircraft, which may have related quality of service factors. Also, because airlines provide a service that involves a large network, some elements of quality may relate to the broad decisions regarding the management of that network. For example, if a flight is delayed on one route, it may affect the timeliness of several downstream flights due to the late arrival of the aircraft, pilots, and flight attendants, and airlines may take these networked effects into consideration in ways that could affect customer service. Still, some decisions that airlines make do have route-specific consequences that could influence customer service, such as decisions on flight scheduling, and which flights to cancel or delay in the face of operational disruptions.

Some empirical airline literature on the impact of competition on certain quality factors predates several airline mergers, and some was conducted more recently. In the earlier literature, several studies found a linkage between the competitiveness of airline markets and customer service outcomes such as on-time performance, cancellations, mishandled baggage and flight frequency. These studies generally found that more competitive markets are associated with an improvement in one or more of these aspects of customer service. For example, one study found a small increase in the number of cancelled flights when a route was served by only one airline, and another found that such routes had, on average, slightly longer delays. However, the extent of these improvements has typically been small, such as an association with a small reduction in cancellations or a reduced average delay of just a few minutes. On the other hand, some studies found that delays and cancellations are less common when they involve airlines’ hub airports—especially when a flight is destined for an airline’s hub airport.

In order to look more closely at the relationship between market competition and airline customer service in recent years, we reviewed several more current studies. Specifically, because the nature of the airline industry—particularly its competitive landscape—transformed after the 2007–2009 recession, we selected studies that included at least some of the study period post-recession. We identified six studies that met our criteria for inclusion, each of which examined some aspect of the link between airline market competition and one or more element of customer service.

As with the earlier studies, these more recent studies generally found greater competition was associated with some improved customer service. Specifically, some studies found that flight delays were, on average, a little longer, and flight cancellations more likely when markets were more highly concentrated or in the aftermath of an airline merger. For example, one study found that a particular level of increased route concentration was associated with about a 4-minute average increase in flight delay. Another study found a similar effect on delay and also found a slightly higher incidence of cancellations on more concentrated routes. These increases in delays and cancellations were generally small. In the case of mergers, the findings are somewhat mixed. One study we reviewed found increased cancellations and more delays after mergers, but the effects tended to diminish over time, while another study did not find an effect of mergers on these measures of customer service. Another study found that the effect of mergers on consumer welfare—as measured by both price and flight frequency—may be idiosyncratic to the specific airlines involved in the merger and the state of competition in the broader market at the time of the merger. Finally, a GAO study that examined the effect of the tarmac delay rule on flight cancellations found that flights on routes where either the originating or destination airport was a hub airport for the airline had a lower likelihood of cancellation, possibly indicating a focus by airlines on maintaining smooth operations as much as possible. Generally, the differing findings on the extent or existence of quality impacts could be the result of varied methodologies in these analyses, including differing model specifications, variable measurements, and analysis time frames.

Finally, while these studies provide insight into the link between competition and certain aspects of service quality, some elements of airline’s service quality are harder to explore in this way. For example, there are no data that would be readily usable in empirical analyses on the effect of competition on certain quality measures such as the extent airline websites are user-friendly, the ability to be rebooked on a different flight when a flight is missed or was cancelled, the helpfulness of airline staff, and consumer satisfaction with airline cabin amenities, such as seat comfort and availability and quality of food for sale. Moreover, while studies examine effects of competition at the route level, the national airline industry has become more concentrated in the past decade due to a series of bankruptcies and mergers. The reduced competition at this broad level may also have implications for customer service, such as the level of service provided at airports and policies on flight cancellations and rebooking.

Appendix II: Objectives, Scope, and Methodology

Our objectives for this report were to: (1) describe trends in DOT data on airline service from 2008 through 2017 and airlines’ actions to improve service; (2) assess how effectively DOT ensures airlines’ compliance with consumer protection requirements; and (3) assess the extent to which DOT’s airline passenger education efforts align with key practices for consumer outreach. We also examined the relationship between airline competition and customer service (app. I).

The scope of this report focused on issues regarding consumer protections for airline passengers (i.e., “consumer protections”) overseen by DOT. We focused our analysis on the time period 2008 through 2017 unless otherwise noted because it encompassed key additions or amendments to consumer protection regulations, including Enhancing Airline Passenger Protections I, II, and III. For each of our objectives, we reviewed documents and data from DOT and airlines, to the extent possible. We also conducted multiple interviews with officials from DOT’s Office of the Assistant General Counsel for Aviation Enforcement and Proceedings and its Aviation Consumer Protection Division, in addition to a non-generalizable sample of 25 stakeholders—including representatives from 11 airlines, 3 market research organizations, 3 aviation academics, and 8 industry associations representing airlines, airline staff, and airline passengers.

To describe trends in airline service, we analyzed DOT operational data and passenger complaints submitted to DOT from 2008 through 2017. Specifically, we analyzed DOT’s data on late flights; cancellations; diverted flights (i.e., flights operated from the scheduled origin point to a point other than the scheduled destination point in the airline’s published schedule); voluntary and involuntary denied boardings; and mishandled baggage to describe airlines’ operational performance. From 2008 through 2017, DOT required airlines with at least one percent of domestic scheduled-passenger revenues in the most recently reported 12-month period to report this data for reportable flights—we refer to these airlines as “reporting airlines” throughout our report. We also obtained data for passenger complaints submitted to DOT and analyzed the data to identify the frequency, types, and changes in complaints over time. We limited our analysis of passenger complaint data to “selected” airlines that were required to report operational data to DOT in 2017— the most recent year of available data when we started our review—because they were the 12 largest U.S. domestic passenger airlines in 2016. To assess the reliability of the operational data and complaints, we conducted electronic testing of the data to identify any outliers, compared our results to DOT published data, and interviewed DOT officials about how the data were collected and used. Because our interviews with DOT officials indicated that no changes had been made to the processes used to collect and maintain both data sources, we also relied on our past data reliability assessments from recently issued GAO reports, assessments that found that both data sources are sufficiently reliable for providing information on trends over time. Therefore, we determined that the data were sufficiently reliable for our purposes, including to present high-level trends in service over time. Moreover, we also reviewed analyses from three market research organizations that we identified during the course of our work— J.D. Power and Associates, the American Customer Satisfaction Index, and the Airline Quality Rankings—to provide additional information on airline service quality. We interviewed the authors to understand how they conducted the analyses; however, we did not evaluate the underlying methodologies. We determined that the results were reliable enough to report their high-level trends on passenger satisfaction.

To understand airlines’ actions to enhance service, we interviewed or received written responses from 11 of 12 selected airlines. We conducted interviews with airline representatives using a semi-structured interview instrument, which included questions pertaining to business practices aimed at improving service from 2013 through 2017, among other things. We conducted three pretests with one airline and two industry groups. Representatives from each group provided technical comments, which we incorporated, as appropriate. We limited our timeframe to the most recent 5 years because business practices in the industry evolve quickly and we wanted to highlight the most relevant and recent practices. During interviews, we asked selected airline representatives whether these practices were documented in contracts of carriage or other customer commitment documents and reviewed those documents as appropriate. During these interviews, we also asked selected airline representatives if they considered certain aspects of their passenger complaint data they receive directly from passengers to be proprietary, and all airline representatives said the data were proprietary. To inform interviews with selected airlines representatives and to understand recent airlines business practices aimed at improving service for passengers, we also conducted a literature search of trade publications and industry reports from 2013 through 2017. Where relevant, we used information from this literature search as additional context and as a basis for our questions to airline representatives regarding specific business practices.

To describe how DOT ensures airlines’ compliance with consumer protection requirements, we reviewed DOT’s documentation of the policies, procedures, and guidance that describe its five key compliance activities. In addition, we conducted multiple interviews with staff from DOT’s Office of the Assistant General Counsel for Aviation Enforcement and Proceedings and its Aviation Consumer Protection Division. To identify trends in DOT’s key compliance activities from 2008 through 2017, we analyzed reports and data DOT provided on the number and results of its airline inspections, investigations, enforcement actions, and civil penalties—including data from DOT’s case management system. To assess the reliability of the data, we interviewed DOT officials to understand how the data are collected and used and the steps DOT takes to ensure the data are accurate, complete, and reliable. We determined that the data were reliable enough to summarize trends in DOT’s investigation and enforcement actions from 2008 through 2017.

To determine how effectively DOT implements its compliance program, we assessed selected key compliance activities—i.e., coding passenger complaints, using the case management system to inform compliance activities, and developing objectives and related performance measures—against selected principles of Standards of Internal Control in the Federal Government related to control activities. We also summarized other leading practices for developing performance measures, in addition to our past work, which has identified other agencies with successful performance measures.

To understand the extent to which passenger education materials developed by DOT align with key practices for consumer outreach, we reviewed DOT’s educational materials and assessed them against nine key practices we previously developed for consumer education planning. In that prior work, GAO convened an expert panel of 14 senior management-level experts in strategic communications to identify the key practices of a consumer education campaign. We believe the key practices the expert panel identified in 2007 remain relevant today since the practices are not time-sensitive. In addition to reviewing relevant materials, we also conducted interviews with DOT officials to understand their outreach efforts. During these interviews, DOT officials agreed that these criteria were relevant to conducting consumer outreach. For a complete list of the criteria and corresponding definitions, see appendix III.

To understand the impact of airline competition on customer service provided to passengers we conducted a literature search of pertinent studies in scholarly, peer-reviewed journals, conference papers, and government publications. We restricted our review to results published between January 1, 2012, and December 31, 2017, and our search yielded 57 academic results and 10 government studies. Of these results, we reviewed each abstract to determine whether it was relevant to our objective based on criteria we established. For example, we limited results to those looking at the U.S. airline system and eliminated results that focused solely on airfares. In total, we found that 5 academic studies and 1 government study were ultimately relevant and sufficiently reliable for our report. Moreover, we also summarized 6 additional studies that we identified by reviewing the bibliographies of our selected studies or that were identified as key pieces of research in the field to summarize prior work in this area.

We conducted this performance audit from September 2017 to November 2018 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix III: Key Practices for Conducting Consumer Outreach

GAO previously identified nine key practices that are important to conducting a consumer education campaign (see table 5).

Appendix IV: Comments from the Department of Transportation

Appendix V: GAO Contact and Staff Acknowledgments

GAO Contact

Andrew Von Ah, (202) 512-2834 or vonaha@gao.gov.

Staff Acknowledgments

In addition to the individual named above, other key contributors to this report were Jonathan Carver, Assistant Director; Melissa Swearingen, Analyst-in-Charge; Amy Abramowitz; Lacey Coppage; Caitlin Cusati; Delwen Jones; Kelsey Kreider; Ethan Levy; Gail Marnik; SaraAnn Moessbauer; Malika Rice; Minette Richardson; Pamela Snedden; and Laurel Voloder.