# Outsmarting Human Design in Airline Revenue Management
###### Abstract  
The accurate estimation of how future demand will react to prices is central to the optimization of pricing decisions. The systems responsible for demand prediction and pricing optimization are called revenue management (RM) systems, and, in the airline industry, they play an important role in the company's profitability. As airlines' current pricing decisions impact future knowledge of the demand behavior, the RM systems may have to compromise immediate revenue by efficiently performing price experiments with the expectation that the information gained about the demand behavior will lead to better future pricing decisions. This earning while learning (EWL) problem has captured the attention of both the industry and academia in recent years, resulting in many proposed solutions based on heuristic optimization. We take a different approach that does not depend on human-designed heuristics. We present the EWL problem to a reinforcement learning agent, and the agent's goal is to maximize long-term revenue without explicitly considering the optimal way to perform price experimentation. The agent discovers through experience that \"myopic\" revenue-maximizing policies may lead to a decrease in the demand model quality (which it relies on to take decisions). We show that the agent finds novel pricing policies that balance revenue maximization and demand model quality in a surprisingly effective way, generating more revenue over the long run than current practices. revenue management; reinforcement learning; demand learning; price experimentation; earning while learning; exploration-exploitation trade-off; active learning +
Footnote †: journal: _arXiv preprint arXiv:1907.03301_  
1  
revenue by up to 4% [1]. Optimizing pricing decisions under the assumption that the demand behavior is unknown, which must be discovered from historical data, has come to be known as the \"earning while learning\" (EWL) problem. At every moment, the airline faces a choice of selecting the \"perceived\" optimal price that maximizes revenue according to its current beliefs or selecting another price that increases the amount of information the airline has about the demand behavior. The EWL problem has attracted strong interest from academia and the airline industry in recent years, most notably after the COVID-19 pandemic, when demand behavior changed significantly from historical behavior as a consequence of an important shift in competitor presence and customer interests [2; 3; 4; 5]. Even though we only consider stationary demand in this work (i.e., no shocks), we believe that many changes in RMSs will be needed to efficiently rediscover new demand behavior during the following years of recovery, effective price experimentation being perhaps one of the most important pieces of the puzzle. Even though the research performed for this study can be extended to the case of market shocks, for simplicity we analyze its performance in well-understood stationary environments, i.e., where the amount of data collected from interactions with the demand is limited. Most of the work conducted with the aim of solving the EWL problem relies on expert-designed heuristics. Some of these heuristics are very effective; however, they often come with constraints that make them difficult to apply to real-world airline RM. RMSs use complex optimization methods to adjust prices according to a flight's remaining inventory capacity, requiring such heuristics to be adaptable to the optimization method that is being used. Moreover, airlines manage several flights simultaneously (e.g., daily flights between destinations), and the booking data of each flight are appended to the historical database and used for demand model estimation, suggesting that the heuristic methods need to adapt price experimentation across many concurrent flights.