# Learning-based pricing policy for substitute itineraries

Yue Su, Antoine Désir, Axel Parmentier

## ► To cite this version:

Yue Su, Antoine Désir, Axel Parmentier. Learning-based pricing policy for substitute itineraries. MSOM 2025 - The INFORMS Manufacturing and Service Operations Management annual conference, London Business School, Jun 2025, London, United Kingdom. hal-05461275

HAL Id: hal-05461275

https://hal.science/hal-05461275v1

Submitted on 16 Jan 2026

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
# Learning-based pricing policy for substitute itineraries

Yue Su¹, Antoine Désir², Axel Parmentier³

In airline revenue management, dynamic pricing for substitutable itineraries presents significant challenges due to the stochastic nature of customer choices. The dynamic pricing problem can be conceptualized as a Markov Decision Process with the objective of maximizing the total expected revenue over a finite selling horizon. However, the complexity of this model, characterized by its multi-dimensional state and action spaces, makes it computationally prohibitive to solve exactly, even for small-to-medium-sized instances. As a result, classic methods such as dynamic programming (DP) prove inefficient in addressing the dynamic pricing problem for larger instances. In this work, we introduce learning-based policies tailored to the dynamic pricing problem of substitute itineraries. We proposed two different machine learning (ML) pipelines to encode pricing policy and designed tailored training processes for each of them. Compared to existing methods, our approach has two advantages:

1. Benchmarking the proposed policies against the optimal pricing policy, we show that learning-based policies seem to be a good approximation for the optimal policy.

2. Our learning-based policies can lead to huge speed-up in solving large-scale instances (with up to million-magnitude state space) by moving all CPU time offline.

## 1 Problem statement

We consider the dynamic pricing problem for substitutable itineraries sold in a single market, where customer demand is revealed progressively. The utilities that customers assign to different itineraries are modeled as random variables, and each customer selects the itinerary with maximum utility. The objective of the pricing optimization problem is to maximize the total expected revenue over a finite horizon. We formulate this problem as a Markov Decision Process (MDP). Our setting is summarized as follows:

1. We have a set of itineraries ($\mathcal{I}$), each with an initial inventory (could be heterogeneous).

2. The pricing problem is discretized into $T$ time steps, with at most one customer arriving per step.

3. The objective is to maximize the expected revenue by dynamically adjusting prices at each time step.

## 2 Methodology

Based on different representations of expected revenue, we introduce two machine learning (ML) pipelines to encode pricing policies: (1) a price-based approach, where we predict prices at each time step to calculate the expected revenue, and (2) an opportunity-cost-based approach, where the revenue is determined from predicting the opportunity cost of each itinerary.
## 2.1 Price-based ML pipeline

In this approach, we regard prices as key variables to be predicted. The prediction leverages a statistical model from input features (e.g., inventory state $x_t$ and customer preference), enabling a data-driven pricing strategy. Then, the obtained price $r_t$ is converted to utility vector $u_t$, which is then input into a prediction model to derive customer choices. Assuming that the multinomial logit (MNL) model (Anderson et al. [1992]) is applied to derive $y_t$. Then, MNL can be interpreted as a special regularized prediction function $\hat{y}_{\Omega}$, with regularization $\Omega(y)$ being the negative entropy. Figure 1 presents the price-based ML pipelines.

<figure><img src="image_2.png" /><figcaption>Figure 1: Price-based ML pipelines illustration</figcaption></figure>

## 2.2 Opportunity-cost-based ML pipeline

In this approach, we compute the revenue using the opportunity cost $O_{t+1}$, representing the expected future revenue that could be lost due to the current pricing decision. Therefore, we predict the opportunity cost from the input state $x_t$ via a statistical module, i.e., we directly estimate the potential revenue trade-offs at each time step. Then, $O_{t+1}$ is transferred into a new parameter $\theta_t$, which is a linear function of $O_{t+1}$. Then, we maximize the expected revenue at $t$ by seeking the optimal probabilities $y_t$ with fixed $O_{t+1}$. Figure 2 implies the configuration of opportunity-cost-based ML pipelines.

<figure><img src="image_3.png" /><figcaption>Figure 2: Opportunity-cost-based ML pipelines illustration</figcaption></figure>

## 2.3 Training

Finally, both of the pipelines are trained by Fenchel-Young loss $\mathcal{L}_{\Omega}$ (Blondel et al. [2020]) generated from specific regularization $\Omega(y)$:

* For price-based ML pipelines: the regularization term $\Omega(y)$ is negative entropy.

* For opportunity-cost-based ML pipelines: the regularization term $\Omega(y) = \sum_{i \in \mathcal{I}} y_i(\log(y_i) - \log(y_0))$ where $y_0$ is the probability of no buying.

Given $\Omega(y)$ is strongly convex in both cases, the generated $\mathcal{L}_{\Omega}$ is smooth and differentiable.

# 3 Main Results

**Datasets** The objective of our learning problem is to find parameters $w$ such that the statistical model $\varphi_w$ predicts appropriate parameters $\theta$ that will be input into the downstream layer to derive “good” decisions. For this reason, we train the pipelines to imitate a good policy $\delta$ under a given environment $q$ (e.g., an MNL-based environment). We build a dataset $\mathcal{D}(q, \delta)$ of numerous data points with pairs of
(x, ȳ) that we obtained from the imitated policy. We generate three deterministic training datasets that are based on: (1) the optimal policy δ*, (2) the anticipative policy δa, (3) the voting policy δv, which are summarized as follows:

* δ*-based dataset (D(q, δ*)): at a given time step t, we can derive optimal probabilities of purchasing i ∈ Ī knowing prices. Then, we sample customer choice ȳt given x_t and add all state-decision pairs to the training dataset

* δa-based dataset (D(q, δa)): we sample episodes η₁, ..., ηn to generate multiple scenarios. It should be noted that an episode denotes a sequence of noises that are independent of the decision taken. Then, we solve these scenarios independently and add all state-decision pairs to D.

* δv-based dataset (D(q, δv)): it consists in sampling several trajectories from any current state, solving them independently, and making them “vote” on the first decision by selecting the most frequent. All generated state-decision pairs are added to D.

**Results** We first benchmark our pipeline-encoded policies against optimal pricing policy (Dong et al. [2009]) on small instances with pattern (n, c, d), where n represents the number of itineraries, c denotes the capacity of each itinerary, and d is the total demand. We consider three instances: (3, 5, 50), (3, 10, 50), and (3, 18, 60). Table 1 summarizes the expected revenue (denoted as R<sub>c</sub>(c ∈ {5, 10, 18})) obtained on different instances with different pipelines and the gap between R<sub>c</sub> and the ones derived from optimal pricing policy.

<table><caption>Table 1: Maximum expected revenue obtained from two pipelines, OP means the optimal policy.</caption><thead><tr><th rowspan="2"></th><th colspan="6">Price-based</th><th colspan="6">Opp-cost-based</th></tr><tr><th>R<sub>5</sub></th><th>Gap<sub>5</sub>%</th><th>R<sub>10</sub></th><th>Gap<sub>10</sub>%</th><th>R<sub>18</sub></th><th>Gap<sub>18</sub>%</th><th>R<sub>5</sub></th><th>Gap<sub>5</sub>%</th><th>R<sub>10</sub></th><th>Gap<sub>10</sub>%</th><th>R<sub>18</sub></th><th>Gap<sub>18</sub>%</th></tr></thead><tbody><tr><td>D(q, δ<sub>*</sub>)</td><td>157.97</td><td>0.1%</td><td>282.36</td><td>0.02%</td><td>429.82</td><td>0.1%</td><td>157.90</td><td>0.1%</td><td>281.62</td><td>0.3%</td><td>429.79</td><td>0.1%</td></tr><tr><td>D(q, δ<sub>a</sub>)</td><td>157.97</td><td>0.1%</td><td>281.93</td><td>0.2%</td><td>420.65</td><td>2.2%</td><td>157.83</td><td>0.2%</td><td>281.62</td><td>0.3%</td><td>428.96</td><td>0.3%</td></tr><tr><td>OP</td><td>158.11</td><td>0</td><td>282.42</td><td>0</td><td>430.10</td><td>0</td><td>158.11</td><td>0</td><td>282.42</td><td>0</td><td>430.10</td><td>0</td></tr></tbody></table>

From Table 1, both pipelines yield high-quality pricing policy compared to the optimal pricing policy of Dong et al. [2009]. The average gap between the expected revenues of pipeline-encoded pricing policy is less than 1% in most cases. It indicates the proposed policies are good approximations of optimal pricing policy. Also, we see that opportunity-cost-based ML pipeline seems to be a better policy than the priced-based ML pipeline, as we consider optimizing the expected revenue in the pipelines.

References

Simon P Anderson, Andre De Palma, and Jacques-Francois Thisse. *Discrete choice theory of product differentiation*. MIT press, 1992.

Mathieu Blondel, André FT Martins, and Vlad Niculae. Learning with fenchel-young losses. *The Journal of Machine Learning Research*, 21(1):1314–1382, 2020.

Lingxiu Dong, Panos Kouvelis, and Zhongjun Tian. Dynamic pricing and inventory control of substitute products. *Manufacturing & Service Operations Management*, 11(2):317–339, 2009.