# An Integrated Capacity Allocation and Dynamic Pricing Model Designed for Air Cargo Transportation
## 2 Materials, Methods, and Models
### Dynamic Pricing
Repetition: The new situation and action are updated as the current situation and action in the next step, and the process of policy formulation is repeated. The main feature of SARSA is that the Q-values updated at each step of the learning process also include the action choices. Although this makes the learning process more consistent and reliable, it may be less optimistic as the update is based on the chosen action. The SARSA algorithm can perform better in environments where exploration is important because it takes into account the actual actions performed by the vehicle. It is suitable for scenarios where exploration and exploitation need to be carefully balanced [34]. In machine learning, the SARSA algorithm is used for reinforcement learning and can be applied to natural language processing, image processing, and recommendation systems [35]. The SARSA algorithm follows an exploration and exploitation policy when interacting with the environment and has been used in various areas such as robotic decision-making, path planning problems, and video games [36]. In our study, we used SARSA as a dynamic pricing algorithm. The input and output parameters of the pricing model we created are shown in Figure 3. In the model, we set the state domain as origin, destination, real-time load factor, total load factor, maximum price, minimum price, flight date, time to flight, and tonnage. We set the action domain as the price between the minimum and maximum price and tonnage less than the aircraft capacity. Our objective in the model was to maximize total revenue per flight, and we completed the model by setting this objective as the reward function. Using a reinforcement learning algorithm, we wanted the model to learn using the reward function and provide the most appropriate price recommendation by learning directly through experimentation rather than relying on pre-labeled data. We trained the model on real data. The data set contains four destinations, and a total of ten origin points selling to each destination were examined. Therefore, the total number of origin and destination pairs is 40. When the other inputs (flight date, fleet type, etc.) are added, 25,203 rows of data are obtained. We modeled the model in Python (PyCharm Community Edition) by separating 80% of the data into training and 20% into testing. Figure 3: Summary of the proposed dynamic pricing model. In the SARSA algorithm, hyperparameters directly affect the learning process and the behavior of the model. To optimize the performance of the model, we used the grid search method, which is one of the hyperparameter optimization methods. The goal of hyperparameter optimization is to find the hyperparameter combination that maximizes the performance of the model. In our study, we optimized four parameters: alpha (\(\alpha\)), gamma (\(\gamma\)), epsilon (\(\epsilon\)) and number of episodes. To evaluate the performance of the optimization model, the highest total reward was used as the performance measure. Despite the examination of a multitude of origin-destination pairs in the data set according to various fleet types, flight months, sales time to flight, etc., the results were taken as averages for the destinations in order to enhance the comprehensibility of the findings. The hyperparameters we used in the model as a result of the optimization are shown in Table 7.  
If we compare the results of the SARSA model with the data allocated for the test on a destination basis, we obtain the results shown in Table 8. For destination number four, the model's price recommendations were closest to the actual prices. Pricing by the system using the established model will help the company to gain manpower, have more control over prices, and generate more revenue. Evaluating the model results, we can say that the model we have built is suitable for use in the air cargo sector to provide dynamic price suggestions. However, the long running time for training the model may be a problem for the real use of the model.