# Application of Reinforcement Learning in Decision Systems: Lift Control Case Study
## 5 Tests and Their Interpretation
### Results and Analysis  
During test simulations, various metrics describing the performance of each strategy were collected. The metrics included average reward per simulation step, average waiting time of served passengers, number of served passengers, and number of illegal moves. The results, presented in Table 1, include both mean and standard deviation of each of the metric values over all 30 prepared simulations.  
The initial observation indicates that the LM strategy is the best among the heuristic algorithms. However, it notably underperforms compared to each trained agent across all metrics. The trained agents not only consistently served more passengers, but also served them in a shorter time. Particularly, the agent trained with Prioritized Replay Buffer achieved the shortest average waiting time at 9.44, around 5% better than the LM strategy and over 20% better than the LQF strategy.  
Additionally, even stronger relative improvement is visible in the average reward, where trained agents achieved around 5% better scoresthan the LM strategy without any overlap of the confidence intervals, indicating statistically significant differences in their performance. However, the average reward is more similar among the trained agents than the average waiting time. This occurred because the agent was trained solely to maximize reward, a method that was not constructed based on passenger waiting times but with an idea to minimize that time as a byproduct of maximizing the reward. Furthermore, the agent lacks information about how long passengers have been waiting, as the state representation only includes pressed buttons. In this setting, the environment is not overly complex yet delivers crucial information.  
The absence of illegal move choices by all agents underscores the accurate \(Q\) function estimations for states in which the lift was positioned on either the highest or lowest floors. This outcome demonstrates the accuracy of the \(Q\) function estimations and suggests that penalizing the agent for incorrect actions is an effective approach for handling state-dependent actions. This is particularly relevant when such actions impact only a limited portion of the state space.  
For a more extensive exploration and analysis of the experimental results, please refer to the Engineering Thesis authored by [PERSON] [25]. The thesis explores diverse environments with different complexities, providing valuable insights into the utility of Reinforcement Learning in addressing decision problems like the lift algorithm. Additionally,  
\begin{table}
\begin{tabular}{l c c c c c c c} \hline \hline \multirow{2}{*}{**Strategy**} & \multicolumn{2}{c}{**Average Reward**} & \multicolumn{2}{c}{**Average Waiting Time**} & \multicolumn{2}{c}{**Served Passengers**} & \multicolumn{2}{c}{**Illegal Moves**} \\  & \(\mu\) & \(\sigma\) & \(\mu\) & \(\sigma\) & \(\mu\) & \(\sigma\) & \(\mu\) & \(\sigma\) \\ \hline LQF & \(-0.1101\) & \(0.0137\) & \(11.8963\) & \(1.0981\) & \(84.1333\) & \(6.2020\) & \(0.0\) & \(0.0\) \\ CA & \(-0.0968\) & \(0.0097\) & \(10.0738\) & \(0.5569\) & \(88.3667\) & \(6.7338\) & \(0.0\) & \(0.0\) \\ LM & \(-0.0954\) & \(0.0094\) & \(9.9724\) & \(0.5191\) & \(87.7667\) & \(6.7705\) & \(0.0\) & \(0.0\) \\ DQL\_MP & \(-0.0914\) & \(0.0082\) & \(9.5548\) & \(0.5229\) & \(89.1000\) & \(6.9746\) & \(0.0\) & \(0.0\) \\ DQL\_RB & \(-0.0909\) & \(0.0086\) & \(9.5050\) & \(0.4727\) & \(89.2667\) & \(6.9378\) & \(0.0\) & \(0.0\) \\ DQL\_PRB & \(-0.0906\) & \(0.0090\) & \(9.4452\) & \(0.5751\) & \(89.6333\) & \(6.9107\) & \(0.0\) & \(0.0\) \\ \hline \hline \end{tabular}
\end{table}
Table 1: Performance metrics during 30 test simulations of 1000 time steps for various lift control strategies. For each metric, its mean (\(\mu\)) and standard deviation (\(\sigma\)) are presented. The best mean value in each metric is bolded.  
it shows the crucial role of the reward function in shaping environments for Reinforcement Learning applications.