## 3 ML Application in O-RAN
### Session Management  
Session management in O-RAN refers to the process of establishing, maintaining, and terminating communication sessions between network components. These sessions are used to transmit data, control signals, and other information between different components of the network. Session management is an essential function of O-RAN, as it enables the coordination and control of network operations and services. It involves the management of session parameters, such as session identifiers, session timeouts, and session initiation and termination procedures. Effective session management in O-RAN is critical for ensuring the efficient and reliable operation of the network, as well as for enabling the delivery of high-quality services to end-users. The authors of [48] explore the efficiency of RL-based methods for intelligent session management when taking advantage of the intelligent gNB architecture of O-RAN. This architecture is again assumed to facilitate the inclusion of AI/ML algorithms mainly due to the introduction of the RIC, which is designed for sustainable deployment of these algorithms. The work is focused on the lack of effort to reduce the packet transmission latency in the Core and Data Networks, which can go up to hundreds of seconds compared to several milliseconds in the gNB fronthaul/backhaul links. By installing intelligent session management schemes based on policy evaluation RL methods such as Q-learning, double and State-Action-Reward-State-Action (SARSA) on the RIC of an O-RAN emulated gNB, it becomes able to effectively predict room to accommodate new PDU sessions with given service requirements. Therefore, the gNB can decide whether to grant a new PDU session or QoS flow, preventing existing and new sessions from violating the latency requirements. In [49], the authors study connection management under the session management function (SMF), specifically for user-cells associating the user with a BS, considering sub-optimal and greedy solutions such as the received signal reference power (RSRP). Even though the greedy approach is simple and effective, it does not take into consideration the network's local and global status. This causes the lack of load balancing where a certain BS can be overloaded while the neighboring BSs are underutilized. So, the O-RAN architecture features support global RAN automation to balance the load over the network resources by deploying ML algorithms as rApps and xApps in the RIC. The authors proposed a Deep Q-learning algorithm to infer the weights of the graph neural networks (GNN) for optimal user-cell association. \begin{table}
\begin{tabular}{c p{284.5 pt}} \hline \hline
**Year** & **Ref** & **Contribution** \\ \hline
2022 & [44] & Demonstrated the performance of on-policy and off-policy DRL methods in the form of PPO and ACER, respectively, and the performance was compared in an O-RAN setup for resource allocation optimization in a real-time surveillance video application. \\ \hline
2022 & [45] & Employed a policy gradient-based RL algorithm as an alternative to the initially proposed ILP, to solve an MDP and address the challenge of fairly allocating limited resources to eMBB and URLLC users from multiple O-RUs while providing a significantly less complex solution. \\ \hline
2021 & [46] & Proposed a RL-based dynamic resource controller leveraging policy differential semi-gradient SARSA to optimize traffic flow management by effectively and dynamically allocating per-flow resources within an O-RAN platform. \\ \hline
2021 & [47] & Designed ML deployed in a Non-RT RIC to adapt the resource allocation policy to environment dynamics while satisfying various energy-driven criteria. \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Summary of current efforts on ML applications for O-RAN resource management.