# A Transfer Reinforcement Learning Approach for Capacity Sharing in Beyond 5G Networks
## 5 Implementation Considerations  
The DQN-MARL capacity sharing solution can be implemented into the O-RAN architecture, which is expected to be widely adopted in 5G and beyond deployments. The O-RAN architecture is composed of a set of open interfaces for the realization of a virtualized RAN with disaggregated functionalities and embedded AI [27]. Figure 8 shows the integration of the DQN-MARL capacity sharing solution into the O-RAN architecture, illustrating the involved components of the O-RAN architecture in blue and those related to capacity sharing in orange. The solution is devised as an _rApp_ running on the non-Real Time RAN Intelligent Controller (non-RT RIC) functionality of the Service Management and Orchestration (SMO). The SMO is responsible for managing the different O-RAN functionalities; the non-RT RIC functionality enables closed-loop control of RAN and SMO operations with timescales longer than 1 s; and _rApps_ are third-party applications that are executed on the non-RT RIC to provide value-added services related to intelligent RAN optimization and operation. Note that approaching capacity sharing solutions as _rApps_ has been identified as a use case by the O-RAN use case specification for the non-RT RIC in [28]. Figure 6: Aggregated offered load per tenant in one cell for \(N=4\) and \(N^{\prime}=5\). Figure 7: Offered load vs. assigned capacity to tenants 1 and 2 for both re-training modes. The _rApp_ includes the learned DQN-MARL capacity sharing policy for tuning the capacity shares \(\sigma_{n}(t)\) in the different cells of the RAN. Following the O-RAN architecture, the different cell functionalities are split among the O-RAN Radio Unit (O-RU), which is in charge of the lower part of the PHY layer functionality; the O-RAN Distributed Unit (O-DU), responsible for high PHY layer processing, Medium Access Control (MAC), and Radio Link Control (RLC); the O-RAN Central Unit-Control Plane (O-CU-CP), hosting the upper layers of the control plane (i.e., Radio Resource Control (RRC), and control plane of Packet Data Convergence Protocol (PDCP)); and the O-RAN Central Unit -User Plane (O-CU-UP), hosting the upper layers of the user plane (i.e., Service Data Adaptation Protocol (SDAP) and user plane of PDCP). All these cell functionalities are managed by the SMO through the O1 interface, providing diverse management services (e.g., provisioning management services, performance assurance services, etc.) [29]. Considering this functional split, the capacity share \(\sigma_{n}(t)\) of a tenant in a cell is configured in the O-DU function through the O1 interface using the parameter _rRMPolicyDedicatedRatio_, which belongs to the 3 GPP Network Resource Model (NRM) for characterizing RRM Policies in [30]. This parameter is provided per slice, hence for each tenant, identified by means of the Single Network Slice Selection Assistance Information (S-NSSAI). To determine the capacity share \(\sigma_{n}(t)\), the policy of the DQN-MARL _rApp_ makes use of performance measurements gathered by the O-DU and O-CU functions and transferred to the SMO through the O1 interface. For more details on the _rApp_ implementation, the reader is referred to our work in [31]. Moreover, the _rApp_ can interact with different services in the non-RT RIC and the SMO frameworks through the R1 interface [32]. These services include interactions of the _rApp_ through the O1 interface and also with functionalities related to the O-RAN AI/ML workflow services [33]: the _ML model training_ functionality, which allows the training of the DQN-MARL capacity sharing solution; the _ML model repository_ functionality, which allows storing trained ML model versions as a result of re-training; and the _ML model monitoring_ functionality, enabling runtime monitoring of the performance of the deployed policy in the _rApp_.