# A Transfer Reinforcement Learning Approach for Capacity Sharing in Beyond 5G Networks
## 4 Performance Evaluation
### Considered Scenario  
The assumed scenario comprises an NG-RAN infrastructure deployment that initially contains \(N=4\) cells, which serve the users of two different tenants, denoted as _Tenant_ 1 and _Tenant_ 2. The cells' configuration considered in the deployment is included in Table 1, along with the SLA parameters established for each tenant in the scenario. The SLA of the \(k\)-th tenant is defined in terms of the Scenario Aggregated Guaranteed Bit Rate, \(SAGBR_{k}\), which is the bit rate to be provided across all cells to the tenant if requested, and the Maximum Cell Bit Rate, \(MCBR_{k}\), which is the maximum bit rate that can be provided to the tenant in each cell. The DQN-MARL capacity sharing solution is used to distribute the capacity of the cells in the scenario among the two tenants, considering that the state of the \(k\)-th tenant in the \(n\)-th cell is given by \(\mathbf{s}_{n}=\!\rho_{n}(t)\), \(\rho_{n}^{A}(t)\), \(\sigma_{n}(t-1)\), \(\sigma_{n}^{A}(t-1)\), \(MCBR_{k,n}\!/\!c_{n}>\), where \(\rho_{n}(t)\) is the fraction of resources occupied by the tenant, \(\rho_{n}^{A}(t)\) is the fraction of available resources not used by any tenant, and \(\sigma_{n}^{A}(t)\) is the available capacity share not assigned to any tenant, all referring to the \(n\)-th cell. Moreover, \(\mathbf{s}^{\textit{ SLA}}=[SAGBR_{k}/C,\sum_{k^{\prime}=1,k^{\prime}\
eq k}^{K}SAGBR _{k^{\prime}}/C]\). The traffic generated by a tenant in one cell is given in terms of the offered load, defined as the ratio between the bit rate required by the tenant and the cell capacity. The offered load of each tenant varies with time to capture the fluctuation of the traffic demands in different hours/days. Moreover, the offered load of a tenant is homogeneous in the scenario, so that all the cells experience the same offered load of that tenant. In the scenario under consideration, a situation of high offered load is experienced. To overcome this situation, the capacity of the scenario is increased by adding a new cell, which absorbs 25% of the offered load of each of the previous cells and is configured according to the cell parameters in Table 1. As a result, the new scenario with \(N^{\prime}=5\) cells can provide more capacity to satisfy the offered loads of the tenants. After the addition of the new cell, a re-training process of the DQN-MARL capacity sharing solution is required to enable the distribution of the capacity among tenants. This re-training is conducted in two modes. The first mode, denoted as _TRL mode_, considers that  
Figure 3: Learning process of the target policy with inter-task mapping. the re-training is conducted by following the inter-task mapping TRL approach proposed in Section 3.2, considering \(\Delta N=1\) cell and using the trained policies for \(N=4\) cells. The second mode, denoted as _Non-TRL mode_, considers that no knowledge transfer is used, so the re-training is performed from scratch for the case of \(N^{\prime}=5\) cells. A Python implementation of the DQN-MARL capacity sharing solution and the TRL approach proposed in Section 3.2 has been carried out to obtain the results. Table 2 presents the DQN configuration parameters used for training the developed DQN agents for \(N=4\) cells and \(N^{\prime}=5\) cells. These parameters apply to both _TRL_ and _Non-TRL_ re-training modes. In the _TRL_ mode, ANN weights are initialized according to (4), while in the _Non-TRL_ mode, weights are initialized randomly. The training is performed through the interaction of the developed DQN agents with a simulated network environment, also developed in Python. This environment is fed with data from a training dataset composed of synthetically generated offered loads with different patterns for the two tenants, each with a one-day duration. The considered patterns cover a wide range of offered load values for both tenants.