## 3 Results
Compared with our proposed method, DDRL based method does not use auto-encoder based leniency control to avoid non-stationary learning environment issue and relative over-generalization issue in MARL. \begin{table}
\begin{tabular}{c c} \hline \hline
**Component** & **Hyper-Parameter** & **Setting** \\ \hline \multirow{8}{*}{Reinforcement Learning} & Learning rate & 0.005 \\ \cline{2-3}  & Discount rate & 0.9 \\ \cline{2-3}  & Synchronization steps & 30 \\ \cline{2-3}  & Memory size & 1500 \\ \cline{2-3}  & Batch size & 100 \\ \cline{2-3}  & Maximum step in a episode & 500 \\ \cline{2-3}  & Cell number & 20 \\ \cline{2-3}  & LSTM hidden state number & 5 \\ \cline{2-3}  & Hidden layer number & 3 \\ \hline \multirow{2}{*}{\(e\)-_greedy_} & Initial value & 1 \\ \cline{2-3}  & Minimum value & 0.05 \\ \hline \multirow{2}{*}{Leniency control} & MaxTemperatur & 1 \\ \cline{2-3}  & Temperature modification coefficient K & 2 \\ \cline{2-3}  & Initial max temperature rate & 1 \\ \hline \multirow{4}{*}{Auto-encoder} & Max temperature decay coefficient & 0.999 \\ \cline{2-3}  & Hash-key dimensions & 5 \\ \cline{2-3}  & Binary LSTM hidden state number & 5 \\ \cline{2-3}  & LSTM layer number & 2 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Hyper-parameters of LC-MARL. Figure 7 shows the overall system rate of different traffic types with 4 SBs, where overall system rate is the sum of each BS's UL/DL transmission rate. In the Figure, the \(x\)-axis and \(y\)-axis represents packet size and sum rate of all UEs. As shown in the Figure, compared with DDRL based framework, our proposed method can reach a higher overall system rate under all four traffic types. Figure 6: Convergence of LC-MARL. (**a**) packet size-550; (**b**) packet size-250;  
Figure 7: Overall rate comparison in different traffic type. (**a**) high load; (**b**) low load; (**c**) high uplink load; (**d**) high downlink load. To verify the performance of our proposed algorithm under different numbers of BSs, we carried out simulations where the number of BSs is 2, 3, 4, and 5. The simulation results are shown in Table 6, where \(Rate_{DDRL}\) and \(Rate_{LC-MARL}\) are the overall system rate of DDRL based method and the overall system rate of LC-MARL based method. We reflect the advantages of the proposed method through performance gain which is calculated by the overall system rate of LC-MARL based method minus the overall system rate of DDRL based method. As shown in Table 6, as the number of agents increases, the performance gains of our proposed method are higher. We further evaluate the performance of our proposed framework by comparing the distributed benefits of our framework and DDRL based solution. Figure 8 describes the data rate of each BS and overall system rate under different packet sizes. It can be seen from the Figure that our proposed framework can achieve a higher overall system rate through distributed learning. Taking Figure 8d as an example, compared with our solution, the DDRL based solution increases the data transmission rate of BS 2, but our solution increases the overall system rate. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**BS Number** & \(Rate_{DDRL}\) **(bps)** & \(Rate_{LC-MARL}\) **(bps)** & **Performance Gain** \\ \hline
2 & 896,956 & 932,579 & 35,623 \\
3 & 1,003,520 & 1,031,180 & 27,660 \\
4 & 1,174,080 & 1,237,370 & 63,290 \\
5 & 1,193,120 & 1,281,630 & 88,510 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Performance comparison in different BS numbers. Figure 8: BS rate and overall rate comparison in different packet size. (**a**) packet size-250; (**b**) packet size-300; (**c**) packet size-350; (**d**) packet size-400; (**e**) packet size-450; (**f**) packet size-500;