# Leveraging Machine-Learning for D2D Communications in 5G/Beyond 5G Networks
## 6 Case Study: MAB Based mmWave D2D Scenario  
This section demonstrates the effectiveness of ML-based methods over conventional solutions of mmWave D2D NDS problem. Figure 4 illustrates the simulated mmWave D2D network, where a mmWave device is located at the center of a micro-BS area of \(125\times 125\) m\({}^{2}\), and it desires to establish a D2D link with one of its neighbor devices. Conventionally, the center device should exhaustively search over all its surrounding devices using BT and select the best one maximizing the achievable data rate of the D2D linkage. This will highly decrease the link throughput because of the incredible training overhead. Instead, the mmWave NDS problem is modeled as a stochastic MAB, where the center device will act as the player aiming to maximize its long term reward, which is the achievable data rate. This is done via playing over the surrounding devices, serving as the arms of the bandit. Through proactive online learning, the center device will reach up at the device, maximizing its achievable data rate while examining one nearby device at a time, which highly reduces the NDS process's training overhead and increases the throughput consequently. UCB and MOSS [33] algorithms are utilized to prove the effectiveness of the MAB based mmWave NDS over the conventional NDS, and random selection [13]. UCB attempts to improve the action selection's confidence every round by reducing the uncertainty, while MOSS is appropriate for both stochastic and adversarial MAB settings. Hence, both are suitable for the mmWave D2D NDS problem. In random selection, a random nearby device is selected every round for establishing the D2D link. Althoughit highly relaxes the NDS overhead, it results in a poor achievable data rate and low throughput. The UCB and MOSS algorithms are modified to select the best nearby device with maximum long-term data rate for constructing the D2D link. The mmWave channel model plus blocking formulations are given in details in [17]. Table 3 summarizes the simulation parameters values utilized in this case study. Figure 5 shows the average throughput comparisons using a separate number of distributed devices at no blocking and all the paths are line of sight (LOS). The proposed MAB based schemes show superior performance over either conventional (Conv) or random NDS methods. The average throughput is increased as we increase the number of devices due to the pros of MAB based algorithms that reduce the overhead, unlike traditional solutions. The conventional NDS method reduces the average throughput as the number of devices increases. Figure 6 presents the average throughput against the percentage of NLOS availability for the compared algorithms. It is worth noting that MAB based solutions have superior performance even at high LOS blockage. The figure confirms the ML-based algorithms' advantage for solving the NDS problem in mmWave D2D considering harsh LOS blockage environment. One of the main challenges for MAB solutions is the convergence of the algorithm. In Figure 7, we study the convergence rate of the utilized MAB algorithms against the horizon where the optimal solution is added as an upper limit. It is clearly shown that the proposed MAB algorithms achieve a high convergence rate towards the optimal data rate obtained through exhaustively searching all available nearby devices. At \(t=400\), MOSS and UCB converge to \(86\%\), \(70\%\) of the ideal average throughput, respectively. \begin{table}
\begin{tabular}{l c} \hline \hline
**Parameter** & **Value** \\ \hline Bandwidth, Data transmission time & 2.16 GHZ, 20 msec \\ Beamform training time, and Data length & 0.28 msec, 1 Gbit \\ Transmitted power, Half power beam width & 10 dBm and 20\({}^{\circ}\) \\ for AOA & 10 dBm and 20\({}^{\circ}\) \\ LOS and NLOS pathloss exponents & 2.22 and 3.88 \\ LOS and NLOS shadowing’s standard deviation & 10.3, 14.6 \\ Blocking object thinning factor, Blocking & 1 and uniform \([0.3-0.6]\) m \\ object radius & \(-174+10 log10(B)+10\) \\ \hline \hline \end{tabular}
\end{table}