## 4 Optimizing EE- and SE-Based Beam Training
### A Low-Complexity AB-HP
RBT-DRL produces better EE-SE trade-offs with less complexity than CNNs and LSTMs, which have limits in high-dimensional areas and real-time adaptation. In addition, DL-based and HP strategies, such as those that use ASM and PMA, improve EE and SE even more, demonstrating the potential of cutting-edge AI technologies for mm-wave networks. The training time of the DNN may rise when considering a mm-wave system, since both the size of the action space in [66] and the state vector in [52] grow with the number of antenna elements. The beam training technique utilizes DRL's general architecture. The process of tracking beams is modeled as a DRL process in the training context. The DRL algorithm develops an EE-SE maximization beam training strategy to improve performance while lowering beam training overhead. Initially, depending on previous channel measurements, the DRL block chooses one of many candidate beam training approaches. As the first action in the EE-SE maximization strategy, the chosen beam training method is used. The predicted number of RF chains needed to obtain the highest EE or SE is based on the outcomes of the beam training. The BS with \(M_{tx}\) antennas communicates \(M_{s}\) data streams to the UE with \(M_{rx}\) antennas. The BS and the UE are assumed to be equipped with \(M^{tx}{}_{rf}\) and \(M^{rx}{}_{rf}\) RF chains, respectively, such that \(M_{s}\leq M^{tx}{}_{rf}\leq M_{tx}\) and \(M_{s}\leq M^{rx}{}_{rf}\leq M_{rx}\). Depending on the condition of the system at the time, the DRL block can switch between DRL-EE and DRL-SE configurations. This includes factors like the UE's downlink queue state and battery level. The mode of RBT-DRL-SE, for instance, is activated to communicate more data using spatial multiplexing when the UE's packet queue is backlogged. In contrast, RBT-DRL-EE is activated to save energy for the UE if its battery level is low, such as below 50%. Upon request from the BS, the UE will send a report of its parameters. The EE counts how many bits are transmitted for each unit of energy, which is given by  
\[M(n)=\frac{\text{B}_{n}}{p_{n}}r(n).\text{ Bit}/\text{Joule} \tag{4}\]  
To create a virtual environment for training RBT-DRL agents and operating in highly reliable systems, DRL-scheduling is proposed. \[p_{n}=\frac{\text{P}_{n}}{\zeta}+M_{rf}(n)\Big{(}p_{rf}+M_{tr}p_{ps}\Big{)}+ M_{rf}(n)\Big{(}p_{rf}+M_{rx}p_{ps}\Big{)}+p_{const}, \tag{5}\]where \(\zeta\) represent amplifier efficiency with \(\zeta\in[0,1]\), \(p_{rf}\) represent the power required per RF chain, \(p_{ps}\) represents phase shifter, and \(p_{const}=M_{tr}p_{tx}+M_{rx}p_{rx}+2p_{common}\) represent accounts for the fixed power consumption, where \(p_{tx}\) and \(p_{rx}\) are the power for each transmit and receive antenna. \[EE_{n}=\frac{r_{n}}{\frac{p_{n}}{\zeta}+M_{rf}(n)\Big{(}p_{rf}+M_{tr}p_{ps} \Big{)}+M_{rf}(n)\Big{(}p_{rf}+M_{rx}p_{ps}\Big{)}+p_{const}}. \tag{6}\]  
With relation to the distributional perspective on DRL [16; 71], the agent seeks to determine the best transmission scheduling. The random return is \(\gamma\), whose expectation is the value \(\mathcal{Q}^{\pi}\). Improving sample complexity to hyper-parameters variation increases the robustness, and environmental noise is shown in [17; 40]. The system state specified by the user is denoted as \(s_{n}=\{S_{1,n},\ \ldots\ldots,\ S_{N,n}\}^{T}\in r^{M}\) of the IoT system at the time step \(n\) for the input layer. The random return is achieved by adhering to a current policy \(\pi\) by performing an action \(\omega\) from the state \(s\) indicated by the random variable \(y_{q}^{\pi}(s,\omega)\) due to the startling predictability of the environment.