## 4 Proposed Antenna Codebook and MAB Approach
### Proposed Nested Two-Stage MAB Approach
#### 4.2.1 Proposed Nested Two-Stage TS Algorithm
To avoid estimating the massive mmWave CSI at both RIS and UE, we proposed using antenna codebooks. Moreover, the problem of jointly optimizing the PS vectors at both mmWave BS and RIS was formulated as a MAB game, which contributes to relaxing the required BT overhead. In this context, a nested two-stage MAB strategy was suggested, and nested

\[PERSON]}\) matrix corresponding to this index, i.e., \(\mathbf{\[PERSON]}_{i^{*}_{t}}\), is obtained. Afterward, its corresponding reward \(\psi_{\mathbf{\[PERSON]}_{i^{*}_{t}}f_{i}}\) is achieved, and its model parameters \(Z_{\mathbf{\[PERSON]}_{i}f_{i-1}^{*}}\),\(\overline{\psi}_{\mathbf{\[PERSON]}_{i};f_{i-1}^{\prime}}\), and \(\sigma^{2}_{\overline{\psi}_{i};f_{i-1}^{\prime}}\) are updated for its posterior distribution as given in Algorithm 1, where the methodology presented in [33] is used for updating \(\overline{\psi}_{\mathbf{\[PERSON]}_{i};f_{i-1}^{\prime}}\) and \(\sigma^{2}_{\mathbf{\[PERSON]}_{i};f_{i-1}^{\prime}}\).

In the second stage TS and based on \(\mathbf{\[PERSON]}_{i}\) coming from the first stage, a value of \(f_{i}^{\prime}\) is adjusted, and its corresponding reward \(\psi_{\mathbf{\[PERSON]}_{i};f_{i}^{\prime}}\) is obtained. In this procedure, random samples \(\tau_{\mathbf{\[PERSON]}_{i};f_{i-1}}\sim\mathcal{N}\bigg{(}\overline{\psi}_{\mathbf{\[PERSON]} _{i};f_{i-1}},\sigma^{2}_{\mathbf{\[PERSON]}_{i};f_{i-1}}\bigg{)}\), \(1\leq j\leq|\mathcal{F}|\), are taken, and the index \(j_{t}^{*}\) corresponding to the maximum sample value is chosen as follows:

\[j_{t}^{*}=\text{arg max}_{j}\Big{(}\tau_{\mathbf{\[PERSON]}_{i};f_{i-1}} \Big{)}, \tag{11}\]1 Spectral efficiency comparisons between the proposed nested two-stage MAB schemes and the scheme proposed in [30]. two-stage TS and UCB algorithms were proposed to implement the proposed strategy. Simulation analyses confirm the superior performance of the proposed two-stage TS compared to the UCB-based one. Moreover, the proposed nested two-stage MAB schemes outperform random selection and other benchmarks with a high convergence rate and low BT and computational complexities. Conceptualization, E.M.M. ; Investigation, E.M.M. and K.H. ; Methodology, E.M.M. and S.H. ; Software, E.M.M. and S.H. ; Supervision, S.A.A. ; Validation, E.M.M. ; Writing--original draft, E.M.M. and S.H. ; Writing--review & editing, K.H. and S.A.A. All authors have read and agreed to the published version of the manuscript. The authors extend their appreciation to the Deputyship for Research & Innovation, Ministry of Education in Saudi Arabia, for funding this research work through the project number IF-PSAU-2021/01/18041. Not applicable. The authors declare no conflict of interest. ## References

* [PERSON] et al. (2020) [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON] Reconfigurable Intelligent Surfaces for Wireless Communications: Principles, Challenges, and Opportunities. _IEEE Trans. Cogn. Commun. Netw._**2020**, \(6\), 990-1002. [CrossRef]
* [PERSON] et al. (2020) [PERSON]; [PERSON]; [PERSON] Intelligent Reflecting Surface Versus Decode-and-Forward: How Large Surfaces Are Needed to Beat Relaying? _IEEE Wirel. Commun. Lett._**2020**, \(9\), 244-248. [CrossRef]
* [PERSON] et al. (2017) [PERSON]; [PERSON]; [PERSON] Wi-Fi coordinated WiGig concurrent transmissions in random access scenarios. _IEEE Trans. Veh. Technol._**2017**, _66_, 10357-10371. [CrossRef]
* [PERSON] et al. (2021) [PERSON]; [PERSON]; [PERSON]; [PERSON] Wi-Fi Assisted Contextual Multi-Armed Bandit for Neighbor Discovery and Selection in Millimeter Wave Device to Device Communications. _Sensors_**2021**, _21_, 2835. [CrossRef] [PubMed]
* [PERSON] et al. (2015) [PERSON] [PERSON]; [PERSON]; [PERSON] [PERSON]; [PERSON] [PERSON]; [PERSON] [PERSON]; [PERSON]; [PERSON] [PERSON]; [PERSON] [PERSON]; [PERSON] [PERSON]; [PERSON]; et al. Millimeter-wave wireless LAN and its extension toward 5G heterogeneous networks. _IEICE Trans. Commun._**2015**, _10_, 1932-1948. [CrossRef]
* [PERSON] et al.