# Context-Aware Beam Selection for IRS-Assisted mmWave V2I Communications
## 4 Proposed Contextual Multi-Armed Bandit Learning Design
In addition, due to path loss from two-hop reflection (BS\(\rightarrow\)IRS\(\rightarrow\)UE), IRS-assisted links often require stronger incident signals or tighter phase alignment to be beneficial compared to direct line-of-sight links. In this work, we adopt the widely used assumption of quasi-static phase configurations for the IRS, consistent with prior works [27]. The IRS is pre-configured to reflect signals toward designated zones based on its physical orientation and deployment geometry. The current algorithm does not perform real-time phase shift optimization but rather selects among available beam-IRS combinations based on learned context-specific rewards. This abstraction allows us to focus on the high-level contextual beam selection problem without entangling it with the physical-layer phase design. However, in practice, the integration of dynamic IRS control--particularly under high-mobility vehicular scenarios--is an open challenge and represents a valuable direction for future work. Algorithm 1 presents the C-MAB algorithm designed for a communication scenario where a BS with with \(B\) beams and \(S\) distributed IRSs serve vehicles in a dynamic environment. The objective is to optimize beam and IRS selection by leveraging contextual information (\(c\)), as detailed in this section, to enhance decision-making. To give an example of operation of the C-MAB algorithm, let \(\mathcal{C}\) represent the set encompassing all possible contextual information, and \(\langle b,s,c\rangle\) denote an ordered triple consisting of beam \(b\), IRS \(s\), and context \(c\). The system records rewards for all combinations of beams, IRSs, and contexts, forming the set \(\mathcal{Q}_{\mathcal{C}}=\{q_{\langle b,s,c\rangle}|b\in\mathcal{B},s\in \mathcal{S},c\in\mathcal{C}\}\). When a beam \(b_{i}\) and an IRS \(s_{i}\) are selected for transmission at time \(t\), an instantaneous context \(c_{i}(t)\) is observed. Subsequently, after a vehicle service duration of \(\Delta t\), the reward \(q_{i}(t+\Delta t)\) associated with the current context is measured. This reward is then utilized to update the average reward value of context \(c_{i}(t)\) using the formula:  
\[\bar{q}_{c_{i}}(t+\Delta t)=\frac{\bar{q}_{c_{i}}(t)\cdot n_{c_{i}}(t)+q_{i}(t +\Delta t)}{n_{c_{i}}(t)+1}, \tag{4}\]  
where \(n_{c_{i}}(t)\) represents the number of times the current context has been observed in the past. The learning strategy adopts an exploration-first approach. The algorithm initializes parameters, including expected rewards (\(Q_{b}^{s}(c)\)) and observation counts (\(N_{b}^{s}(c)\)), for all beam-IRS configurations. In the exploration phase, a subset of beams is activated randomly, and vehicles are also selected randomly based on observed contexts. Depending on beam  
Figure 2: IRS-assisted beam communication under blockages. The dotted red line with a cross symbol represents a blocked (NLOS) path between the mmBS and the vehicle UE. availability, transmission occurs directly or via an IRS. The system updates expected rewards and observation counts through a weighted update rule to adapt to changing conditions. Transitioning to the exploitation phase after a specified number of exploration rounds, the algorithm intelligently selects the top \(K\) beams with the highest expected rewards. For each active beam, the algorithm determines the optimal transmission mode (direct or IRS-assisted) to maximize rewards. Upon connection loss due to blockage or vehicle movement, the reward values for each context-beam-IRS triplet are updated to optimize long-term performance.