  Access points (APs) in millimeter-wave (mmWave) and sub-THz-based
user-centric (UC) networks will have sleep mode functionality. As a result of
this, it becomes challenging to solve the initial access (IA) problem when the
sleeping APs are activated to start serving users. In this paper, a novel deep
contextual bandit (DCB) learning method is proposed to provide instant IA using
information from the neighboring active APs. In the proposed approach, beam
selection information from the neighboring active APs is used as an input to
neural networks that act as a function approximator for the bandit algorithm.
Simulations are carried out with realistic channel models generated using the
Wireless Insight ray-tracing tool. The results show that the system can respond
to dynamic throughput demands with negligible latency compared to the standard
baseline 5G IA scheme. The proposed fast beam selection scheme can enable the
network to use energy-saving sleep modes without compromising the quality of
service due to inefficient IA
