## 1 Introduction  
Millimeter-wave (mm-wave)-communications-specific problems including high route loss and line-of-sight limitations can be solved with the use of sensing technology. Sensing provides the deep reinforcement learning (DRL) agent with extensive environmental feedback, which helps it adjust beam patterns and better forecast channel fluctuations. This collaboration not only enhances the performance of hybrid precoding systems but also strengthens mm-wave communication's dependability and resilience in complex, dynamic scenarios, a critical feature for networks in the beyond fifth generation (B5G). Hybrid beamforming is a potential technique for mm-wave massive multiple-input multiple-output (mMIMO) systems that provide excellent spectrum economy, high data rate transmission, and a favorable trade-off between hardware complexity and transmission performance. While the reliance on low-frequency bands limits the capacity to provide low latency, ultra-high speeds, higher service quality, and seamless wireless coverage, the rapid expansion in mobile users and devices has increased the burden on mm-wave spectrum resources (300 kHz to 3 GHz) [1, 2]. However, in large-scale dense deployments, higher frequency waves can deliver multi-gigabit rates [3]. The newest commercial mobile communication technology, 5G, was created to meet mobile consumers' wireless transmission needs and offer rudimentary support for massive communication systems, like the Internet of Things (IoT) and enterprise networks. Exploring higher frequency spectrum is essential to finding a solution to the issue of the many access requirements of terminal devices in 5G networks and the constrained availability of spectrum below 6 GHz [4, 5, 6]. Antenna array deployment is essential for drastically lowering the number of necessary radio frequency (RF) chains in mm-wave mMIMO systems. This feature is particularly helpful in real-world wireless communication situations because there are frequently more antennas than RF chains, which makes precise channel estimates extremely difficult. In recent years, a variety of beamforming-based channel estimate techniques have been put out to solve these issues. In particular, a low-complexity two-way channel estimation method was described in [7, 8, 9]. This technique uses beam training to locate antennas associated with dominant beams first, between the users and the base station (BS). The estimation process then considers only the channel elements related to these selected antennas. However, the quantity of pilot symbols needed to cover every potential beam is directly correlated with the quantity of BS antennas, which is a very high amount. Also, some traditional compressive sensing-based algorithms, like the orthogonal matching pursuit algorithm employed in [10], could estimate the beamforming channel with a lower pilot overhead by taking advantage of the sparsity of beamforming channels [11, 12, 13]. Regretfully, none of the aforementioned beamforming channel estimation algorithms [11, 12, 13, 14] can attain enough estimate accuracy in low signal-to-noise ratio (SNR) regions. Furthermore, these methods are computationally complex, especially when dealing with high-sparsity beamforming channels. The antenna array at the BS was also employed [7, 15, 16, 17, 18] in the mm-wave mMIMO system. This allows signals to be concentrated on various antennas and transforms the spatial channel into a beamspace channel. Low-computational-complexity beamspace channel estimation is possible with the approximate message passing (AMP) technique, a potent iterative algorithm for sparse signal recovery [19, 20]. The best shrinkage parameters for the AMP approach are hard to find and empirical shrinkage values are usually utilized instead. This restriction limits the useful channel estimation performance of the method.