## 1 Introduction
However, the protocol assumes that devices know the exact number of devices colliding by their chosen RA slot. The performance results are better than those obtained by the independent QL approach of [28]. Nonetheless, Refs. [28; 29; 30] do not assume a realistic system model and do not consider channel effects like multipath fading, path loss, thermal noise, and inter-cell interference (ICI), besides assuming that devices know the exact congestion levels in the case of [30]. In [3], the authors compare the most relevant IoT connectivity technologies, highlighting the main challenges and promising solutions. The existing technologies' main bottlenecks are high signaling overhead, wireless resource scarcity, and inefficient wireless resource usage. On the other hand, massive MIMO and machine learning tools are among the promising solutions to overcome those issues. Therefore, designing RA protocols leveraging massive MIMO technology in conjunction with machine learning tools for overcoming the bottleneck of current IoT technologies is quite relevant [3]. In this work, we propose a QL-based GF RA protocol specially designed for realistic mMTC B5G scenarios, leveraging massive MIMO technology to improve connectivity performance, avoiding pilot collisions. Our network scenarios consider realistic wireless propagation effects, including multipath fading, shadowing, path loss, thermal noise, and ICI. Besides, the devices compute the collaborative QL rewards in collision cases by estimating the congestion levels with _minimal signaling overhead_. Different figures of merit, including latency, network throughput, and per-device throughput reveal the proposed scheme's improved performance compared to others available in the literature while approaching the benchmark performance with perfect congestion level estimates. Besides, numerical results demonstrate the robustness of the proposed scheme regarding different system parameter variations, like the number of antennas or the number of packets each device has to send. _Contributions_. The paper's contributions are threefold:  
1. We propose an effective GF RA protocol designed for realistic mMTC B5G scenarios, leveraging the massive number of BS antennas to improve performance and employing QL to avoid pilot collisions. 2. The proposed method employs an improved congestion level estimation at the devices with _minimal signaling overhead_, owing to massive MIMO propagation features. 3. Extensive numerical results demonstrate the competitive performance achieved by our scheme, approaching the case with perfect congestion level estimates and showing robustness against system parameter changes. _Innovation to prior work_. RL methods for channel access have already been investigated in the literature in different scenarios; however, we can highlight the following innovations of our work. The works in [19; 20] proposed RL methods to optimize the spreading factors in LoRaWAN technology; differently, our focus is the RA problem of cellular IoT networks, which are known to provide extended coverage in comparison to low power wide area networks [26]. Besides, a remarkable benefit of addressing the B5G cellular IoT scenario is the opportunity to exploit massive MIMO, which has been seen as a promising technology to overcome the bottlenecks of current IoT connectivity technologies [3]. Furthermore, while [28; 29; 30] also proposed RL-based RA protocols for cellular IoT, they assumed a simple collision channel as a system model. Contrarily, we assume herein a much more realistic wireless system model for the communication environment, taking into account realistic effects like multipath fading, shadowing, path loss, thermal noise, and ICI while leveraging massive MIMO propagation features to achieve improved performance in such challenging mMTC use mode applications. _Organization_. The remainder of this paper is organized as follows. Section 2 presents our adopted system model. Section 3 revisits the main QL-based GF RA protocols available in the literature.