## 1 Introduction
Whenever a particular device chooses a non-colliding slot, its payload is successfully decoded, and its interference in the other slots is canceled through successive interference cancellation, increasing the occurrence of other non-colliding slots. Although achieving exceptional performances, these protocols have the _drawbacks_ of requiring packet re-transmissions, a substantial overhead for side information signaling, increased complexity for successive interference cancellation evaluation, and the possibility of propagation errors in the decoding process. RA optimization is challenging in ultra-dense mMTC networks while using machine learning tools is quite promising [17]. Among them, _reinforcement learning_ (RL) has drawn much attention in the research community and has been demonstrated to assign RA slots to MTC devices effectively. RL is a machine learning technique that enables agents/devices to interact with the environment and learn efficient strategies that maximize long-term system performance. The most typical RL algorithm is the Q-learning (QL) algorithm, which can be implemented in a decentralized way at the user equipment even without an operating model of the environment. Several recent works deal with the RA problem for IoT applications in low power wide area networks using LoRaWAN as access technology [18; 19; 20]. LoRaWAN networks use chirp spread spectrum modulation at the physical layer and pure the ALOHA method at the link layer [20], and can achieve low power and long-range communications. In [18], an adaptive algorithm is proposed to select the spreading factors of the nodes in a LoRaWAN scenario with multiple gateways, improving the throughput and packet delivery ratio of the network. Ref. [19] tackles the same problem by using a decentralized RL algorithm known as the multi-armed bandit, optimizing the spreading factors to minimize interference and maximize the energy efficiency of the end devices in the network. Similarly, Ref. [20] uses a deep RL algorithm optimizing the distribution of network resources such as spreading factor, transmission power, and channel, aiming to minimize the LoRaWAN energy transmission. Besides, several recent works have also applied RL algorithms for optimizing RA in non-orthogonal multiple access (NOMA) systems [21; 22; 23], in industrial edge-cloud networks [24], and in vehicle-to-everything (V2X) networks [25]. A different context is the use of cellular technology for IoT connectivity. As discussed in [26], using cellular technology for IoT access instead of wide area networks such as LoRaWAN presents significant benefits in terms of coverage. Since cellular IoT uses pre-existing mobile networks, an extensive coverage area is already in place. This allows one to manage device deployments in different locations. Network switching is also an advantage when using cellular IoT since the device will automatically connect to the network with the strongest signal in the area, ensuring constant and reliable connectivity. Given the above scenario, cellular IoT networks are widely disseminated worldwide and are expected to reach a number of 5.4 billion IoT connections by the end of 2028 [27]. In [28], an RL-based GF RA for pilot collision control is proposed in a cellular IoT scenario. The RL algorithm used in [28] is the Q-Learning, where each accessing device is an independent agent, and the rewards are simply \(+1\) when it chooses a unique RA slot or \(-1\) otherwise. A similar procedure is proposed in [29], in which the BS also sends \(+1\) or \(-1\) depending on the outcome of the device's transmission. However, in the case of collision, the actual reward computed by the device is \(-1\) times the ratio of packets already transmitted by it, e.g., if the device has already transmitted 20% of its packets and collided in the current instant, its actual reward is \(-0.2\). As an improvement, a collaborative QL RA scheme is proposed in [30], in which the negative rewards in case of collisions are proportional to the congestion level of the chosen RA slot.