# FIT-IRSA: Feedback-Integrated Two-Phase IRSA with Deep Reinforcement Learning

Andrei-Valentin Stirbu, Cédric Adjih

► To cite this version:

Andrei-Valentin Stirbu, Cédric Adjih. FIT-IRSA: Feedback-Integrated Two-Phase IRSA with Deep Reinforcement Learning. PEMWN 2025 - 14th IFIP/IEEE International Conference on Performance Evaluation and Modeling in Wired and Wireless Networks, Nov 2025, Paris, France. pp.1-10, 10.23919/PEMWN67312.2025.11304049 . hal-05469412

HAL Id: hal-05469412

https://hal.science/hal-05469412v1

Submitted on 21 Jan 2026

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
# FIT-IRSA: Feedback-Integrated Two-Phase IRSA with Deep Reinforcement Learning

Andrei-Valentin Stirbu

Ecole Polytechnique & Inria

andrei.stirbu@polytechnique.edu

Cédric Adjih

Inria, France

cedric adjih@inria.fr

**Abstract**—Efficient random access can be used in scenarios with a massive number of IoT devices. Among modern random access protocols, Irregular Repetition Slotted ALOHA (IRSA) offers excellent asymptotic performance (for large frame sizes), but its finite-frame efficiency is lower and difficult to optimize analytically. In this work, we introduce limited mid-frame feedback to better coordinate users and improve performance: Feedback-Integrated Two-phase IRSA (FIT-IRSA). We formulate IRSA with feedback as a deep reinforcement learning (DRL) problem. Using policy gradient methods, we learn transmission strategies that improve throughput under varying loads, as demonstrated in our simulation results. This provides a practical alternative to classical density-evolution-based optimization, which applies mainly to large frames.

**Index Terms**—grant-free, IoT, mMTC, IRSA

## I. INTRODUCTION

IoT is connecting billions of heterogeneous devices, ranging from sensors and smart meters to vehicles and mobile equipment, via wireless technologies. On the device side, many of these devices are constrained in terms of power, computation, and memory, while on the base station (BS) side, available spectrum is ultimately limited, which makes efficient communication protocols essential.

In some common scenarios, labeled as massive Machine-Type Communication (mMTC), and unlike traditional cellular usage, some IoT nodes are active only sporadically, transmitting very short packets compared to their long idle periods. In this context, the overhead of centralized scheduling or grant-based schemes of traditional cellular networks may outweigh the cost of transmitting the data itself.

To support such massive, sporadic, and resource-limited access, grant-free protocols in general, including some random access protocols or Non-Orthogonal Multiple Access (NOMA) schemes, have been proposed [1]. In such settings, random access schemes (even ALOHA-style) remain attractive for their simplicity, stateless operation, and compatibility with existing technologies that do not necessarily require new transmitter or receiver designs. However, the performance of classical random access is far from perfect (a bound being 0.587 packets/slot [2]).

One recent family of protocols, sometimes called modern random access, is particularly of interest for improving this

performance. One such protocol, Irregular Repetition Slotted ALOHA (IRSA) [3], is an improvement of Slotted ALOHA [4], [5], where users repeat their packets several times (as in Diversity Slotted ALOHA), and where the technique of Successive Interference Cancellation (SIC) is used to remove all copies of successfully received packets. IRSA can asymptotically reach a performance of 1 packets/slot, but IRSA works by frames, and this result is for asymptotically large frame sizes; for small frame sizes it is lower. For instance, there are reports of a maximum throughput of around 0.65 packets/slot for frame sizes of 50 slots (with heuristic optimization). Another line of work considers traditional tree-based random access protocols, with feedback for every slot, and incorporates the idea of SIC, as in SICTA [6], achieving an asymptotic performance of 0.693 packets/slot.

Observing that traditional random access protocols are about iteratively coordinating users using feedback, we introduce the idea of feedback to IRSA, but without going as far as operating slot-by-slot like SICTA. In a scenario where users transmit to a base station, in this work, we aim to make the access behavior adaptive. We split the frame into two subframes, and perform transmission in two phases. In the first phase, users select slots of the first subframe to transmit replicas of their packet as in classical IRSA (see Sec. II). After that, early decoding is performed at the base station. The result of the early decoding, on a per-slot basis, constitutes feedback that is sent back to the users. With this knowledge, the users perform another set of transmissions in the second part of the frame, and the base station decodes the two subframes as if it were a single IRSA frame.

Our key idea is that the given feedback is providing some information about the selections of the other users (as well as the already decoded slots), and can somehow help users experience fewer collisions. However, it is not a simple task to design an optimal selection of slots based on that feedback (since IRSA behavior is also not simple to analyze). In this work, we opt to entirely make this decision through AI techniques, thus entirely learning the random access protocol, through Deep Reinforcement Learning (DRL).

The rest of the article is organized as follows: in Sec. II, we give a brief overview and review the state of the art in IRSA. In Sec. III, we describe our approach and in Sec. IV how DRL is applied to our model. In Sec. V, we provide some experimental results through simulation (including training).

This work was partly supported by the ANR under the France 2030 program, grant "NF-PERSEUS: ANR-22-PEFT-0004", and by the PhC-Maghreb project 24MAG18 "ANGEL"
Finally in Sec. VI, we conclude.

## II. IRREGULAR REPETITION SLOTTED ALOHA (IRSA)

### A. Overview of Modern Random Access Techniques

What is sometimes called modern random access refers to protocols using grant-free access. They are often applied in the uplink for users communicating with a single base station (and indeed, the seminal [7] targeted satellite communications). They are based on three principles: 1) a fixed-size frame divided into slots; 2) on the user side, each packet is sent by transmitting exact copies in two or more slots; 3) on the receiver side, SIC is used, so that when one copy is received without collision, it is removed from all other slots.

Contention Resolution Slotted ALOHA (CRDSA) [7] is one of the early proposals: in CRDSA, each user repeats their packet twice over randomly chosen slots and the receiver applies SIC to peel collisions; a key idea is that in the header of each replica, the transmitter includes the positions of the two replicas: hence whenever the base station recovers one, it knows where the other copy is (to physically remove).

Instead of forcing the number of replicas to be exactly two, Irregular Repetition Slotted ALOHA (IRSA) [3] lets the user draw randomly from a fixed predefined distribution. In addition, the tool of density evolution from LDPC codes is introduced to estimate performance on asymptotically large frames. By numerically solving an optimization problem, degree distributions are found, under large frames, suitably optimized distributions approach a throughput near 0.97 packets/slot.

A rich literature on the topic exists, and we note that several works use AI techniques to optimize IRSA, including [8], [18], [19].

### B. IRSA System Model

IRSA [3] is a frame-based random access protocol, in which each frame has a fixed number of slots. We denote $M$ as the number of slots per frame, $N$ as the number of active users at the start of the frame, and $T$ as the mean/expected throughput (decoded users per frame). $M = 5$, $N = 4$, in the example of Fig. 1. As is common in parts of the IRSA literature, we assume a collision channel model, where the BS recovers a packet if and only if it is the only transmission in a slot.

**User side:** Each active user first selects a number of repetitions. Assume this number is $d$. Then they select exactly $d$ slots uniformly in the frame and send exact copies of their packets. In traditional IRSA, there is a preselected degree distribution $\Lambda = (\Lambda_i)_{i=2...D}$ that indicates the probability that the user will select the degree $d$: $\text{Pr}(\text{user transmits } d \text{ replicas}) = \Lambda_d$

**Base station side (detailed SIC):**

* Singleton decoding. The BS scans all slots and first decodes singletons (slots containing a single replica).

* Replica localization. Using replica pointers (e.g., from headers as in CRDSA) or the placement known from the graph view in IRSA, the BS identifies all other slots carrying replicas of the decoded packet.

* Waveform regeneration and subtraction. The BS re-encodes the decoded bits, remodulates them, and applies

<figure><img src="image_2.png" /><figcaption>Fig. 1: Example of simple grid with 5 slots and 4 active users</figcaption></figure>

the channel estimate per-slot (amplitude/phase, timing). The resulting baseband waveform is subtracted from each slot holding a replica (inter-slot SIC). This physically removes the packet's contribution to those collisions.

* Iterative decoding. Subtraction can turn previously collidied slots into new singletons. The BS repeats the singleton-regenerate-subtract cycle until no new singletons appear (fixed point).

**Example:** Fig. 1 represents an example of IRSA with 5 slots and 4 active users ($M=5, N=4$). They are in the same frame and frequency. SIC proceeds as a chain reaction: slot 2 is a singleton allowing the BS to recover A, so the BS decodes A and subtracts A's replicas from slots 1 and 4; slot 4 then becomes a singleton with C, which is decoded, and C is subtracted from slot 1 and slot 3; slot 1 now contains only D, so D is decoded and removed from slots 3 and 5; each of those slots is left with only B, which the BS decodes. Thus decoding goes as: $A \to C \to D \to B$, and all packets are recovered.

The performance of IRSA is highly dependent on the chosen degree distribution and the frame size. For asymptotically large frames, optimized degree distributions can approach theoretical throughput limits (close to one packet per slot). However, in practical finite-length regimes, stopping sets often appear, preventing full decoding and limiting performance. This motivates the exploration of learning-based methods to adaptively optimize IRSA in realistic scenarios.

## III. OUR APPROACH: FIT-IRSA

We consider a random access system with $N$ active users and a frame of $M$ slots. In classical IRSA, each user draws a degree $d$ from a degree distribution $\Lambda$, selects $d$ slots uniformly at random in the frame, and transmits replicas in those slots; the base station (BS) then applies successive interference cancellation (SIC) over the full frame.

FIT-IRSA modifies this process in two ways: (i) the frame is split into two subframes, and (ii) users choose which slots to use (not only how many), allowing mid-frame feedback to guide placements.

Let the frame be divided into two subframes, each of length $M/2$:

* Phase 1 (P1): Each active user selects a set of slots in the first subframe $\{1, ..., M/2\}$, and transmits replicas there.

* The BS then applies standard SIC restricted to the first subframe and classifies P1 slots into three states: EMPTY,
<figure><img src="image_3.png" /><figcaption>Fig. 2: State - Action - Output Representation</figcaption></figure>

DECODED (singletons or solved collisions), and UNDECODED (collision persists). The BS broadcasts a compact slot-state summary for the first subframe (one label per slot: EMPTY, DECODED, or UNDECODED).

* Phase 2 (P2): Each user then selects slots in the second subframe $\{M/2+1, \dots, M\}$. Users can use the broadcast feedback map to avoid obviously congested positions or to target likely singletons.

* Finally, the BS runs SIC on the entire frame (both subframes jointly). Users resolved in P1 remain resolved after the final pass; P2 transmissions mainly serve the still-undeveloped packets.<fnref n="1" />

We have a simpler variant of FIT-IRSA that we also implemented. It is called P1-IRSA, and serves as a comparison baseline: in this variant only the first phase is executed over all the M slots of the frame (it is identical to FIT-IRSA where the second phase would have a subframe of 0 slots).

Our approach improves on Deep-IRSA [18], where the model only outputs a degree distribution. It also simplifies DS-IRSA [19] by retaining the standard IRSA frame-and-slot structure and avoiding the additional minislot-based sensing phase.

## A. Interface, inputs, and outputs (algorithm-agnostic)

To keep the description implementation-agnostic, we view each user as running a generic selection rule that, given the information available at that moment, outputs a set of slots.

* **User-side inputs.** In P1: subframe length $M/2$. In P2: the BS slot-state map for P1 plus subframe length $M/2$.

* **User-side outputs.** A set of $d^{(p)}$ distinct slots within the current subframe ($p \in \{1, 2\}$).

* **BS responsibilities.** Run SIC after the end of P1 to produce the feedback slot-state map; after P2, run SIC on the full frame to produce final decoding.

# IV. APPLYING REINFORCEMENT LEARNING

We want to maximize throughput, that is, to maximize the average number of users decoded per frame. We do not have an algorithm that would turn mid-frame feedback into optimal slot selection. This is not straightforward. We therefore treat

the selection of the slots in P1 and P2 as a learned policy, through the use of Deep Reinforcement Learning (DRL).

To achieve this, we cast FIT-IRSA optimization as a Markov Decision Process (MDP), using the inputs and the outputs of Sec. III-A, as state and actions, respectively:

We model FIT-IRSA as a Markov decision process (MDP). Each episode is one frame (and has exactly two rounds within a frame). In P1, the agent randomly chooses slots of the subframe according to a stochastic transmission policy (where slots are selected individually). The BS, considered as part of the environment, then performs some early decoding, and feedback is sent to the agent. Then the agent observes the feedback (e.g., slot states after P1) and outputs stochastic transmission decisions on a per-slot basis for the second subframe. The objective is to maximize expected throughput after SIC. More formally:

## A. MDP Formulation

We consider an IRSA system with $N$ active users transmitting to a single base station (BS) over frames of $M$ slots. Each user selects a repetition degree and transmits that many replicas in (distinct) slots of the frame. The BS applies SIC to decode as many packets as possible. The normalized throughput is

$$T = \frac{\#\{\text{decoded users in a frame}\}}{M}.$$

IRSA is cast as a Markov Decision Process (MDP) as represented in Fig. 2, with:

* **State $s_t$:** compact user/BS feedback for the current frame, that is: per-slot labels $\{\text{decoded, empty, undeveloped}\}$ encoded as bitmaps, a summary of previous actions (which slots this user has already chosen in the first subframe), some source of entropy ("noise input", randomly drawn values) useful because the policy needs to be stochastic. And finally, an indicator for whether we are in P1 or P2. The feedback is filled with blank values in P1.

* **Action $a_t$:** a transmission decision per user, represented as a slot-level activation vector $a \in \{0, 1\}^M$. The stochastic policy outputs a probability vector $\pi_\theta(\cdot | s_t) \in [0, 1]^M$, where each entry is the probability that the corresponding slot is selected.

* **Reward $r_t$:** terminal frame reward equal to the fraction of decoded users after SIC, with a small penalty on unnecessary replicas.

<footnote n="1">¹Phase 2 transmissions are allowed for all users. Packets already resolved by P1 remain resolved after the final SIC over the whole frame, so any redundant P2 replicas by those users do not change their final status.</footnote>
Our objective now becomes learning a stochastic policy $\pi_{\theta}(a|s)$ that maximizes expected return across frames. Let $\pi_{\theta}$ be a stochastic policy with parameters $\theta$ and value function $V_{\phi}$ with parameters $\phi$. The objective is

$$J(\theta) = \mathbb{E}_{\tau \sim \pi_{\theta}} \left[ \sum_{t=0}^{T-1} \gamma^t r_t \right],$$

with $\gamma=1$ in our episodic setting. The reward and the stochastic optimization of the policy is described in the next section.

We have a single agent performing all the actions on behalf of each user (using only information available at the user), hence for FIT-IRSA, an episode consists of $2N$ steps: $N$ steps (slot selections for each of the $N$ users) for P1, followed by feedback computation, and $N$ additional steps, one per user, in P2.<fnref n="2" />

## B. Deep Reinforcement Learning Algorithms

We use Deep Reinforcement Learning because the action space is large. We train one single neural network model for all users. We experimented with two algorithms *Vanilla Policy Gradient*, VPG (also known as REINFORCE [9]), and *Proximal Policy Optimization* (PPO).

We found very similar performance; so we report VPG results (and used PPO only as an implementation cross-check).

## C. Vanilla Gradient Policy

For VPG, we updated the model through stochastic gradient ascent, through Monte Carlo estimates of the gradient. For gradient estimation, we used the policy-gradient theorem, which yields:

$$\nabla_{\theta} J(\theta) = \mathbb{E} \left[ \sum_{t=0}^{T-1} \nabla_{\theta} \log \pi_{\theta}(a_t|o_t) A_t \right],$$

where $A_t$ is an *advantage* estimate, that is the reward $r_t$ minus a baseline $b_t$ computed within each batch (reducing variance), and $T$ is the number of subframes, i.e. $T = 2$.

We use our objective function, the number of decoded users, essentially as our reward. This reward is used after the end of P2 for all actions (P1 and P2) of all users. Precisely:

$$r_T = R_{thr} - \eta R_{rep}, \quad R_{thr} = \frac{\#\text{decoded users}}{N},$$

where $R_{rep}$ penalizes unnecessary replicas to encourage scarcity, with $\eta \ge 0$ small. This reward shaping biases the policy toward “just-enough” repetitions while maximizing throughput. In our results of Sec. V, we selected $\eta = 0.02$ and we observed a maximum difference of only 0.7% in the throughput compared to the case $\eta = 0$.

## D. Proximal Policy Optimization (PPO)

PPO improves stability and sample efficiency by constraining each policy update to remain proximal to the behavior policy via a clipped surrogate objective. Let

$$r_t(\theta) = \frac{\pi_{\theta}(a_t|o_t)}{\pi_{\theta_{old}}(a_t|o_t)}.$$

The clipped objective is

$$\mathcal{L}^{\text{CLIP}}(\theta) = \mathbb{E} \left[ \min \left( r_t(\theta) \hat{A}_t, \text{clip}(r_t(\theta), 1-\epsilon, 1+\epsilon) \hat{A}_t \right) \right], \\ \text{augmented by value and entropy terms:}$$

$$\max_{\theta} \mathcal{L}^{\text{CLIP}}(\theta) - c_v \mathbb{E} \left[ (V_{\phi}(o_t) - \hat{R}_t)^2 \right] + c_{\text{ent}} \mathbb{E} \left[ H(\pi_{\theta}(\cdot|o_t)) \right], \\ \text{optimized over several epochs of shuffled mini-batches from the same batch of trajectories, with gradient-norm clipping.}$$

We tried PPO for three reasons. First, its clipped objective helps to keep training stable by preventing overly large policy updates when the estimates are noisy. Second, PPO reuses the same batch of trajectories for several training passes, making it more sample-efficient than VPG, which updates only once per batch. Third, it keeps policy changes within a safe range without the extra complexity of KL penalties or costly line searches, which works well for our masked, combinatorial action space (degree selection and slot placement).

## V. RESULTS

We are interested in seeing if, and how much, our feedback scheme may help improve IRSA performance. Therefore we compare two scenarios: a scenario with our FIT-IRSA, with $N$ users and two subframes of $M$ slots (2-phase protocol); and a scenario with P1-IRSA, with $N$ users in a frame of $2M$ slots (1-phase protocol). Parameters are such that if feedback is not exploited, the performance would be identical.

Note that it is not obvious a priori whether the feedback information could be used: one might think that a node which has already been recovered in the first phase, could avoid transmitting in the second phase: but this does not change performance, as with SIC, all their replicas would be removed in the second phase anyway.

We evaluate the proposed learning-based random access on a Python IRSA simulator with SIC. We implemented the policy as a feedforward neural network with two hidden layers of 128 ReLU units each, followed by a linear output layer. Preliminary experiments with alternative depths and widths did not yield noticeable improvements.

Given a frame (users $N$, slots $M$), the simulator constructs transmissions, applies iterative SIC under a collision channel, and records the number of decoded users. Unless noted otherwise, we report means over independent runs (four random seeds); the vertical whiskers on each bar show 99% confidence intervals for one given seed.

For any choice of parameters ($N$ and $M$), we run episodes, training the model for 1000 batches of 1000 episodes, and report statistics computed over the last 1/10 of the training (hence 100,000 episodes). The clearer bar is for FIT-IRSA, and the darker bar (of the same color) is for P1-IRSA.

<footnote n="2">²For P1-IRSA we have only the $N$ steps for a single frame.</footnote>
<figure><img src="image_4.png" /><figcaption>Fig. 3: Mean decoded users versus number of users for a fixed frame length ($M = 20$). FIT-IRSA (with 2 phases) consistently dominates P1-IRSA (1 phase) across seeds; whiskers denote 99% confidence intervals</figcaption></figure>

### A. Fixed Frame Length ($M = 20$ slots)

Fig. 3 shows the mean decoded users as the load grows ($N = 11-30$). We see that first, there is a modest increase in the maximum throughput reached; about 11.93 with 15 users ($\approx 60\%$ of $M$) vs around 11.68 with classical IRSA.

Across all seeds, the two-phase variant consistently outperforms the single-phase baseline. The gain is modest at light loads and becomes more visible around the knee of the curve (e.g., $N \approx 17-19$), where two-phase improves the mean decoded users by roughly 1-5% relative to one-phase. Performance peaks near $N \approx 14-16$ (about 11.7-11.9 decoded on average) and then gradually declines as collisions increase with load.

### B. Throughput with Matched Frames ($M = N$)

We also report normalized throughput $T = \frac{\#\text{decoded users}}{N}$ as the number of users varies ($N = 2-30$) with an equal number of slots ($M = N$) in Fig. 4. The question is how quickly IRSA converges to 1 packets/slot (the asymptotic performance of IRSA for infinitely long frames), and how effective is FIT-IRSA in closing the gap.

Across all $N$, the FIT-IRSA (two-phase policy) attains higher mean throughput than the one-phase baseline P1-IRSA. The absolute gap is largest at very small loads (e.g., $N = 2$ shows an improvement on the order of 0.04-0.10) and then settles to a consistent but modest margin ($\approx 0.01-0.03$) as $N$ increases. The throughput curve starts high at $N = 2$, dips around $N \in \{4, 6\}$, and then exhibits an upward trend toward $N = 30$. This rise is consistent with asymptotic IRSA analysis: with offered load $G = N/M \to 1$ as $(N, M) \to \infty$, for which finite-length

penalties diminish and peeling/SIC approaches its threshold behavior predicted by density evolution.

Globally we see that FIT-IRSA closes about 5-10% of the gap between the IRSA performance and the bound of 1 packets/slot, which is significant but leaves room for further improvement.

## VI. CONCLUSION AND FUTURE WORK

We have studied an improvement of the classical IRSA modern random access protocol, specifically for small frame sizes.

We proposed and studied FIT-IRSA: a learning-based two-phase random access policy built on IRSA with SIC and trained and evaluated it on a Python simulator.

Although there is no obvious way to exploit the information obtained by the feedback, we have shown that across all tested settings, the learned two-phase scheme FIT-IRSA delivers consistent gains over a one-phase baseline (classical IRSA, P1-IRSA).

We note that the gains are only about 5-10% of the gap between classical IRSA and the absolute bound of 1 packets/slot, although the theoretical limit remains unclear.

Further options can be considered to improve the performance: starting from different neural network model architectures and capacity. It is also possible to make other design choices that explicitly partition the frame budget, for instance: a k-phase policy that splits the M-slot frame into $k \ge 2$ subframes of size $M/k$ each, with all users allowed to transmit in every subframe and SIC performed once over the concatenated frame.
<figure><img src="image_5.png" /><figcaption>Fig. 4: Mean normalized throughput $T = \#\text{decoded}/N$ versus number of users with matched frames ($M = N$), no sensing. Whiskers denote 99% confidence intervals.</figcaption></figure>

Or alternatively, a two-phase variant with a learnable split $x \in (0, 1)$ that allocates $xM$ and $(1-x)M$ slots to the two phases, respectively.

On the analysis side, another interesting direction would be to derive a bound on the achievable performance of IRSA for small frame sizes, either with classical IRSA or with FIT-IRSA. Additional directions include integrating priority classes and energy-aware repetition control, and testing robustness under non-ideal channels (capture, fading, imperfect SIC).

## REFERENCES

[1] Muhammad Basit Shahab, Rana Abbas, Mahyar Shirvanimoghaddam, and Sarah J. Johnson. "Grant-Free Non-Orthogonal Multiple Access for IoT: A Survey." IEEE Communications Surveys & Tutorials, 22(3):1805-1838, 2020.

[2] B. S. Tsybakov and V. A. Mikhailov, "An Upper Bound to the Capacity of Random Multiple Access Systems," *Problems of Information Transmission*, vol. 17, no. 1, pp. 90-95, 1981.

[3] G. Liva, "Graph-based analysis and optimization of contention resolution diversity slotted ALOHA," *IEEE Trans. Commun.*, vol. 59, no. 2, pp. 477-487, Feb. 2011.

[4] N. Abramson, "The ALOHA System—Another alternative for computer communications," in *Proc. AFIPS FJCC*, 1970, pp. 281-285.

[5] L. G. Roberts, "ALOHA packet system with and without slots and capture," *SIGCOMM Comput. Commun. Rev.*, vol. 5, no. 2, pp. 28-42, Apr. 1975.

[6] Y. Yu and G. B. Giannakis, "High-throughput random access using successive interference cancellation in a tree algorithm," IEEE Transactions on Information Theory, vol. 53, no. 12, pp. 4628-4639, Dec. 2007.

[7] E. Casini, R. De Gaudenzi, and O. del Rio Herrero, "Contention Resolution Diversity Slotted ALOHA (CRDSA): An enhanced random access scheme for satellite access packet networks," in *Proc. IEEE ICC*, 2007, pp. 2828-2835.

[8] E. Nisioti and N. Thomos, "Fast Q-learning for improved finite length performance of IRSA," *IEEE Trans. Cogn. Commun. Netw.*, vol. 6, no. 1, pp. 228-241, Mar. 2020.

[9] R. J. Williams, "Simple statistical gradient-following algorithms for connectionist reinforcement learning," *Machine Learning*, vol. 8, pp. 229-256, 1992.

[10] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, "Proximal Policy Optimization Algorithms," *arXiv:1707.06347*, 2017.

[11] E. Paolini, C. Stefanovic, G. Liva, and P. Popovski, "Coded random access: How coding theory helps to build random access protocols," *IEEE Commun. Mag.*, vol. 53, no. 6, pp. 144-150, Jun. 2015.

[12] A. Graell i Amat and G. Liva, "Finite length analysis of irregular repetition slotted ALOHA in the waterfall region," *arXiv:1803.01368*, 2018.

[13] S.-L. Shieh and S.-H. Yang, "Enhanced irregular repetition slotted ALOHA under SIC limitation," *IEEE Trans. Commun.*, vol. 70, no. 4, pp. 2268-2283, Apr. 2022.

[14] J. Sun, R. Liu, Y. Wang, and C. W. Chen, "Irregular Repetition Slotted ALOHA with Priority (P-IRSA)," in *Proc. IEEE Vehicular Technology Conference (VTC-Spring)*, 2016, pp. 1-5. doi:10.1109/VTCSPRING.2016.7504302.

[15] Z. Chen, Y. Feng, C. Feng, L. Liang, Y. Jia, and T. Q. S. Quek, "Optimal distribution design for IRSA with multi-packet reception," *arXiv:2110.08166*, 2021.

[16] Z. Chen, Y. Feng, Z. Tian, Y. Jia, M. Wang, and T. Q. S. Quek, "Energy Efficiency Optimization for Irregular Repetition Slotted ALOHA-Based Massive Access," *IEEE Wireless Communications Letters*, vol. 11, no. 5, pp. 982-986, May 2022. doi:10.1109/LWC.2022.3151931.

[17] H. Jia, Z. Ni, C. Jiang, L. Kuang, S. Guo, and J. Lu, "Enhanced IRSA with degree distribution adjustment in satellite network," in *Proc. IEEE GLOBECOM*, 2019.

[18] I. Ayoub, I. Hmedoush, C. Adjih, K. Khawam, and S. Lahoud, "Deep-IRSA: A deep reinforcement learning approach to IRSA," in *Proc. IFIP PEMWN*, Oct. 2021, pp. 1-6.

[19] I. Hmedoush, P. Gu, C. Adjih, P. Mühlethaler, and A. Serhrouchni, "DS-IRSA: A deep reinforcement learning and sensing based IRSA," in *Proc. IEEE GLOBECOM*, 2023, pp. 1-6.

[20] S. Jadoon, R. Khan, and S. Kashif, "Deep reinforcement learning for random access in machine-type communication," in *Proc. IEEE WCNC*, 2022, pp. 1-6.

[21] R. S. Sutton and A. G. Barto, *Reinforcement Learning: An Introduction*, 2nd ed. MIT Press, 2018.
[22] P. Diaconis and D. Freedman, “Finite exchangeable sequences,” *Annals of Probability*, vol. 8, no. 4, pp. 745–764, 1980.

## APPENDIX A
### SUPPLEMENTAL MATERIAL

In this section we provide additional results complementing Sec.V.

1) *Model Depth*: The first question is whether deeper per-user policies yield higher throughput, since the policy network in FIT-IRSA is a feedforward MLP. To check this, we fixed the scenario to 15 users and 20 slots (two-phase FIT-IRSA) and retrained policies with $L \in \{1, \dots, 7\}$ hidden layers of width 128, following the exact training/measurement procedure from Sec.V. Figure 5 reports the mean decoded users at convergence; each bar corresponds to one random seed.

The outcome is that, for this operating point, depth has only a minor effect on the achieved throughput. Even a single hidden layer (essentially a linear policy with ReLU features) matches the deeper models (within the estimated confidence intervals). Exploring whether deeper networks help for much larger frames or under different load regimes remains an open question.

<figure><img src="image_6.png" /><figcaption>Fig. 5: Mean decoded users vs. number of hidden layers for FIT-IRSA at 15 users, 2 × 10 slots (one seed).</figcaption></figure>

2) *Scalability and Convergence*: The experiments of Fig. 4 already show that both FIT-IRSA and its one-phase baseline can be trained reliably (even if the true optimum is unknown). Motivated by this, we increased the frame size further and trained both policies with $N = M \in \{30, 40, \dots, 100\}$ users/slots, still using two subframes of $M/2$ slots for FIT-IRSA so that the total number of slots equals the one-phase configuration. All runs use 1000 epochs of 1000 episodes, transmission cost 0.02, and the same metrics as in Sec. V. The resulting normalized throughput (decoded packets per slot) is plotted in Fig. 6.

Throughput grows steadily yet sublinearly with the frame length. FIT-IRSA improves from 0.609 packets/slot at $N = 30$ to 0.702 at $N = 100$, while P1-IRSA moves from 0.585 to 0.686. The gap between the two curves remains around 0.015–0.025 packets/slot throughout, and at $N = 100$, FIT-IRSA reaches 0.702 packets/slot<fnref n="3" />. The one-phase curve at $N = M = 50$ reaches 0.628 packets/slot, fully consistent with

<figure><img src="image_7.png" /><figcaption>Fig. 6: Mean normalized throughput (decoded packets per slot) versus frame length for matched-frame runs with $N = M$ and transmission cost 0.02 (seed 31). Bars show 99% confidence intervals over the last tenth of training.</figcaption></figure>

the 0.62–0.63 range in [18, Fig. 7], observed for respectively both Deep-IRSA (a model that selects the degree) and degree-distribution optimization by density evolution, for this exact setting. This highlights that slot selection does not necessarily surpass the best degree selection, although feedback+slot selection as in FIT-IRSA does, on the other hand (0.650).

Perhaps the one of the interesting findings is that we obtain this good behavior even with vanilla policy gradient: every update uses a single scalar reward per episode, even though an episode comprises up to 100 slot-selection decisions for each of the 100 users. Despite this long credit-assignment chain, the learning signal remains stable and both policies converge reliably, as illustrated by the training curves for $N = M$ and frame size 40, 70, and 100 respectively, in Fig. 7. Note that

<figure><img src="image_8.png" /><figcaption>Fig. 7: Training curves (mean decoded users per frame) versus episodes for matched-frame runs with $N = M \in \{40, 70, 100\}$. Each point is the mean over the episodes in one epoch (1000 episodes); the x-axis counts episodes (epoch index ×1000). Both FIT-IRSA and the baseline P1-IRSA show stable learning and convergence.</figcaption></figure>

we start with a learning rate equal to 0.01 and it is halved every 100 epochs.

3) *Poisson Arrivals*: In the previous results, for a given IRSA configuration we fixed the number of users and slots. This is not realistic for massive random access, where each user independently decides to transmit. A Poisson model is

<footnote n="3">³Incidentally, this exceeds the 0.693 packets/slot reported for SICTA [6].</footnote>
more natural (more complex models with correlated events are also possible). To investigate this, we train with a fixed Poisson arrival rate $\mu$: at the beginning of each episode, the number of users is drawn from Poisson($\mu$) and then FIT-IRSA (resp. P1-IRSA) is run.<fnref n="4" />

<figure><img src="image_9.png" /><figcaption>Fig. 8: Mean decoded users versus average number of users for a fixed frame length ($M = 20$), and Poisson arrivals. FIT-IRSA (with 2 phases) consistently dominates P1-IRSA (1 phase) across seeds; whiskers denote 99% confidence intervals</figcaption></figure>

For Poisson traffic, Fig. 8 gives the wide-range counterpart of Fig. 3, and Fig. 9 is the equivalent of Fig. 4. Throughputs are slightly lower overall, which is natural because the number of contending users now fluctuates from frame to frame hence the optimal policy should change. As the load increases in Fig. 8, both FIT-IRSA and P1-IRSA flatten out, yet FIT-IRSA maintains up to roughly a 10% advantage over P1-IRSA starting at the knee of the curve. The scenario of increasing frame size in Fig. 9 shows the same ≈ 10% margin.

<figure><img src="image_10.png" /><figcaption>Fig. 9: Mean normalized throughput $T = \#decoded/N$ versus average number of users (Poisson arrivals) with matched frames ($M = N$), no sensing. Whiskers denote 99% confidence intervals.</figcaption></figure>

4) *Analysis of the Policy Models:* We also performed a more detailed post-hoc analysis of the trained policy in the first phase. Instead of sampling actions, we inspected the raw probability vector over slots output by the policy network.

For almost all trained models (both FIT-IRSA and P1-IRSA), the empirical distribution of these probability vectors is well approximated by the following sampling procedure, which we call latent sampling procedure:

* There exists a latent distribution $\mathcal{D}$ on $[0, 1]$.

* To generate a policy vector for a given input, we first draw a latent parameter $\theta \sim \mathcal{D}$.

* Given $\theta$, the policy assigns (approximately) the same probability $\theta$ to each of the $M$ slots, i.e., the slot indicators behave like i.i.d. Bernoulli($\theta$) and the probability vector is close to $[\theta, \theta, \dots, \theta]$.

In hindsight, this pattern is closely related to classical results by de Finetti (see, e.g., [22]) on symmetric collections of random variables. Informally, if a large family of binary random variables does not depend on the order in which we list them (their joint behavior is essentially unchanged under permutations of the indices), then they can often be described as follows: first draw a hidden parameter $\theta$, and then, given this $\theta$, draw many independent Bernoulli($\theta$) variables. This is the latent sampling procedure.

This matches closely what we observed and described above (even though we are in a finite setting): our hypothesis is that, at least in the first phase, the trained models do not end up making much distinction between slots; they preserve the symmetry already present in the initial framed random access model, where one can freely permute the slots. As a result, slot permutations remain essentially irrelevant, and this approximate symmetry is precisely why we observe “de Finetti”-like behavior.

Observe also that this is compatible with degree distributions: only the number of selected slots matters and the chance of seeing $k$ active slots is the same no matter which slots they are.

For the case of 30 users and 30 total slots (P1-IRSA) and, separately, 30 users with 15 slots in the first phase (FIT-IRSA), we drew one million outputs from the final policy and computed summary statistics. The results are presented in Fig. 10.

We treat the policy as if it were generated by the latent sampling procedure and estimate the latent distribution $\hat{\mathcal{D}}$ from the expirical distribution of the mean slot probability. The results are in the top row of Fig. 10. Notice that we observed that the $\hat{\mathcal{D}}$ seem to widely vary depending on the parameters in general.

Furthermore, to give an estimate of the deviations from the latent sampling procedure, we compare each slot's probability to the per-sample mean (which would be exactly equal under the latent model). The resulting ratios (expected to be near 1) are summarized by this mean but also by their quantiles (50%, 80%, 90%, 98%), as shown in Fig. 10 (bottom row). Because there is no a priori structure in the slot order (including during training), slots are sorted by their means.

We can see that in both cases (P1-IRSA, FIT-IRSA), the mean probability of any given slot differs by at most about 5% from the overall mean across slots, i.e., it is quite close to the latent sampling procedure. On the other hand, the variation for low quantiles can be rather high (e.g. 30–50%).

<footnote n="4">⁴During training we ignore episodes with 0 users. In post-processing, we compensate by rescaling the measured throughput using the empirical ratio $\mu/\bar{N}$, where $\bar{N}$ is the mean number of active users recorded in the logs (thus removing the bias from conditioning on $N > 0$).</footnote>
<figure><img src="image_11.png" /><figcaption>(a) P1-IRSA: $\hat{D}$ (mean slot prob)</figcaption></figure>

Per-slot envelopes (order-invariant, sample-mean normalized)

<figure><img src="image_12.png" /><figcaption>(c) P1-IRSA: per-slot envelopes</figcaption></figure>

<figure><img src="image_13.png" /><figcaption>(b) FIT-IRSA, first phase: $\hat{D}$ (mean slot prob)</figcaption></figure>

Per-slot envelopes (order-invariant, sample-mean normalized)

<figure><img src="image_14.png" /><figcaption>(d) FIT-IRSA, first phase: per-slot envelopes</figcaption></figure>

Fig. 10: Empirical latent distribution $\hat{D}$ (top) and per-slot envelopes (bottom) for P1-IRSA (30 slots, 30 users) and the first phase of FIT-IRSA (15 slots/phase, 30 users).

To check whether this matters, we focused on P1-IRSA and compared throughput under the empirically estimated $\hat{D}$ (using the latent sampling procedure) versus the policy model itself (evaluated on 100,000 frames):

* Latent sampling procedure, mean throughput: 0.5854

* Policy model, mean throughput: 0.5852

Thus, in this case, the impact appears negligible (difference of less than 0.05% here). We repeated the same check on some other runs of P1-IRSA, and the conclusion held.

5) *Analysis of Two FIT-IRSA Policies (Different Random Seeds)*: We also examined the Phase 2 policy learned by FIT-IRSA. For the same configuration ($u = 8$ users and $2 \times 4$ slots per frame), we observed noticeably different policy distributions across two independent runs (different random seeds), even though their final normalized throughputs were almost identical (less than a 0.4% difference, i.e. 0.5406 and 0.5424 respectively).

Figure 11 shows histograms of slot selection probabilities for Phase 1 and Phase 2. For each phase, we generated $10^6$ policy vectors. Each policy vector assigns a probability to each slot; we then collect all these per-slot probability values from all vectors into a single list and plot their histogram (i.e., we ignore the slot index and treat all slots together; they are *pooled*).

In Phase 1, the policies of the two runs already differ substantially: in Run A, the pooled probabilities are spread over a wide range (roughly beyond [0.15, 0.23]), whereas in

Run B they concentrate in a very narrow interval (approximately [0.202, 0.205]).

In Phase 2, the distributions are more similar overall, but Run B exhibits more pronounced high-probability spikes. Importantly, in both runs the Phase 2 policy frequently assigns higher probability to some slots (e.g., above 0.4, and occasionally above 0.6). Very low probabilities (including values close to 0) also occur relatively often, as indicated by the mass near 0 on the x-axis; this behavior is much less pronounced in Phase 1.

This suggests that, based on the feedback, the Phase 2 policy sometimes makes near-deterministic choices on certain slots—effectively deciding not to transmit. That echoes the emergence of learned protocols seen in DS-IRSA [19, Fig. 9]. However, unlike DS-IRSA, we rarely observe the opposite pattern here (a single slot being “reserved” with near-one transmit probability for one user and near-zero for others). Possible reasons include: (1) FIT-IRSA, as used here, has only two phases (one interaction opportunity), whereas DS-IRSA has many minislot interactions during its sensing phase; (2) FIT-IRSA’s feedback is ternary (decoded/empty/undecoded) and therefore carries less information than DS-IRSA’s richer sensing feedback.

In DS-IRSA, some sensing outcomes create an implicit consensus that effectively reserves a slot for one user: that user transmits with probability close to one, while the others transmit with probability close to zero (or they are guaranteed
<figure><img src="image_15.png" /><figcaption>(a) Phase 1 (pooled), Run A</figcaption></figure>

<figure><img src="image_16.png" /><figcaption>(b) Phase 2 (pooled), Run A</figcaption></figure>

<figure><img src="image_17.png" /><figcaption>(c) Phase 1 (pooled), Run B</figcaption></figure>

<figure><img src="image_18.png" /><figcaption>(d) Phase 2 (pooled), Run B</figcaption></figure>

<caption>Fig. 11: All-slot pooled slot-probability histograms for a FIT-IRSA configuration, with 8 users and 2 × 4 slots, for two independent runs (different random seeds), shown separately for Phase 1 and Phase 2.</caption>

to be recovered via their own reserved slots and subsequent
SIC). We do not observe this “slot reservation” phenomenon
in these FIT-IRSA runs. Although, upon inspection of the
DS-IRSA runs, we observed that slot reservation tended to
occur when a node was the sole transmitter in a minislot,
the corresponding case in FIT-IRSA would imply that the
packet is recovered and therefore requires neither second-phase
transmissions nor slot reservations. An open question is whether
it could emerge with more iterations or with richer feedback

(e.g., energy measurements on collided slots).

Finally, overall, the Pearson correlation coefficients computed
from sampled Phase-2 slot probabilities remain high in both
runs (RunA and RunB), ranging from 0.971 to 0.999; see
Fig. 12 for the corresponding joint-probability heatmaps (2D
histograms) of these pairs. Importantly, we do not observe any
"low-vs-high" occurence (i.e., one slot being selected with high
probability while the other is low).

<figure><img src="image_19.png" /><figcaption>(a) Run A: slots 1 vs 3 (corr 0.999)</figcaption></figure>

<figure><img src="image_20.png" /><figcaption>(b) Run A: slots 0 vs 2 (corr 0.971)</figcaption></figure>

<figure><img src="image_21.png" /><figcaption>(c) Run B: slots 1 vs 2 (corr 0.975)</figcaption></figure>

<caption>Fig. 12: Phase 2 joint slot-probability heatmaps (2D histograms) for two runs (different seeds) of the same FIT-IRSA configuration. Run A shows slot pairs (1, 3) and (0, 2); Run B shows slot pair (1, 2).</caption>