# AoI Minimization in Mixed Traffic Full-Duplex Uncoordinated Communication Systems with NOMA

Joseph Doumit, Marie-Josepha Youssef, Charbel Abdel Nour, Joumana Farah, Catherine Douillard

<figure><img src="image_1.png" /></figure>

## To cite this version:

Joseph Doumit, Marie-Josepha Youssef, Charbel Abdel Nour, Joumana Farah, Catherine Douillard. AoI Minimization in Mixed Traffic Full-Duplex Uncoordinated Communication Systems with NOMA. IEEE Internet of Things Journal, 2023, 10.1109/JIOT.2023.3296437. [hal-04168014v2](https://hal.science/hal-04168014v2)

HAL Id: hal-04168014

https://hal.science/hal-04168014v2

Submitted on 9 Aug 2023

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# AoI Minimization in Mixed Traffic Full-Duplex Uncoordinated Communication Systems with NOMA

Joseph Doumit, Student Member, IEEE, Marie-Josepha Youssef, Charbel Abdel Nour, Senior Member, IEEE, Joumana Farah, Senior Member, IEEE, Catherine Douillard, Senior Member, IEEE

**Abstract**—The number of Internet of Things (IoT) applications and devices has been continuously increasing and is expected to continue to do so in the upcoming years. In this context, we study the uplink uncoordinated spectrum and power allocation problems in a non-orthogonal multiple access (NOMA) IoT network while minimizing the age of information (AoI). Power-limited IoT devices aim at auto-organizing their transmissions on available resources, i.e., the subband-power level pairs, without any coordination between them. To enhance spectral efficiency, full-duplex relays are deployed with queuing capabilities. To solve the problem of minimizing the AoI in both the coordinated and uncoordinated communication settings, we propose algorithms based on matching theory and the multi-armed bandit (MAB) framework respectively. Simulation results show that the proposed uncoordinated MAB-based solution achieves full rate satisfaction for the users. Moreover, it reveals to be very efficient in minimizing the system AoI, transmit power and overhead compared to the coordinated solution.

**Index Terms**—Non-orthogonal multiple access, uncoordinated system, relays, full duplex, power allocation, age of information, matching theory, multi-armed bandits.

## I. INTRODUCTION

THE Internet of Things (IoT) has become a major driving use case in the evolution of wireless communication networks. In fact, IoT devices are expected to make up most of the mobile traffic exchanged in fifth generation (5G) and beyond 5G (B5G) networks. They are being deployed to enable different applications spanning various sectors such as e-health, smart agriculture, autonomous cars. With the large diversity of target IoT applications, the characteristics and requirements of the generated mobile traffic are becoming increasingly heterogeneous. Indeed, depending on the application, the traffic can be continuous, periodic or sporadic. In addition, with massive device deployment scenarios, the number of transmitted IoT packets may induce unacceptable latency. Therefore, the network should adopt topologies and resource allocation strategies designed to meet the low-latency and low-complexity requirements of target applications and pave the way for scalable and self-configurable massive IoT systems [1], [2]. These constraints necessitate the design of appropriate novel frameworks to optimize the performance of

modern wireless communication systems, since classical centralized optimization techniques fail to address the scalability requirement and the continual evolution of IoT networks.

To enhance the capacity of the system, non-orthogonal multiple access (NOMA) has been introduced as a promising multiple access technique to accommodate a larger number of devices. Specifically, power-domain NOMA (PD-NOMA) consists of scheduling multiple users on the same resource block by multiplexing them in the power domain [3]–[6]. Therefore, NOMA can achieve a higher spectral efficiency and higher data rates when compared to orthogonal multiple access (OMA) schemes [7]. To decode signals at the receiver level, successive interference cancellation (SIC) is employed. Most of the early works studying NOMA focused on downlink scenarios. While some work studied the maximization of the average achieved throughput [3], others focused on minimizing the average downlink transmission power while satisfying user rate requirements [4], [6]. In [5], the authors showed an enhancement of the spectral efficiency using novel mutual SIC techniques. To ensure successful decoding at the receiver side in a NOMA system, power allocation is key. Hence, a variety of power allocation schemes were proposed in the literature. While some are based on a back-off step between power levels [8] to safely decode signals, others focus on an allocation scheme that aims at guaranteeing the minimum user rate requirements [9]. As for uplink communication systems, the authors in [10] addressed the challenge of maximizing the minimum user rate by jointly optimizing the transmit powers and decoding order. Others showed that adding a new precoding block before the subcarrier mapping may help in reducing peak to average power ratio (PAPR) [11]. Moreover, many studies showed that NOMA applied whether in uplink or downlink communication systems ensures low latency communication and massive connectivity [12], rendering it a promising technique to support IoT network communications [2]. In this paper, we consider NOMA for an uplink IoT network communication system.

In the case of NOMA coordinated communication systems, matching theory has been commonly applied to solve optimization problems [13]. In fact, matching theory is an economic mathematical framework [14] that aims to facilitate the formation of mutually beneficial relationships. It is employed to address assignment problems between two distinct sets of players. The fundamental challenge of the matching problem is to assign resources from players of one set to those of the other set in an optimal manner. To achieve this, each player

M. J. Youssef is with Capital Fund Management, Paris, France (email: mariejosephay@gmail.com).

J. Farah is with Univ Rennes, INSA Rennes, CNRS, IETR-UMR 6164, F-35000 Rennes, France (email: joumana.farah@insa-rennes.fr).

J. Doumit, C. Abdel Nour and C. Douillard are with IMT Atlantique, Lab-STICC, UMR CNRS 6285, F-29238 Brest, France, (email: firstname.lastname@imt-atlantique.fr).
builds a preference list based on a utility function, in order to rank players of the opposing set. In the context of this paper, the first set of players corresponds to the devices, while the other corresponds to the available network resources [15].

Added to NOMA, relaying can be quite useful in massive communication settings to reduce network congestion at the central base station (BS) level, extend cellular coverage and increase achieved rates. Mobile relays that can be deployed in a quick manner, such as unmanned aerial vehicles, can also re-store communication in damaged networks infrastructure [16], [17]. In order to achieve high data rates, reduce latency and enhance spectral efficiency, full-duplex relays can be considered. By allowing the simultaneous transmission and reception of messages on the same frequency resource, full-duplex relays achieve a lower communication latency but suffer from self interference [17]. The joint use of NOMA and full-duplex relay techniques allows for enhanced spectrum efficiency and massive connectivity through multiple transmissions on the same resource block [18]. Furthermore, to prevent data loss in case of system congestion, relays can be equipped with queuing capabilities. Queuing was mostly considered in the literature in coordinated spectrum access settings, where users receive, in addition to their messages, information about the relay buffer size [19]. On the other hand, few works have considered queuing in an uncoordinated manner, where no information about the relay buffer state is communicated to users. However, whether applied in a coordinated or uncoordinated setting, queuing has the drawback of increasing the age of information (AoI) [20]. AoI is a metric referring to the degree of freshness of the information that quantifies the time elapsed from the source generation of the message till its reception [21]. AoI can vary depending on network congestion, transmission delays, and the efficiency of information processing and delivery mechanisms. It is usually measured at the receiver end, but can also be measured at the transmitter if the latter receives acknowledgments from the former. The higher the AoI, the older the information and the less useful it is [22]. Several works studied the optimization of AoI in downlink systems [23], others studied the optimization of a queue depending on its size and packet replacement [24]. Moreover, many studies have compared the AoI for different types of queues, e. g., for first-come first-served (FCFS), last-come first-served (LCFS) with or without preemption, or proportional fair (PF) queues [25]. In [26], the average AoI was analyzed and optimized for update rates in simple queuing systems like M/M/1, M/D/1, and D/M/1. This analysis sparked further research on both peak and average AoI in various queuing systems [24], [27]–[31]. Many of these studies focused on optimizing peak and average AoI by adjusting the update service rates. Additionally, researchers investigated queue scheduling disciplines and packet management strategies to minimize AoI. The benefits of employing parallel servers to improve AoI were demonstrated in [32]. Furthermore, improvements in AoI were observed by reducing buffer sizes and introducing packet deadlines, where a packet deletes itself after exceeding its deadline, as shown

in [27], [24] and [28], respectively. The consideration of AoI in updates traversing a network of queues was addressed in [33]. It was found that the LCFS queue scheduling discipline, with preemptive service, is age-optimal when the service times follow an exponential distribution. In this paper, we use AoI as a design criterion, especially for latency-sensitive applications.

On a different note, enabling scalable networks that can accommodate a variable numbers of devices and traffic types is crucial for unleashing the full potential of IoT. Uncoordinated or grant-free spectrum access strategies can be well-suited to this context. Indeed, uncoordinated strategies gained great interest in recent literature [34], [35] motivated by their ability to reduce signaling overhead and latency in communication networks. However, most of the available literature on uplink NOMA considered fully coordinated spectrum access. Only few works considered either partial or full uncoordinated spectrum access [36], [37]. Dealing with uncoordinated spectrum access requires novel techniques such as reinforcement learning or game theory to allocate available resources, since information sharing is restricted. Several studies adopted reinforcement learning [38]–[40], and more specifically the multi-armed bandit (MAB) framework, to allow users to auto-organize their transmissions on the available channels [38]. In the MAB framework, the users, referred to as players, compete to find the actions, i.e., resources called arms, that maximize their expected reward or welfare, usually their rate [41]. Arm rewards are unknown beforehand at the player level, hence an exploration phase is needed to estimate them. Then, the system can proceed to the exploitation phase where the particular arms that maximize the achieved rewards are continuously played by users. Multiple algorithms have been proposed to solve the MAB problem. They include the ε-greedy, ε-decreasing and upper confidence bound (UCB) algorithms [39]. These algorithms have already been used to solve the allocation problem in cognitive radio systems [42] for an uncoordinated spectrum access [38]. In our work, we use the MAB framework to optimize the allocation of resources.

To summarize, to the best of our knowledge, no previous work has considered the use of full-duplex relays in an uncoordinated uplink NOMA system with transmit power limits and AoI constraints for the devices with mixed traffic types. Hence, in this paper, we study an uplink uncoordinated NOMA transmission system, where multiple full-duplex relays with LCFS queues are deployed. IoT devices aim at auto-organizing their communications with the available relays using the MAB framework and without communicating with each other. Relays then forward the received signals to the BS, resulting in a two-hop communication system. IoT devices being deployed for multiple use cases with varying traffic characteristics and requirements, we consider a mixed traffic system consisting of continuous, sporadic and periodic transmissions. Moreover, each IoT device has a limited power budget. We propose a multi-step algorithm based on the MAB framework where we conduct the power-subband allocation while guaranteeing the transmit power constraints and minimizing the AoI for the
devices in an uncoordinated manner. The main contribution of this paper concerns the proposal of an efficient method to:

* allocate the available spectrum and power resources to devices with mixed traffic types in an uncoordinated system,

* minimize the transmit power at the devices and relays while maximizing the achieved rates,

* minimize the AoI for the IoT devices using the MAB framework.

The proposed MAB-based uncoordinated method is compared to a benchmark consisting of a coordinated spectrum and power allocation technique based on matching theory that minimizes AoI.

The rest of the paper is organized as follows: In section II, the system model is described while the considered optimization problem is formulated in section III. Section IV provides the foundations of the proposed MAB-based uncoordinated framework, while the details are given in section V. The coordinated system based on matching theory for efficient resource allocation and taken as a reference is developed in section VI. The proposed algorithms are evaluated using numerical simulations whose results are provided in section VII and conclusions are drawn in section VIII.

## II. SYSTEM MODEL

Consider a hexagonal cell with one BS located at the cell center, R relays and N IoT devices. Grouped in a set $R = \{r_1, r_2, ..., r_R\}$, the relays are distributed on a circle at mid-distance between the BS and the cell edges. Let $D^{cont} = \{d_1^{cont}, d_2^{cont}, ..., d_{n_1}^{cont}\}$, $D^{per} = \{d_1^{per}, d_2^{per}, ..., d_{n_2}^{per}\}$ and $D^{spo} = \{d_1^{spo}, d_2^{spo}, ..., d_{n_3}^{spo}\}$, where $n_1 + n_2 + n_3 = N$, denote the sets of devices having continuous, periodic and sporadic traffic types, respectively. The set of all IoT devices is denoted by $D = \{D^{per}, D^{spo}, D^{cont}\}$. For devices with a sporadic traffic, the probability of successful spectrum access is represented by $p_{d_i^{spo}}^a$. For the devices with a periodic traffic, the period between two consecutive transmissions is $T_{d_i^{per}}^a$. They can be, for example, sensors for medical use.

Due to possible large distances between the BS and the power-limited IoT devices, we assume that the latter can not transmit their signals directly to the BS. Hence, a two-hop communication network is considered where the devices transmit their signals to relays that forward them to the BS, as shown in Figure 1. A grant-free or uncoordinated uplink communication system is studied where IoT devices aim at organizing their transmissions without any communication between them. During each time slot, every IoT device is allowed to communicate only with one relay over one subband. The relays operate in a full-duplex mode, i.e., simultaneously receive the signals from IoT devices and forward them to the BS on the same frequency resource. Therefore, relays suffer from self-interference, possibly impeding the successful decoding of the received signals. Each relay is assigned a frequency subband and conventional OMA is implemented between the relays and the BS. Thereby, relays do not suffer from inter-relay interference. Moreover, by selecting a subband, a device automatically selects the corresponding relay. For multiple access, NOMA is considered between the devices and the relays.

<figure><img src="image_3.png" /><figcaption>Figure 1: System model with one BS, one relay and devices with different traffic types.</figcaption></figure>

### A. Power Allocation Scheme

In this paper, the uplink NOMA power allocation scheme proposed in [9] and [6] is generalized to the case of full-duplex transmissions and the consideration of multiple collisions.

Consider L available received power levels per channel at the relay. Let $\Gamma_{l,req}$ be the minimum signal-to-interference-plus-noise ratio (SINR) required to correctly decode the signal of each IoT device. Inspired by [9], it can be shown that the lth received power level at the relay, $v_l$, $l = 1, ..., L$, such that $v_1 > v_2 > ... > v_L$, guarantees the target SINR when facing $M_l$ supported collisions at each power level lower than l if and only if:

$$v_l = (\sigma^2 + C_{SI} \times P_{r,c}^{max}) \times \Gamma_{l,req} \prod_{k=l+1}^{L} (M_k \Gamma_{k,req} + 1). \quad (1)$$

In (1), $\sigma^2$ is the additive Gaussian noise power, $C_{SI}$ the residual self-interference factor, and $P_{r,c}^{max}$ the maximum transmit power of relay r on each of its channel c. Hence, $C_{SI} \times P_{r,c}^{max}$ is the maximum self-interference power experienced at the relay.

For successful SIC decoding at the relays, the signals of devices transmitting on the same channel should be received with different power levels. In other words, if two or more users have chosen the same power level $v_l$ on the same subband of a certain relay, a collision occurs, rendering their signals non-decodable. In uplink NOMA, SIC is performed in the decreasing order of channel gains. Therefore, when a collision occurs at level l, the signals of devices having chosen lower received power levels are also non-decodable. Nevertheless, upper-level signals remain decodable as long as the number
of colliding signals at each lower level is less than or equal to $M_l$.

In the considered system, IoT devices with periodic and sporadic traffic are allowed to transmit only at the lowest power level. The reason for this choice is to ensure that the signals of the top $L-1$ levels are not lost, in the case where two or more but less than $M_L + 1$ devices with non-continuous transmissions collide. This means that devices with continuous transmission are given a higher priority than the others. Perfect channel estimation is considered in this work. Channel gains account for large-scale fading, i.e., path loss and shadowing. In order to guarantee the received power level $v_l$ at the $r^{th}$ relay, each device $d_i$ sets its transmit power according to:

$$P_{d_i,l}^t = \frac{v_l}{h_{d_i,r}^2}, \qquad (2)$$

where $h_{d_i,r}$ is the channel gain between user $d_i$ and relay $r$.

## B. Uncoordinated Joint Channel and Power Allocation

In a system with uncoordinated spectrum access, the devices aim at auto-organizing their transmissions without any external intervention. To solve the problem of joint subband and power allocation, a multi-player MAB framework with zero-reward on collision is adopted [41]. Let $\mathcal{D}_{active} = \{\mathcal{D}^{cont}, \mathcal{D}^{spo}, \mathcal{D}^{active}, \mathcal{D}^{per}\}$ be the combined set of active IoT devices, where $\mathcal{D}^{spo}_{active}$ and $\mathcal{D}^{per}_{active}$ are the active devices with sporadic and periodic traffic respectively. Note that $\mathcal{D}_{active} \subset \mathcal{D}$. The set of arms is the set of available subband-power level pairs. Having different traffic types and a transmit power budget $P_{d_i}^{max}$, each device $d_i$ builds its action space $\mathcal{A}_{d_i}$, accounting for the subband-power level pairs on which it can transmit without violating its power budget. Let $\mathcal{A}_{d_i} = \{a_1, a_2, ..., a_{K_{d_i}}\}$, where $K_{d_i}$ represents the available number of arms for user $d_i$.

When choosing an action, each device $d_i$ receives a reward from the relay. In this work, the reward is considered equal to the achieved rate at the relay level, given by the Shannon formula [43]:

$$Rate_{d_i,a_k} = B_c \log_2(1 + \Gamma_{d_i,a_k}), \qquad (3)$$

with $B_c$ being the channel bandwidth and $\Gamma_{d_i,a_k}$ the achieved SINR of device $d_i$, given by:

$$\Gamma_{d_i,a_k} = \frac{v_l(a_k)}{\sum_{j=l+1}^{L} v_j + C_{SI} P_r^t + \sigma^2}. \qquad (4)$$

In (4), $v_l$ is the power level chosen in action $a_k$. $\sum_{j=l+1}^{L} v_j$ is the residual NOMA interference not canceled by SIC, and $P_r^t$ the transmit power of relay $r$ per subband at timeslot $t$. Hence, $C_{SI} P_r^t$ represents the instantaneous self-interference experienced by the relay.

If two or more devices select the same subband-power level pair, a collision occurs, causing the decoding of the involved users' signals to fail. Hence, the reward achieved by the

concerned users is equal to 0. Let $\eta_{d_i}$ represent the collision indicator for device $d_i$ on arm $a_k$, i.e.:

$$\eta_{d_i,a_k} = \begin{cases} 0, & \text{in case of a collision,} \\ 1, & \text{otherwise.} \end{cases} \qquad (5)$$

Hence, the reward or utility received by device $d_i$ when selecting arm $a_k$, $U_{d_i,a_k}$, can be expressed as:

$$U_{d_i,a_k} = \eta_{d_i,a_k} \times Rate_{d_i,a_k}. \qquad (6)$$

Due to the absence of communication between IoT devices, each device only observes its own list of actions and its corresponding list of rewards. Since the rewards are unknown beforehand at the device level, there is the need for an exploration phase during which the devices learn the rewards. To enhance system performance, the objective of this work is to maximize the welfare of all IoT devices, given by:

$$w = \sum_{i=1}^{n_1} U_{d_i^{cont},a_k} + \sum_{i=1}^{n_2} U_{d_i^{per},a_k} + \sum_{i=1}^{n_3} U_{d_i^{spo},a_k}, \qquad (7)$$

where each sum represents the welfare of the devices having continuous, periodic and sporadic traffic, respectively.

## C. Relay-BS Communication

As previously stated, relays are equipped with full-duplex communication capabilities. To satisfy the quality of service (QoS) requirements of devices when forwarding their messages to the BS, the transmit power of each relay must be optimized.

Let $X_{r,cont}^t$ and $X_{r,ncont}^t$ be the number of continuous and non-continuous signals correctly decoded by relay $r$ at timeslot $t$. The minimum rate required for the relay to be able to transmit all successfully decoded signals to the BS is equal to:

$$Rate_{r,req} = X_{r,cont}^t \times Rate_{cont,req} + X_{r,ncont}^t \times Rate_{ncont,req}, \qquad (8)$$

where $Rate_{cont,req}$ and $Rate_{ncont,req}$ are the required rates for devices with continuous and non-continuous transmission, respectively.

Each relay transmits its data over C subbands. Moreover, as already mentioned, we assume orthogonal channel allocation between different relays, i.e., different relays do not share the same subband. Thus, the rate achieved by relay $r$ on channel $c$ is given by:

$$Rate_{r,c} = B_c \times \log_2 \left( 1 + \frac{h_{r,BS}^2 P_r^t}{\sigma^2} \right), \qquad (9)$$

where $h_{r,BS}$ is the channel gain between relay $r$ and the BS. Assuming an equal rate repartition among the C subbands, to meet the rate requirement in (8), the transmit power of the relay $r$ on a subband $c$ should satisfy:

$$P_{r,c}^t = \min \left\{ \frac{\sigma^2}{h_{r,BS}^2} \left( 2^{\frac{Rate_{r,req}}{C \times B_c}} - 1 \right), P_{r,c}^{max} \right\}. \qquad (10)$$

When the transmit power needed by the relay exceeds its budget, an overflow scenario is expected. In case of an overflow
due to continuous traffic, the system becomes unstable. To avoid this, a solution based on the application of a timeout period can be considered, in which the devices having continuous traffic are invited, through the value of their received rewards, to connect to another relay following the mechanism described in [6]. This solution guarantees that continuous traffic is transmitted to the BS during the first timeslot after the timeout period. It is motivated by the sensitivity to latency of the underlying application for continuous devices. Therefore, the devices having continuous traffic would not suffer from high values of AoI.

### D. Sources of AoI

Let $\Delta_1(d_i, t)$ denote the AoI related to the first link, i.e., device-to-relay communication, and $\Delta_2(d_i, t)$ denote the AoI related to the second link, i.e., relay-to-BS communication, measured at timeslot $t$. Both $\Delta_1(d_i, t)$ and $\Delta_2(d_i, t)$ are measured at the relay level since the latter is able to identify the timeslot of each message transmitted by users as well as the timeslot it can retransmit the signal to the BS. The total AoI for a device $d_i$ is equal to:

$$ \Delta(d_i, t) = \Delta_1(d_i, t) + \Delta_2(d_i, t). \quad (11) $$

$\Delta_1(d_i, t)$ depends on the number of collisions the signal undergoes before being successfully received at the relay. Hence, optimizing the subband-power level allocation can reduce the AoI of the first link, $\Delta_1(d_i, t)$, by minimizing the number of signal collisions.

On the other hand, $\Delta_2(d_i, t)$ depends on the state of the data buffer at the relay. This state represents the number of timeslots that a packet has to wait in the buffer of the relay before being forwarded to the BS. Considering the variable throughput at the relay level due to the nature of the traffic, this buffer is introduced for the cases where the capacity of the relay-to-BS link is lower than the aggregated throughput of the device-to-relay links. Equivalently, this corresponds to situations where the power required to forward the received device signals exceeds the relay power budget i.e.:

$$ P_{r,c}^t \geq P_{r,c}^{max}. \quad (12) $$

While being beneficial to avoid data loss, buffering at the relay level increases the AoI for the devices. Eq. (10) and (12) are now modified to include a new parameter $X_{0,r}$ that represents the maximum number of received signals that a relay can forward per timeslot to the BS without the need for buffering. Using Shannon's formula, parameter $X_{0,r}$ can be formulated as:

$$ X_{0,r} = \frac{B_c \times C}{Rate_{d_i,req}} \log_2 \left( 1 + \frac{P_{r,c}^{max} h_{r,BS}^2}{\sigma^2} \right). \quad (13) $$

$Rate_{d_i,req}$ is the rate required by user $d_i$ and is either equal to $Rate_{cont,req}$ or $Rate_{ncont,req}$ depending on the traffic type of the device. Note that when the number of correctly decoded signals at the relay, $X_t^t$, exceeds $X_{0,r}$, $\Delta_2(d_i, t)$ increases, leading to an increase of the total AoI $\Delta(d_i, t)$.

### E. AoI Computation

We consider that the IoT devices submit new messages at timeslots $t_1, t_2, \dots$ and that these messages are successfully received at the relay at timeslots $t'_1, t'_2, \dots$, respectively. The instantaneous AoI for device $d_i$ $\Delta_1(d_i, t)$ is given by:

$$ \Delta_1(d_i, t) = t'_j - t_j, \quad j = 1, 2, \dots \quad (14) $$

As for $\Delta_2(d_i, t)$, it is given by:

$$ \Delta_2(d_i, t) = t''_j - t'_j, \quad (15) $$

where $t''_j$ is the timeslot where the signal is correctly received at the BS. The average AoI is given by:

$$ \Delta_{d_i} = \frac{1}{T} \int_{0}^{T} \Delta(d_i, t) dt, \quad (16) $$

where $T$ is the time horizon of the allocation.

## III. PROBLEM FORMULATION

The main purpose of this study is to minimize the AoI of IoT devices and relays while satisfying their rate requirements and minimizing their transmit power. The optimization problem is formulated as:

$$ \max_{P_{d_i}^t, P_{r,c}^t, z_{d_i,c}} \left( \sum_{i=1}^{n_2} \Delta_{d_i}^{per} + \sum_{i=1}^{n_3} \Delta_{d_i}^{spo} \right), \quad (17) $$

such that:

$$ \sum_{c} z_{d_i,c} Rate_{d_i,c} \geq Rate_{d_i,req}, \forall d_i \in \mathcal{D} \quad (17a) $$

$$ \sum_{i=1}^{n_1} \sum_{c} z_{d_i}^{cont, c} Rate_{d_i}^{cont, c} \leq \sum_{c} Rate_{r,c}, \quad (17b) $$

$$ \sum_{c} z_{d_i,c} P_{d_i, l}^{t} \leq P_{d_i}^{max}, \quad (17c) $$

$$ P_{r,c}^t \leq P_{r,c}^{max}. \quad (17d) $$

In (17), $z_{d_i,c} = 1$ if device $d_i$ transmits using subband $c$, and 0 otherwise. $\Delta_{d_i}^{per}$ and $\Delta_{d_i}^{spo}$ represent the average AoI for periodic and sporadic traffic devices respectively. (17a) is the rate requirement constraint per IoT device, whereas (17b) is the rate constraint for avoiding overflow scenarios while serving continuous traffic devices. (17c) and (17d) are the transmit power constraints for devices and relays, respectively.

Problem (17) consists in minimizing the AoI by optimizing the power and subband allocations. To solve this problem, a multi-step algorithm is proposed. For each device, it starts by assigning a subband-power level pair while satisfying their rate requirements. Then, it checks for overloaded relays where the non-continuous traffic devices suffer from AoI and proceeds to address the latency issue.
IV. SUBBAND POWER-LEVEL ASSIGNMENT

We first propose a solution for the channel-power level assignment problem. As previously mentioned, IoT devices are energy-limited and need to minimize their transmit power. To satisfy this goal, the UCB algorithm is used to find the allocation of subband-power level pairs while accounting for the power budget of communicating devices.

The objective is to satisfy the rate requirement of each device while minimizing its transmit power. Hence, two system parameters are considered: the rate achieved by device $d_i$, denoted by $Rate_{d_i,a_k}$, and its transmit power $P_{d_i,l}^t$. As shown in (2), a device $d_i$ can minimize its transmit power $P_{d_i,l}^t$ by communicating on subbands with higher channel gains. Since each device is power-limited, it is not able to transmit at power levels exceeding its power budget. Moreover, it may happen that one subband-power level pair may be optimal for multiple users, resulting in a collision if all those devices transmit using that level. Therefore, the assignment of arms, i.e., of subband-power level pairs, to devices should be designed to maximize system welfare $w$. In the following, this assignment is discussed based on the traffic type of the devices.

For devices with continuous traffic, the UCB algorithm of [6] is used. This algorithm is divided into two phases: exploration and exploitation. In the exploration phase, the devices learn the arms and their rewards before the exploitation phase where they select the best arm. In our situation, each IoT device is allowed to choose its arm in a way to achieve a trade-off between minimizing its transmit power and maximizing its achieved rate. In fact, instead of choosing only the arm that maximizes the achieved rate, the user takes also into consideration the used transmit power. For this purpose, weight $\beta$ is introduced to influence the decision function with the achieved rates and the power budgets. The impact of weight value $\beta$ is discussed in Section VII.

For devices with sporadic traffic, a different approach is considered. Depending on its access probability, a device may not be active for all available timeslots. Hence, we define $T_{on}$ as the total number of timeslots where a device is transmitting. Note that this type of device may successfully transmit on an arm at a timeslot $t_{on}$ and fail to do so at a timeslot $t'_{on}$ using the same arm. Note that $t_{on}, t'_{on} \in \{1, \dots, T_{on}\}$. In addition, a weight function $f(W, t_{on})$ of window size $W$ representing the weight of the last $W$ rewards at the device level is introduced. We define $f(W, t_{on})$ as an increasing linear function based on the last $W$ active timeslots $[t_{on} - W, t_{on}]$, $t_{on} \in \{1, \dots, T_{on}\}$ such that:

$$f(W, t_{on}) = \begin{cases} 0, & \text{for } \tau = t_{on} - W, \\ \frac{\tau}{W}, & \text{for } t_{on} - W < \tau < t_{on}, \\ 1, & \text{for } \tau = t_{on}. \end{cases} \quad (18)$$

The device with sporadic traffic calculates the achieved rewards in the last $W$ active timeslots by multiplying the rewards with $f(W, t_{on})$. Consequently, the choice of the sporadic devices only depends on the achieved rewards in the last $W$ active allocation choices. For such devices, there is always a

need to minimize the transmit power while meeting the rate requirements. Therefore, sporadic IoT devices also choose the arms that maximize their decision function:

$$a_i^* = \underset{a_k \in \mathcal{A}_{d_i}}{\operatorname{argmax}} \left( (1 - \beta) \frac{Rate_{d_i, a_k}}{Rate_{max}} - \beta \frac{P_{d_i, k}^t}{P_{d_i}^{max}} \right), \quad (19)$$

where $Rate_{max} = B_c \log_2(1 + \frac{\gamma L}{\sigma^2})$ is the maximum rate achievable by a device with sporadic traffic. Inspired by [6], the proposed technique to solve the subband and power allocation problem for sporadic devices is summarized in Algorithm 1 where $n_{d_i}(a_k, t_{on})$ is the number of times arm $a_k$ is played by device $d_i$ during the $t_{on}$ active timeslots. Note that in case of

**Algorithm 1:** UCB subband-power level assignment for devices with sporadic traffic

**Initialization:** Each device $d_i^{spo}$ generates its action profile set $\mathcal{A}_{d_i}$ depending on its power budget and channel gains. These devices only transmit on the lowest power level of each subband.

**for** $t=1:T$ **do**

**for** $i=1:n_3$ **do**

// If $t$ is an active timeslot:

$$t_{on} = t_{on} + 1$$

// Utility function sent by the relay:

$$U_{d_i, a_i^*}(t) = \eta_{d_i, a_k} \times Rate_{d_i, a_k}$$

// Best arm identification:

$$a_i^* = \underset{a_k \in \mathcal{A}_{d_i}}{\operatorname{argmax}} \left( (1 - \beta) \frac{Q(d_i, a_k) + \sqrt{\frac{2 \log(t_{on})}{n_{d_i}(a_k, t_{on})}}}{Rate_{d_i, req}} - \beta \frac{P_{d_i, k}^t}{P_{d_i}^{max}} \right)$$

// Parameters update:

$$s_i(a_i^*, t_{on} - W : t_{on}) = U_{d_i, a_i^*}(t_{on} - W : t_{on})$$

$$n_i(a_i^*, t_{on} + 1) = n_i(a_i^*, t_{on}) + 1$$

$$Q(i, a_i^*) = f(W, t_{on}). * s_i(a_i^*, t_{on} - W : t_{on})$$

**end**

collision between two sporadic devices, they retransmit their signals in the next timeslot.

For each IoT device $d_i$ with periodic transmission, the solution for devices having continuous traffic is applied in its active period $T_{d_i}^{per}$. However, since sporadic and periodic devices share the same power levels on the subbands, collisions may occur. In this case, the IoT device with periodic transmission goes into a pseudo-sporadic mode with an increasing access probability value in order to prioritize the sporadic traffic device competing for the same arm. In other words, the corresponding periodic traffic device retransmits its message at a later stage in order to free the arm it occupies. When the pseudo-sporadic device retransmits successfully its signal, it returns to the initial state, i.e., to the periodic mode.

The complexity of this assignment technique is given by the complexity of the UCB algorithm. Hence, it amounts to $O(\log(N))$, $N$ being the number of devices.
V. PROPOSED SOLUTION FOR AOI MINIMIZATION

Being in an uncoordinated spectrum access setting, the problem of AoI minimization is solved using the MAB framework with zero-reward for unsent messages. In fact, when the messages from the IoT devices successfully reach the relay with queuing capabilities, the relay transmits all the traffic from the continuous devices and schedules the messages of the sporadic ones with respect to the following priority:

$$P_r(d_i, t) = \frac{number_{arr}(d_i)}{number_{sent}(d_i)} \times \frac{t}{t - \Delta_1(d_i, t)}. \quad (20)$$

In (20), $number_{arr}$ and $number_{sent}$ stand for the number of messages arriving at the relay and the number of messages sent by the relay for a given device $d_i$, respectively. This priority metric ensures that the less the sporadic device is served by the relay, the higher its priority. Eq. (20) also avoids the dominance of the first type of AoI $\Delta_1(d_i)$. Indeed, in the case where $\Delta_1(d_i, t)$ is large, $t-\Delta_1(d_i, t)$ is small, hence the sporadic device $d_i$ is prioritized.

In order to minimize the overall AoI, we consider a new reward $U_{d_i,a_k^+}$ received by all the devices from the relay. It accounts not only for the rate achieved by the devices but also for the AoI induced by link 2. $U_{d_i,a_k^+}$ is given by:

$$U_{d_i,a_i^+} = \frac{Rate_{d_i,a_k}}{Rate_{d_i,max}} \times \frac{1}{\Delta_2(d_i, t)}. \quad (21)$$

In (21), $Rate_{d_i,max}$ stands for the maximum rate a device with non-continuous traffic can achieve. It is given by $Rate_{d_i,max} = B_c \log_2 \left(1 + \frac{v_L}{\sigma^2}\right)$ and is taken equal to the rate required by the device, $Rate_{d_i,req}$. Thus, the first fraction is a binary function equal to either 0 in case of collision or 1 otherwise. Hence the reward function can be expressed as:

$$U_{d_i,a_i^+} = \eta_{d_i,a_k} \times \frac{1}{\Delta_2(d_i, t)}. \quad (22)$$

In case of a collision, a zero-reward is maintained. As a result, the first type of AoI $\Delta_1(d_i, t)$ is the dominant part in the AoI expression, which can be directly determined by the device itself. Moreover, the utility function is inversely proportional to the second AoI type. Therefore, if the information reaches the relay successfully but its forwarding to the BS is delayed, the utility function of the arms belonging to the given relay becomes smaller. In that case, the device $d_i$ is encouraged to a new exploration phase where it finds a temporary gateway to transmit its messages to the BS. The same approach is applied for devices with periodic traffic. The technique is summarized in Algorithm 2.

The complexity of the proposed technique is also dominated by the complexity of the UCB algorithm, i.e., in the order of $O(\log(N))$. Note that in case of overflow due to continuous traffic, it is recommended to use the timeout period method proposed in [6]. By doing so, the continuous traffic is guaranteed to be forwarded to the BS in the same timeslot.

Further improvements in AoI can be achieved by predicting the corresponding impact of the number of active devices with sporadic traffic in upcoming timeslots. In fact, in Algorithm 2,

**Algorithm 2:** UCB AoI minimization for devices with non-continuous traffic

**Initialization:** Run Algorithm 1 to adapt for every device its channel-power level pair while minimizing its transmit power for T timeslots.

**for** t=T:2T **do**

**for** i=1:n₂+n₃ **do**

**// If** t is an active timeslot:

$$t_{on} = t_{on} + 1$$

**// Utility function sent by the relay:**

$$U_{d_i,a_i^+} = \eta_{d_i,a_k} \times \frac{1}{\Delta_2(d_i,t)}$$

**// Best arm identification:**

$$a_i^+ = \underset{a_k \in \mathcal{A}_{d_i}}{\operatorname{argmax}} \left( (1 - \beta) \frac{Q(d_i, a_k^+) + \sqrt{\frac{2 \log(t)}{n_{d_i}(a_k, t)}}}{Rate_{d_i,req}} - \beta \frac{p_{d_i,k}^t}{p_{d_i}^{\max}} \right)$$

**// Parameters update:**

$$s_i(a_i^+, t_{on} - W : t_{on}) = U_{d_i,a_i^+}(t_{on} - W : t_{on})$$

$$n_i(a_i^+, t_{on} + 1) = n_i(a_i^+, t_{on}) + 1$$

$$Q(i, a_i^+) = f(W, t_{on}). * s_i(a_i^+, t_{on} - W : t_{on})$$

**end**

**end**

the relay replies with the current AoI $\Delta_2(d_i, t)$ depending on the current state of its queue whatever the value of the next state. In other words, a congested relay may be able to clear its queue in the next timeslot if the number of received messages allows for it. Consequently, it will be able to accept new messages from the IoT devices instead of encouraging them to go through another relay by replying with low rewards. In such a case, the predictive AoI $\Delta_2(d_i, t+1)^*$ is low, but the current one $\Delta_2(d_i, t)$ is high.

The predicted number of active sporadic IoT devices follows a simple probabilistic calculation. Let $Y$ be a discrete random variable representing the number of potential active sporadic devices at timeslot $t$ and taking values in $\{0, ..., n_3^c\}$ with $n_3^c$ being the non-collided devices with sporadic traffic at timeslot $t-1$. Note that we do not consider the collided sporadic traffic devices at $t-1$ since they surely retransmit their signals at timeslot $t$. The probability of having $k$ active sporadic devices among the uncertain ones is given by:

$$pr(Y = k) = \left(1 - p_{d_i^{spo}}^a\right)^{n_3^c - k} \times \left(p_{d_i^{spo}}^a\right)^k \times \binom{n_3^c}{k}. \quad (23)$$

Let:

$$k^* = \underset{k}{\operatorname{argmax}} pr(Y = k) \quad (24)$$

be the number of messages having the largest probability. Hence, the number of estimated sporadic messages will be equal to $k^* + n_3 - n_3^c$. By relying on the estimated number of messages, the relay performs the scheduling of the next timeslot
and estimates the corresponding AoI. Then, it replies with the estimated AoI $\Delta_2^*(d_i, t+1)$. The reward is then expressed as:

$$U_{d_i, a_i^+} = \eta_{d_i, a_k} \times \frac{1}{\Delta_2^*(d_i, t+1)}. \quad (25)$$

The approach then follows the algorithm described earlier by just modifying the utility function.

## VI. RESOURCE ALLOCATION USING COORDINATED MATCHING

This section describes the coordinated allocation alternative based on the matching framework [13] introduced to benchmark the proposed uncoordinated MAB-based method.

Starting from a fully coordinated system, the BS is assumed to have full knowledge of the power budget and user traffic types, as well as the power budget, the buffer size and the current state of the queues at the relays.

The subband allocation problem is modeled as a one-to-one matching game. In this model, the set of arms and the set of devices form two independent sets depending on their preferences. In this scenario, each arm can be allocated with just one device and each device can transmit on just one arm per timeslot. Hence, each device sorts the channels in a descending order with respect to their gains. By doing so, when all devices are matched with different arms, the rate requirements are satisfied while tending to consume the least amount of transmit power possible. Also, since only large-scale fading is considered in this paper, the channel preference of a device is equivalent to a relay preference (i.e. choosing a channel results in choosing a particular relay). Consider the preference relation of device $d_i$ to be given by:

$$f_1 = h_{d_i,r}^2 (r, d_i). \quad (26)$$

The matching preference of the arms depends on the queue state of all the relays, the channel conditions and the transmission power and activity of the devices. Thus, the arms preference function is modeled as follows:

$$f_2 = \frac{\frac{h_{r,BS}}{\max_{r} h_{r,BS}}}{\frac{Queue_r(t)}{Bu_r} \times \psi(d_i)}. \quad (27)$$

In (27), $Queue_r(t)$ stands for the number of packets currently in the queue of the relay $r$, $Bu_r$ is its buffer size and $\psi(d_i)$ is a function defined according to the traffic type of each user:

$$\psi(d_i) = \begin{cases} 1 & \text{if the traffic is continuous or periodic,} \\ 1 - p_{d_i^{spo}}^a & \text{otherwise.} \end{cases} \quad (28)$$

The arms preference function is designed to be an inversely proportional function to the occupancy of the relay queue. In other words, a congested relay is less likely to admit devices. Also, this preference relation ensures that a higher access probability of a non-continuous traffic device results in a lower $\psi(d_i)$. Hence, the sporadic traffic devices having high access probability are mostly matched with non-congested relays. The problem is then solved using the Gale-Shapley algorithm [14] at the BS level.

## VII. SIMULATION RESULTS

Extensive simulations were performed to validate the performance of the proposed algorithms. We first consider an uplink communication system with one BS at the cell center, one relay and $N = 60$ IoT devices divided among $n_1 = 36$ continuous traffic devices, $n_2 = 12$ periodic traffic devices and $n_3 = 12$ sporadic traffic devices with an access probability equal to 0.3. The periodic traffic devices are further divided into 4 groups of 3 devices each, transmitting in subsequent periods so that they do not interfere with each other. The cell radius is equal to 250 m. The number of available subbands and power levels are given respectively by $C = 9$ and $L = 5$. Hence, the maximum number of available resources is given by $C \times L = 45$. IoT devices are distributed around the relay according to a Poisson distribution [44]. Signals undergo a distance-dependent path loss with a decay factor of 3.76, and a zero-mean log-normal shadowing with an 8 dB variance. The noise power spectral density is equal to $N_0 = 4 \times 10^{-21}$ W/Hz. The system is then evaluated with two relays. Other system parameters are summarized in Table I.

<table><caption>Table I: Design parameters values</caption><tr><td>Rate<sub>cont,req</sub></td><td>0.75 Mbps</td><td>P<sup>max</sup><sub>D<sub>i</sub></sub></td><td>23 dBm</td></tr><tr><td>Rate<sub>periodic,req</sub></td><td>0.3 Mbps</td><td>Rate<sub>sporadic,req</sub></td><td>0.3 Mbps</td></tr><tr><td>C &times; P<sup>max</sup><sub>r</sub></td><td>5W</td><td>C<sub>SI</sub></td><td>-120 dB</td></tr><tr><td>B<sub>c</sub></td><td>156 kHz</td><td>W</td><td>10</td></tr><tr><td>p<sup>a</sup><sub>d<sub>i</sub><sup>spo</sup></sub></td><td>0.3</td><td>&beta;</td><td>0.5</td></tr></table>

The performance of the proposed algorithms with and without AoI estimation, as described in Algorithm 2, is evaluated for $T = 1,000$ timeslots. Moreover, comparisons are made against two uncoordinated methods:

* the random access NOMA method (called RA-NOMA in the figure captions), where devices choose their arms randomly,

* an allocation technique where the devices solely allocate their arms using Algorithm 1 without taking AoI into consideration (Algorithm 1 - no AoI),

and against the benchmark coordinated method based on the matching theory. The latter applies a centralized setting which assumes that the BS has full knowledge of the channel gains between devices and relays, and those between relays and BS, as well as the queue status of the relays, and performs resource allocation accordingly. This method is referred to as “Coordinated MT”.

### A. Performance of the Arm Assignment Technique

The assignment of arms is evaluated in Figure 2, where the instantaneous successful transmission percentage is plotted as a function of time. Figure 2 shows that the proposed UCB technique based on Algorithm 1 reaches convergence with 100% satisfaction, hence the exploitation phase, within 80 timeslots for all IoT devices. During the first 80 timeslots,
<figure><img src="image_4.png" /><figcaption>Figure 2: Instantaneous successful transmission percentage as a function of time.</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>Figure 3: Average transmit power for a varying requested rate.</figcaption></figure>

the devices explore the different arms to learn their rewards. Then, each device selects the arm that maximizes its decision function. Since the matching technique is fully coordinated by the BS, convergence to the full satisfaction state is reached within the very first timeslots. However, RA-NOMA cannot reach a successful rate better than 40% with an average of 28%.

The average transmission power per device as a function of the requested rate is evaluated in Figure 3. By varying weight $\beta$, we observe that $\beta = 0.5$ is the best choice for minimizing the power consumption. Moreover, for the same average transmit power, the proposed UCB-based method with $\beta = 0.5$ is within 0.2 Mbps from the coordinated solution. In Figure 4, we evaluate the method proposed to estimate the number of active devices in the next timeslot. We observe that the proposed probabilistic approach provides estimates close to the actual numbers of active devices, leading to a slightly better

<figure><img src="image_6.png" /><figcaption>Figure 4: Estimation of active IoT devices.</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>Figure 5: Cumulative distribution function of AoI.</figcaption></figure>

performance when estimating the AoI $\Delta_2(d_i, t)$ for a device $d_i$, as it will be shown in the subsequent results in Figures 5, 6, 7 and 8.

## B. Performance Assessment of the Proposed Solution

We consider now a system having two relays and we assume that one of the relays is congested and cannot transmit all the signals of the sporadic devices with the continuous ones to the BS. In Figure 5, we show that application of Algorithm 1 only leads to a large AoI value, especially due to the increase in $\Delta_2(d_i, t)$, since it does not account for $\Delta_2(d_i, t)$. Thus, the devices always transmit to the same relay without taking into account the state and occupation of the relay. Consequently, they do not consider switching to another less congested relay. By using the proposed UCB-based technique for AoI minimization, we see a considerable decrease in the achieved AoI. Moreover, predicting the future number of active devices brings an additional non-negligible benefit. The choice of the
<figure><img src="image_8.png" /><figcaption>Figure 6: Average AoI for a varying number of devices with sporadic traffic.</figcaption></figure>

<figure><img src="image_9.png" /><figcaption>Figure 8: Average AoI in the particular case where $p_{d_i^{spo}}^a \times n_3 = 6$.</figcaption></figure>

<figure><img src="image_10.png" /><figcaption>Figure 7: Average AoI for a varying access probability for 9 sporadic devices.</figcaption></figure>

adopted method depends on the complexity the relay can afford. Also note that the performance of both UCB-based techniques is close to the centralized technique based on matching theory.

In Figures 6 and 7, we show the average AoI while varying the total number of sporadic players and the access probability, respectively. The different variants of the proposed approach significantly outperform basic random access approaches.

In Figure 8, the average peak AoI is plotted while keeping the product of access probability and total number of sporadic devices constant and equal to 6. It is observed that the proposed UCB algorithm presents an inflection point around an access probability of 0.2 after which it becomes more efficient in decreasing the AoI.

## VIII. CONCLUSION

In this paper, the uncoordinated joint channel and power allocation problem was studied in a mixed-traffic NOMA uplink system, with the aim of minimizing the AoI. The problem was formulated using the MAB framework with zero-reward on collisions. An algorithm based on the UCB was proposed to assign devices with different traffic types to the available subband-power level pairs, and to minimize the AoI of non-prioritized users, i.e., users with a non-continuous traffic. The efficiency of this technique in terms AoI minimization and device transmit power, as well as achieved rates maximization, was shown to outperform competing techniques and to approach closely a centralized approach with a perfect knowledge of system and channel state parameters. Future work will involve extending this study to better estimate the number of active devices using neural networks. Also, the scalability of the method needs to be further explored to cover a larger number of devices while meeting their QoS requirements and power constraints. The robustness of the proposed framework to imperfect or partial channel state information is another important study item to be investigated.

## ACKNOWLEDGEMENT

This work was carried out in the context of Beyond5G, a project funded by the French government as part of the economic recovery plan, namely "France Relance", and the investments for the future program.

For the purpose of open access, the authors have applied a Creative Commons Attribution (CC BY) license to any Accepted Manuscript version arising.

## REFERENCES

[1] E. Balevi, F. T. A. Rabee, and R. D. Gitlin, "ALOHA-NOMA for Massive Machine-to-Machine IoT Communication," in 2018 IEEE Int. Conf. on Commun. (ICC), Kansas City, MO, USA, May 2018, pp. 1-5.
[2] A. Mazin, M. Elkourdi, and R. D. Gitlin, "Comparison of Slotted Aloha-NOMA and CSMA/CA for M2M Communications in IoT Networks," in 2018 IEEE 88th Veh. Technol. Conf. (VTC-Fall), Chicago, IL, USA, Aug. 2018, pp. 1–5.

[3] M.-R. Hojeij, C. Abdel Nour, J. Farah, and C. Douillard, "Waterfilling-Based Proportional Fairness Scheduler for Downlink Non-Orthogonal Multiple Access," *IEEE Wireless Commun. Lett.*, vol. 6, no. 2, pp. 230–233, Feb. 2017.

[4] J. Farah, E. Sfeir, C. Abdel Nour, and C. Douillard, "New resource allocation techniques for base station power reduction in orthogonal and non-orthogonal multiplexing systems," in 2017 IEEE Intern. Conf. on Commun. Workshops (ICC Workshops), Paris, France, May 2017, pp. 618–624.

[5] A. Kilzi, J. Farah, C. Abdel Nour, and C. Douillard, "Mutual Successive Interference Cancellation Strategies in NOMA for Enhancing the Spectral Efficiency of CoMP Systems," *IEEE Trans. on Commun.*, vol. 68, no. 2, pp. 1213–1226, Feb. 2020.

[6] J. Doumit, M.-J. Youssef, C. Abdel Nour, J. Farah, and C. Douillard, "Resource Allocation in Full-Duplex Uncoordinated Communication Systems with NOMA," in 2021 IEEE 32nd Annu. Int. Symp. on Pers., Indoor and Mobile Radio Commun. (PIMRC), Helsinki, Finland, Sept. 2021, pp. 1104–1110.

[7] A. Benjebbour, K. Saito, A. Li, Y. Kishiyama, and T. Nakamura, "Non-orthogonal multiple access (NOMA): Concept, performance evaluation and experimental trials," in 2015 Int. Conf. on Wireless Netw. and Mobile Commun. (WINCOM), Marrakech, Morocco, Oct. 2015, pp. 1–6.

[8] N. Zhang, J. Wang, G. Kang, and Y. Liu, "Uplink Non-orthogonal Multiple Access in 5G Systems," *IEEE Commun. Lett.*, vol. 20, no. 3, pp. 458–461, Mar. 2016.

[9] J. Choi, "NOMA-based random access with multichannel aloha," *IEEE J. on Sel. Areas in Commun.*, vol. 35, no. 12, pp. 2736–2743, Oct. 2017.

[10] J. Kim, J. Kim, and S.-H. Park, "Joint Design of Power Control and SIC Decoding Order for Max-Min Fairness optimization in Uplink NOMA Systems," in 2021 Int. Conf. on Inf. Net. (ICOIN), Jeju Island, Korea (South), Jan. 2021, pp. 339–342.

[11] I. Baig, N. ul Hasan, M. Zghaibeh, I. U. Khan, and A. S. Saand, "A DST precoding based uplink NOMA scheme for PAPR reduction in 5G wireless network," in 2017 7th Int. Conf. on Modeling, Simul., and Applied Optim. (ICMSAO), Sharjah, United Arab Emirates, Apr. 2017, pp. 1–4.

[12] C. Xiao, J. Zeng, W. Ni, X. Su, R. P. Liu, T. Lv, and J. Wang, "Downlink MIMO-NOMA for ultra-reliable low-latency communications," *IEEE J. Sel. Areas Commun.*, vol. 37, no. 4, pp. 780–794, Feb. 2019.

[13] M.-J. Youssef, J. Farah, C. Abdel Nour, and C. Douillard, "Resource Allocation in NOMA Systems for Centralized and Distributed Antennas With Mixed Traffic Using Matching Theory," *IEEE Trans. on Commun.*, vol. 68, no. 1, pp. 414–428, Oct. 2020.

[14] D. Gale and H. S. Shapley, "College Admissions and the Stability of Marriage," *American Mathematical Monthly*, vol. 69, no. 1, pp. 9–15, 21962.

[15] W. Chang, Y.-T. Jau, S.-L. Su, and Y. Lee, "Gale-Shapley-algorithm based resource allocation scheme for device-to-device communications under-laying downlink cellular networks," in 2016 IEEE Wireless Commun. and Netw. Conf., Doha, Qatar, Apr. 2016, pp. 1–6.

[16] M. J. Youssef, C. Abdel Nour, J. Farah, and C. Douillard, "Backhaul-Constrained Resource Allocation and 3D Placement for UAV-Enabled Networks," in 2019 IEEE 90th Veh. Technol. Conf. (VTC-Fall), Honolulu, HI, USA, Sept. 2019, pp. 1–7.

[17] M. J. Youssef, J. Farah, C. Abdel Nour, and C. Douillard, "Full-Duplex and Backhaul-Constrained UAV-Enabled Networks Using NOMA," *IEEE Trans. on Veh. Technol.*, vol. 69, no. 9, pp. 9667–9681, Sept. 2020.

[18] C. Zhong and Z. Zhang, "Non-Orthogonal Multiple Access With Cooperative Full-Duplex Relaying," *IEEE Commun. Lett.*, vol. 20, no. 12, pp. 2478–2481, Dec. 2016.

[19] J. Li, Y. Q. Zhao, F. R. Yu, and X. Huang, "Queuing Analysis of Two-Hop Relay Technology in LTE/LTE-A Networks With Unsaturated and Asymmetric Traffic," *IEEE Internet of Things J.*, vol. 3, no. 3, pp. 378–385, June 2016.

[20] H. Chen, Y. Gu, and S.-C. Liew, "Age-of-Information Dependent Random Access for Massive IoT Networks," in IEEE INFOCOM 2020 - IEEE Conf. on Computer Commun. Workshops (INFOCOM WKSHPS), Toronto, ON, Canada, Jul. 2020, pp. 930–935.

[21] J. F. Grybosi, J. L. Rebelatto, and G. L. Moritz, "Age-of-Information of SIC-Aided Massive IoT Networks with Random Access," *IEEE Internet of Things J.*, pp. 1–1, May 2021.

[22] D. C. Atabay, E. Uysal, and O. Kaya, "Improving Age of Information in Random Access Channels," in *IEEE INFOCOM 2020 - IEEE Conf. on Computer Commun. Workshops (INFOCOM WKSHPS)*, Toronto, ON, Canada, Jul. 2020, pp. 912–917.

[23] J.-B. Seo and J. Choi, "On the Outage Probability of Peak Age-of-Information for D/G/1 Queuing Systems," *IEEE Commun. Lett.*, vol. 23, no. 6, pp. 1021–1024, Apr. 2019.

[24] C. Kam, S. Kompella, G. D. Nguyen, J. E. Wieselthier, and A. Ephremides, "Controlling the age of information: Buffer size, deadline, and packet replacement," in *MILCOM 2016 - 2016 IEEE Military Commun. Conf.*, Baltimore, MD, USA, Nov. 2016, pp. 301–306.

[25] R. D. Yates, Y. Sun, D. R. Brown, S. K. Kaul, E. Modiano, and S. Ulukus, "Age of Information: An Introduction and Survey," *IEEE J. on Sel. Areas in Commun.*, vol. 39, no. 5, pp. 1183–1210, Mar. 2021.

[26] S. Kaul, R. Yates, and M. Gruteser, "Real-time status: How often should one update?" in 2012 *Proc. IEEE INFOCOM*, Orlando, FL, USA, Mar. 2012, pp. 2731–2735.

[27] S. Kaul, M. Gruteser, V. Rai, and J. Kenney, "Minimizing age of information in vehicular networks," in 2011 8th Annu. IEEE Commun. Soc. Conf. Sensor, Mesh Ad Hoc Commun. Netw., Salt Lake City, UT, USA, Jun. 2011, pp. 350–358.

[28] Y. Inoue, "Analysis of the Age of Information with Packet Deadline and Infinite Buffer Capacity," in 2018 IEEE Int. Symp. Inf. Theory (ISIT), Vail, CO, USA, Jun. 2018, pp. 2639–2643.

[29] L. Huang and E. Modiano, "Optimizing age-of-information in a multi-class queueing system," in 2015 IEEE Int. Symp. Inf. Theory (ISIT), Hong Kong, China, Jun. 2015, pp. 1681–1685.

[30] A. Soysal and S. Ulukus, "Age of Information in G/G/1/1 Systems: Age Expressions, Bounds, Special Cases, and Optimization," *IEEE Trans. Inf. Theory*, vol. 67, no. 11, pp. 7477–7489, 2021.

[31] C. Kam, S. Kompella, and A. Ephremides, "Age of information under random updates," in 2013 IEEE Int. Symp. Inf. Theory, Istanbul, Turkey, Jul. 2013, pp. 66–70.

[32] R. D. Yates, "Status Updates through Networks of Parallel Servers," in 2018 IEEE Int. Symp. Inf. Theory (ISIT), Vail, CO, USA, Jun. 2018, pp. 2281–2285.

[33] A. M. Bedewy, Y. Sun, and N. B. Shroff, "Minimizing the Age of Information Through Queues," *IEEE Trans. Inf. Theory*, vol. 65, no. 8, pp. 5215–5232, 2019.

[34] X. Li and J. Hwu, "A Frequency Hopping Spread Spectrum Transmission Scheme For Uncoordinated Cognitive Radios," in 2009 IEEE Int. Conf. on Acous., Speech and Signal Proc. (ICASSP), Taipei, Taiwan, Apr. 2009, pp. 2345–2348.

[35] M. Bande and V. V. Veeravalli, "Adversarial Multi-user Bandits for Uncoordinated Spectrum Access," in 2019 IEEE Int. Conf. on Acous., Speech and Signal Proc. (ICASSP), Brighton, United Kingdom, May 2019, pp. 4514–4518.

[36] Z. Ding, R. Schober, P. Fan, and H. V. Poor, "Simple Semi-Grant-Free Transmission Strategies Assisted by Non-Orthogonal Multiple Access," *IEEE Trans. on Commun.*, vol. 67, no. 6, pp. 4464–4478, Mar. 2019.

[37] H. Jiang, Q. Cui, Y. Gu, X. Qin, X. Zhang, and X. Tao, "Distributed Layered Grant-Free Non-Orthogonal Multiple Access for Massive MTC," in 2018 IEEE 29th Annu. Int. Symp. on Pers., Indoor and Mobile Radio Commun. (PIMRC), Bologna, Italy, Sept. 2018, pp. 1–7.

[38] M. J. Youssef, V. V. Veeravalli, J. Farah, and C. Abdel Nour, "Stochastic Multi-Player Multi-Armed Bandits with Multiple Plays for Uncoordinated Spectrum Access," in 2020 IEEE 31st Annu. Int. Symp. on Pers., Indoor and Mobile Radio Commun. (PIMRC), London, United Kingdom, United Kingdom, Sept. 2020, pp. 1–7.

[39] M. A. Adjif, O. Habachi, and J. P. Cances, "Joint Channel Selection and Power Control for NOMA: A Multi-Armed Bandit Approach," in 2019 IEEE Wireless Commun. and Netw. Conf. Workshop (WCNC), Marrakech, Morocco, Morocco, Apr. 2019, pp. 1–6.

[40] D. P. Zhou and C. J. Tomlin, "Budget-Constrained Multi-Armed Bandits with Multiple Plays," *CoRR*, vol. abs/1711.05928, 2017.

[41] I. Bistritz and A. Leshem, "Distributed Multi-Player Bandits - a Game of Thrones Approach," in *Advances in Neural Inf. Process. Syst.*, S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, Eds., vol. 31. Curran Associates, Inc., 2018, pp. 7222–7232.
[42] K. Liu and Q. Zhao, "Distributed Learning in Multi-Armed Bandit With Multiple Players," *IEEE Trans. on Signal Process.*, vol. 58, no. 11, pp. 5667-5681, Nov. 2010.

[43] F. Man and W. Lenan, "Extension to Shannon's channel capacity —the theoretical proof," in *2007 6th Int. Conf. on Inf., Commun. Signal Process.*, 2007, pp. 1–4.

[44] M. Afshang and H. S. Dhillon, "Poisson Cluster Process Based Analysis of HetNets With Correlated User and Base Station Locations," *IEEE Trans. on Wireless Commun.*, vol. 17, no. 4, pp. 2417-2431, Jan. 2018.