Kyushu University Platformof Inter/Transdisciplinary
Energy Research

Energy-Efficient Reinforcement Learning-Based UE Pairing in Non-Orthogonal
Multiple Access Wireless Communication Systems
Ahmad Gendia1, Osamu Muta2
1Graduate School of Information Science and Electrical Engineering, Kyushu University
2Center of Japan-Egypt Cooperation in Science and Technology, Kyushu University

1. Introduction
Non-orthogonal multiple access (NOMA) allows multiple user equipment (UE) to simultaneously share the same
resource blocks using varying levels of transmit power at the base station (BS) side. Proper selection of candidate
users for pairing over the same resource block is critical for an efficient utilization of the available resources. In
this article, a reinforcement learning (RL)-based scheme is proposed for user pairing in downlink NOMA systems,
where optimized UE selection is accomplished by the continuous interaction between an RL agent and the NOMA
environment with the aim of increasing the total system throughput while conserving the energy consumed for
NOMA transmissions.
Keywords : Wireless communication networks, NOMA, Reinforcement Learning, Energy efficiency.

NOMA Networks

Agent-Environment Dynamic Interaction

 NOMA can achieve
very high spectrumefficiency operation
by aggregating thedata of multiple UEs
overthesame
resource block (RB).
 NOMA can utilize
energyefficiently
through proper UE
pairing so as tocombat the inherent
interference.

Fig. 1: Downlink NOMA system transmission

System Model❑ Consider a downlink NOMA system where 𝑁 candidate UEs arerequesting popular contents (e.g. popular internet data) for
download:
1. The BS has to first fetch the requested information via
capacity-limited backhaul link to the core network.
2. For 𝑀-user NOMA transmission over a certain RB, 𝑀 < 𝑁, theBS then compiles and forwards a NOMA message containing
the content of the 𝑀 UEs with maximum potential
contribution to the system throughput:
𝑀𝛼𝑚 ℎ𝑚 2
𝑅=෍min 𝐶, 𝐵 𝑙𝑜𝑔2 1 + 𝑚−1
2 + 𝜎2σ𝛼ℎ𝑚=1𝑖 𝑚1
where 𝐶 is the backhaul capacity allocated for an active UE, 𝐵 is
the transmission bandwidth, and 𝜎 2 is the noise power.

Fig. 2: Top-level view of the proposed scheme The agent takes an action based on its internal policy, receives a rewardand makes a transition to a new state. This dynamic interaction enables
the agent to adapt to the effects of the various factors within theenvironment that impact the sum-rate and energy consumption.

Simulation Results➢The proposed scheme is Fig. 3: Normalized sum-rate vs transmit power.
compared to the optimal
andrandom
pairing
approaches as well as thechannel-sorting
pairing(CSP) scheme where UEs
with high channel-gain
separation are paired [2].
➢The RL agent learns tooptimize the selection ofpaired UEs to utilize
available energy efficiently.
[2] H. Zhang, D. Zhang, W. Meng, and C. Li, “User pairing algorithm with sic in nonorthogonal multiple access system,” in IEEE ICC, pp. 1–6.

Reinforcement Learning-Based UE PairingSummary➢ The proposed scheme deploys a caching unit at the BS side tostore high-demand contents to alleviate the data throttlingcaused by the backhaul link [1].
➢ An RL-based agent interacts with the NOMA environment tolearn the pairing decisions resulting in higher sum rates andoptimized energy consumption for the activated UEs:

▪ We have proposed an RL-based scheme for user-pairing in NOMAsystems [3-4] with a limited-capacity backhaul connection toachieve near-optimal sum-rate levels while conserving the energy
consumed for UE data transmission as indicated in the table below.

✓ Initially, the agent starts with a random policy that maps thestate of the environment to a random pairing action.
✓ the action is executed and a corresponding reward is observed.
✓ The agent then updates its policy based on the cumulative
rewards observed over many interactions.
[1] F. R. Yu and Y. He, “Deep reinforcement learning for interference alignment wirelessnetworks,” in Deep Reinforcement Learning for Wireless Networks. Springer, 2019, pp. 21–44.

SchemeTransmit power savings (dB)

OptimalProposedCSP

15.5
1510.75

Power saving = 𝟏𝟎𝐥𝐨𝐠 𝟏𝟎 ( 𝜸𝒓 /𝜸𝐬 ) dB
𝛾𝑟 is the random pairing transmit power threshold and 𝛾𝑠 ∈{𝛾𝑂𝑝𝑡𝑖𝑚𝑎𝑙 , 𝛾𝑃𝑟𝑜𝑝𝑜𝑠𝑒𝑑 , 𝛾𝐶𝑆𝑃 } is the transmit power of the respective scheme.
 The proposed scheme achieves 6 bps/Hz at 15 dB transmit power saving.
[3] A. H. Gendia, Osamu Muta, “Reinforcement Learning-Based User Pairing in NOMA Systems,” inIEICE general conference, March 2021.
[4] __, “User Pairing in NOMA Systems via Reinforcement Learning Selection,” in RCS, March 2021.