# Reinforcement Learning-Based Antenna and Time Adaptation Energy-Efficient Strategies in 6G Networks

Julien Ali El Amine, Ibtissem Oueslati, Loutfi Nuaymi, Anne-Cécile Orgerie

<figure><img src="image_1.png" /></figure>

## ► To cite this version:

Julien Ali El Amine, Ibtissem Oueslati, Loutfi Nuaymi, Anne-Cécile Orgerie. Reinforcement Learning-Based Antenna and Time Adaptation Energy-Efficient Strategies in 6G Networks. ICC 2026 - IEEE International Conference on Communications, May 2026, Glasgow, United Kingdom. hal-05520433

HAL Id: hal-05520433

https://hal.science/hal-05520433v1

Submitted on 20 Feb 2026

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons CC BY-SA 4.0 - Attribution - ShareAlike - International License
# Reinforcement Learning-Based Antenna and Time Adaptation Energy-Efficient Strategies in 6G Networks

Julien Ali El Amine*, Ibtissem Oueslati†, Loutfi Nuaymi† and Anne-Cécile Orgerie‡

*College of Engineering and Technology, American University of the Middle East, Egaila, Kuwait

†IMT Atlantique, IRISA, UMR CNRS 6074, F-35700 Rennes, France

‡Univ. Rennes, Inria, CNRS, IRISA, Rennes, France, e-mail: anne-cecile.orgerie@irisa.fr

e-mail: julien.el-amine@aum.edu.kw, {ibtissem.oueslati, loutfi.nuaymi}@imt-atlantique.fr, and anne-cecile.orgerie@irisa.fr

**Abstract**—Among the strategies for improving the energy efficiency of 5G and beyond networks, sleep-mode control remains one of the most promising techniques for reducing the power consumption of base stations (BSs). The recently standardized 3GPP TR 38.864 BS energy model provides a realistic framework for analyzing multi-level sleep modes and energy-saving mechanisms in next-generation networks. Building upon this model, this paper investigates joint antenna and time adaptation for energy-efficient BS control. The proposed approach allows the BS to autonomously adjust both its temporal activity and antenna usage according to the traffic load. To this end, we design a reinforcement learning (RL) framework in which the BS acts as an intelligent agent that jointly selects the sleep-mode level and the number of active antennas. The cost function balances energy consumption and achievable throughput, while penalizing excessive power use and rate shortages. Simulation results based on the 3GPP TR 38.864 power model show that the proposed joint antenna-time adaptation achieves up to 50% power savings compared to time-only adaptation and about 94% compared to antenna-only adaptation, while maintaining satisfactory throughput performance. These results confirm that combining antenna deactivation with time adaptation provides an efficient and scalable solution for energy-aware 6G base-station control.

**Index Terms**—Energy consumption, quality of service, multi-level sleep mode, 5G, Q-learning

## I. INTRODUCTION

The transition toward 6G networks is expected to deliver ultra-dense connectivity and unprecedented performance levels, but it also intensifies the long-standing challenge of energy consumption in mobile systems. Base Stations (BSs) remain the main contributors to this issue, accounting for nearly 80% of the total network power usage [1]. During low-traffic periods, a large portion of hardware components—such as radio chains and antenna elements—continue operating despite minimal user demand, leading to unnecessary energy expenditure and lower overall Energy Efficiency (EE) [2], [3]. This highlights the need for adaptive and context-aware control strategies that can dynamically adjust BS operation according to real-time traffic conditions.

Sleep-mode management has been widely studied as a key mechanism to reduce static power consumption in next-generation BSs. By deactivating selected hardware compo-

nents during idle or lightly loaded intervals, multi-level sleep schemes—typically categorized as micro, light, and deep sleep—achieve tunable trade-offs between energy savings and wake-up delay [4], [5], [6], [7]. However, aggressive or poorly synchronized transitions may introduce excessive switching overheads and latency penalties, limiting the attainable EE improvement in dynamic traffic scenarios [8]. Consequently, time adaptation alone is insufficient to achieve energy proportionality under realistic load conditions.

Beyond sleep-mode schemes, dynamic radio-resource adaptation across domains—time, power, and antenna—has become a key approach to improving BS energy proportionality. Recent works have explored techniques such as antenna muting in massive-Multiple Input Multiple Output (MIMO) systems, adaptive power control, and short time-domain muting for micro-sleep opportunities [9], [10]. To enable consistent evaluation of these mechanisms, the 3GPP standardized the BS power model TR 38.864 (Release 18), defining active and multi-level sleep states (micro, light, deep) with their transition delays and energy costs [11]. This model also quantifies the contribution of components such as RF chains, baseband units, and antenna modules, facilitating hardware-aware analysis of energy-saving strategies. It has been used in several studies to evaluate efficient RAN operation: Borja et al. [12] analyzed DTX/DRX cycles and achieved 40–55% power reduction, while Laselva et al. [9] reported over 30% savings via joint antenna and power adaptation. Analytical models such as Elayoubi et al. [13] further highlight that micro-sleep provides the best trade-off between energy and latency. Overall, these studies confirm that standardized power modeling combined with multi-domain adaptation is fundamental to realistic evaluation of BS energy efficiency in 6G networks.

Building upon the standardized framework of the 3GPP TR 38.864 power model, recent research has increasingly explored the use of Reinforcement Learning (RL) to achieve autonomous and adaptive energy management in base stations. Unlike static rule-based control, RL agents can continuously learn from network dynamics to determine optimal operating states—such as selecting appropriate sleep levels, adjusting transmission power, or activating antenna chains—according
to traffic variations [14], [15], [16]. These learning-driven strategies have demonstrated significant potential for improving long-term EE while preserving Quality of Service (QoS) constraints. However, most existing studies remain limited to a single adaptation domain (e.g., time-domain sleep scheduling or power control) or rely on simplified power-consumption models that neglect state-transition overheads and component-level dependencies [17], [4]. In response to these challenges, we design a Q-learning-based control scheme grounded on the 3GPP TR 38.864 model to enable coordinated time-antenna adaptation of BS operation. Our agent learns to balance energy consumption and performance by simultaneously adjusting the sleep-mode level and the number of active antennas according to traffic dynamics.

The main contributions of this paper are summarized as follows:

* We introduce a new energy-efficient BS control framework that combines the 3GPP TR 38.864 power model with reinforcement learning, enabling joint optimization of time-domain sleep modes and antenna-domain adaptation.

* We design a lightweight Q-learning algorithm capable of autonomously selecting, at each decision epoch, the most suitable sleep level and number of active antennas, guided by a cost function that balances normalized power consumption and achievable throughput.

* We perform a 3GPP-compliant evaluation campaign to validate the proposed method and benchmark it against conventional single-domain adaptations (sleep-only or antenna-only), highlighting the achievable trade-offs between energy efficiency and service quality.

The remainder of this paper is organized as follows. Section II introduces the 3GPP TR 38.864 power model. Section III presents the system model and the problem formulation. Section IV details the proposed Q-learning control mechanism. Simulation results and key findings are discussed in Section V, followed by conclusions in Section VI.

## II. BASE STATION POWER MODEL

To assess the energy consumption of a gNodeB (gNB), we employ the 5G base station power model introduced by 3GPP in [11]. In this paper, we use the terms gNB and BS interchangeably. This model captures the effect of radio configuration parameters on power usage, acknowledging that a gNB's energy demand largely depends on how its radio units are deployed and operated. Table I summarizes three reference configurations. Based on their characteristics, Sets 1 and 2 represent macro base stations, whereas Set 3 corresponds to a micro base station.

The specification also distinguishes between two BS categories. Category 1 exhibits power behavior comparable to that of user equipment (UE), while Category 2 represents conventional base stations. Since our focus is on network-side energy efficiency, we restrict our analysis to Category 2 BSs. Moreover, the 3GPP model defines several operational states for the gNB, including active DownLink (DL), active UpLink

(UL), micro sleep and deep sleep. In the active modes, the BS transmits or receives traffic. When no DL or UL activity is required, the BS can enter a sleep state allowing partial hardware deactivation to save energy. Relative power values for Category 2 BS states are listed in Table II. Throughout this work, we use the reference configuration Set 1 (FR1) and refer to micro, light and deep sleep as SM1, SM2 and SM3, respectively.

Transition times and additional transition energies between these power states are shown in Table III. Each transition period includes both reactivation and deactivation phases. The micro-sleep mode acts as a standby condition, effectively representing the BS's static power when operational; hence, its transition time and energy overheads are neglected.

In the active state, several factors influence power consumption: the number of active transmit/receive antennas, the occupied bandwidth or number of Resource Blocks (RBs), the count of Component Carriers (CCs), the Power Spectral Density (PSD) and the number of DL/UL symbols per slot. Accordingly, the total power in the downlink active state can be expressed as the sum of static and dynamic components as follows:

$$P^{DL} = P_{static}^{DL} + P_{dynamic}^{DL} \qquad (1)$$

The dynamic component varies with load conditions and scales as:

$$P_{dynamic}^{DL} = s_a \cdot (P_{dyn,ante} + \frac{s_f \cdot s_p}{\eta(s_f, s_p)} \cdot P_{dyn,joint}), \quad (2)$$

where $s_a, s_f, s_p$ are respectively the fraction of active TX antenna, the resource usage ratio in the frequency domain and the ratio of PSD per transmit antenna, respectively, for a scaled DL transmission compared to the reference configuration. The Power Amplifier efficiency factor $\eta(s_f, s_p)$ is assumed to be equal to 1. According to [11]:

$$P_{dyn,ante} = A \cdot (P_{active}^{DL} - P_{static}^{DL}), \qquad (3)$$

$$P_{dyn,joint} = (1 - A) \cdot (P_{active}^{DL} - P_{static}^{DL}), \qquad (4)$$

with $A \in \{0.1, 0.4, 0.7\}$. Following [11], in this work, we use $A = 0.4$. Similar to [9], we introduce the scaling factor $s_t$. It denotes the fraction of time where the BS is active. The overall BS power consumption can thus be expressed as:

$$P^{DL} = s_t \cdot (P_{static}^{DL} + P_{dynamic}^{DL}) + s_{SM1} \cdot P_{micro} + s_{SM2} \cdot P_{light} + s_{SM3} \cdot P_{deep}, \qquad (5)$$

Such that: $s_{SM1} + s_{SM2} + s_{SM3} = (1 - s_t)$,

where $s_{SM1}$ and $s_t$ denote the fraction of time the BS spends in sleep mode $i$ and active state, respectively. $P_{micro}$, $P_{light}$ and $P_{deep}$ are the corresponding power consumption levels of the BS in micro, light and deep sleep modes, and are denoted by $P_1$, $P_2$ and $P_3$ in Table II.
<table><caption>TABLE I
REFERENCE CONFIGURATION FOR BS POWER CONSUMPTION MODEL.</caption><thead><tr><th>Parameters</th><th>Set 1, FR1</th><th>Set 2, FR2</th><th>Set3, FR2</th></tr></thead><tbody><tr><td>Duplex</td><td>TDD</td><td>FDD</td><td>TDD</td></tr><tr><td>System Bandwidth</td><td>100MHz</td><td>200MHz</td><td>100MHz</td></tr><tr><td>Subcarrier spacing</td><td>30kHz</td><td>15kHz</td><td>120kHz</td></tr><tr><td>Number of TRP</td><td>1</td><td>1</td><td>1</td></tr><tr><td>Number of DL TX RUs</td><td>64</td><td>32</td><td>2</td></tr><tr><td>Total DL power level</td><td>55dBm</td><td>49dBm</td><td>33dBm</td></tr><tr><td>Total number of UL Rx RUs</td><td>64</td><td>32</td><td>2</td></tr></tbody></table>

<table><caption>TABLE II
RELATIVE POWER VALUES *P* FOR REFERENCE CONFIGURATION SET 1, SET 2 AND SET 3 OF BS CATEGORY 2</caption><thead><tr><th>Power state</th><th>Set 1</th><th>Set 2</th><th>Set 3</th></tr></thead><tbody><tr><td>Deep sleep (<i>P</i><sub>1</sub>)</td><td>1</td><td>1</td><td>1</td></tr><tr><td>Light sleep (<i>P</i><sub>2</sub>)</td><td>2.1</td><td>2.1</td><td>2.1</td></tr><tr><td>Micro sleep (<i>P</i><sub>3</sub>)</td><td>5.5</td><td>5</td><td>3</td></tr><tr><td>Active DL (<i>P</i><sub>4</sub>)</td><td>32</td><td>26</td><td>17.6</td></tr><tr><td>Active UL (<i>P</i><sub>5</sub>)</td><td>6.5</td><td>5.8</td><td>4.2</td></tr></tbody></table>

<table><caption>TABLE III
TRANSITION TIME *T* AND ADDITIONAL TRANSITION ENERGY *E* FOR REFERENCE CONFIGURATION SET 1, SET 2 AND SET 3 OF BS CATEGORY 2.</caption><thead><tr><th>Power state</th><th>Transition time</th><th>Transition energy Relative power unit .ms</th></tr></thead><tbody><tr><td>Deep sleep (<i>T</i><sub>t1</sub>, <i>E</i><sub>1</sub>)</td><td>10 s</td><td>17000</td></tr><tr><td>Light sleep (<i>T</i><sub>t2</sub>, <i>E</i><sub>2</sub>)</td><td>640 ms</td><td>1088</td></tr><tr><td>Micro sleep (<i>T</i><sub>t3</sub>, <i>E</i><sub>3</sub>)</td><td>0 ms</td><td>0</td></tr></tbody></table>

### III. SYSTEM MODEL

This study considers a single gNB serving *K* users with DL data transmissions. To reduce energy consumption, the gNB can transition between different sleep modes - SM1 (micro sleep), SM2 (light sleep) and SM3 (deep sleep) - whose operational characteristics and power consumption profiles are detailed in Section II. Beyond the sleep mode mechanism, the gNB can also dynamically adjust the number of active antennas in response to the varying traffic load and service demands. By deactivating certain antenna elements when traffic is low, the system can significantly lower its circuit and RF-chain power consumption without severely compromising service quality.

In the proposed model, user arrivals follow a Poisson process and their service durations (holding times) are modeled using a geometric distribution. Each user *u* requires a minimum rate threshold *R*<sub>min</sub>. Given the total number of antennas *N*<sub>tot</sub> and the noise spectral density *N*<sub>0</sub>, the instantaneous rate of a user (*r*<sub>u</sub>) and the average cell throughput (*R*<sub>avg</sub>) can be expressed as follows:

$$r_u = s_a \times B \times \log_2(1 + \text{SNR}), \quad (6)$$

$$R_{\text{avg}} = \frac{\sum_{u=1}^{K} r_u}{K} \quad (7)$$

where *s*<sub>a</sub> represents the antenna activation ratio, i.e., the fraction of active antennas (*N*<sub>active</sub>/*N*<sub>tot</sub>), *B* is the total gNB

bandwidth, SNR is the signal-to-noise ratio, and *K* denotes the number of users connected during a given time slot.

#### A. Problem Formulation

The primary objective of this work is to analyze the trade-off between energy consumption and throughput performance, aiming to enhance the overall system EE. At each time slot *t*, the gNB determines two key parameters: the operational mode *m*<sub>*t*</sub> ∈ {Active, SM1, SM2, SM3} and the antenna activation ratio *s*<sub>*a*,*t*</sub> ∈ [0.1], based on the current network conditions. The optimization problem can thus be formulated as follows:

$$\max_{\{m_t, s_{a,t}\}_{t=1}^T} \sum_{t=1}^T \left[ -w \frac{P(m_t, s_{a,t})}{P_{\max}} + (1-w) \frac{R_{\text{avg}}(m_t, s_{a,t})}{R_{\min}} - \lambda_s \text{ short}_{u,t} - \lambda_o \text{ over}_{u,t} \right] \quad (8)$$

where *P*(*m*<sub>*t*</sub>, *s*<sub>*a*,*t*</sub>) is the total power consumption under mode *m*<sub>*t*</sub> and antenna activation ratio *s*<sub>*a*,*t*</sub> derived from eq. (5), while *P*<sub>max</sub> is the power in the fully active state (*m*<sub>*t*</sub> = Active and *s*<sub>*a*,*t*</sub> = 1). The weighting factor *w* ∈ [0, 1] adjusts the balance between minimizing power consumption and maximizing throughput. The coefficients *λ*<sub>s</sub> and *λ*<sub>*o*</sub> are penalty terms that account for under-service and over-provisioning conditions, represented by the normalized variables short<sub>*u*,*t*</sub> and over<sub>*u*,*t*</sub>, respectively. The term short<sub>*u*,*t*</sub> quantifies the deficit between the achieved and required rate, whereas over<sub>*u*,*t*</sub> captures excessive resource allocation beyond *R*<sub>min</sub>.

The optimization problem is modeled as a Markov Decision Process (MDP), where the gNB observes a state *s*(*t*), takes an action *a*(*t*), and transitions to a new state *s*(*t* + 1). The state space is defined as *S* = *M*, *U*, *τ*, *A*, where:

* *M* corresponds to the BS's current mode,

* *U* represents the traffic load,

* *τ* denotes the remaining time until the end of a mode transition, and

* *A* refers to the previously used antenna activation ratio at time slot *t* − 1.

To solve the MDP, one must derive the transition probabilities and the corresponding state-action rewards. A conventional dynamic programming approach could yield to the optimal value function, but this requires complete knowledge of the environment, which is unrealistic in practical, real-time systems. Therefore, a Q-learning algorithm is employed to enable the BS to learn the optimal control policy through continuous interaction with the environment, eliminating the need for prior knowledge of the system dynamics.

### IV. PROPOSED ENERGY-EFFICIENT ALGORITHM

In this work, we consider a single-BS scenario where Q-learning is applied as an online optimization technique to jointly manage the sleep-mode level, the antenna activation ratio, and the time-slot activity of the BS. By continuously interacting with its environment, the BS progressively learns an optimal policy that determines the appropriate operating mode (Active, SM1, SM2, or SM3) and the proportion of
active antennas to be used during each time slot. The learning objective is to maximize the cumulative expected reward, capturing the trade-off between energy efficiency and service performance over multiple episodes.

At each time step, the BS observes the current system state, which includes the number of active users, the previously selected sleep-mode level, the last applied antenna activation ratio, the remaining transition time between two consecutive sleep states, and the fraction of time spent active during the previous slot. Based on these local observations, the BS chooses an action $a^t$ from the action set $\mathcal{A}$, consisting of the operational mode and the fraction of active antennas, to optimize energy efficiency while satisfying user rate requirements. The system then transitions to a new state $s^{t+1}$ and returns a reward $r^t$, which is used to update the Q-value $Q(s^t, a^t)$—representing the expected benefit of taking action $a^t$ in state $s^t$—according to the update rule:

$$Q(s^t, a^t) \leftarrow Q(s^t, a^t) + \alpha[r^t + \gamma \max_{a \in \mathcal{A}} Q(s^{t+1}, a) - Q(s^t, a^t)], \quad (9)$$

where $\alpha$ is the learning rate controlling how rapidly new information replaces prior estimates, and $\gamma \in [0, 1]$ is the discount factor that determines the importance of future rewards.

During the learning phase, the BS follows an $\epsilon$-greedy policy to balance exploration and exploitation. With probability $1 - \epsilon$, the BS exploits its current knowledge by choosing the action associated with the maximum Q-value; with probability $\epsilon$, it explores alternative actions randomly:

$$a^t = \begin{cases} \underset{a \in \mathcal{A}}{\operatorname{argmax}} Q(s^t, a), & \text{if } y > \epsilon, \\ \operatorname{rand}(\mathcal{A}), & \text{otherwise.} \end{cases} \quad (10)$$

Through iterative exploration, the BS gradually identifies the optimal combination of sleep mode, antenna activation, and time-slot utilization, effectively avoiding local minima and converging toward a policy that minimizes energy consumption while maintaining user-level performance.

## V. RESULTS AND ANALYSIS

To assess the performance of the proposed Q-learning framework for joint antenna adaptation and multi-level sleep-mode control, the algorithm was implemented and evaluated using a Python-based simulator developed in compliance with the 3GPP TR 38.864 specifications. The scenario considers a single BS positioned at the center of a circular cell with a coverage radius of 40 m. User arrivals follow a Poisson process with rate $\lambda_a = 0.5$ users/s, while user session durations are geometrically distributed with an average duration of $\lambda_d = 15$ s. Upon arrival, users are randomly and uniformly distributed within the cell coverage area, and the number of active users at any time emerges from the interaction between the arrival and departure processes.

The simulation runs for 200 episodes, each divided into 10,000 time slots of 640 ms. The BS can operate in four modes—Active, Micro Sleep, Light Sleep, and Deep

<figure><img src="image_3.png" /><figcaption>Fig. 1. Convergence of Q-values for different values of w.</figcaption></figure>

Sleep—each associated with distinct power consumption and wake-up delay values, as specified in [11]. The main simulation parameters are summarized in Table IV.

<table><caption>TABLE IV
SIMULATION AND MODEL PARAMETERS</caption><thead><tr><th>Parameter</th><th>Value</th></tr></thead><tbody><tr><td>Time slot duration (T)</td><td>640 ms</td></tr><tr><td>System bandwidth (B)</td><td>100 MHz</td></tr><tr><td>Users' average arrival rate (λ<sub>a</sub>)</td><td>0.5 user/sec</td></tr><tr><td>Users' average departure rate (λ<sub>d</sub>)</td><td>15 seconds</td></tr><tr><td>Minimum user rate (R<sub>min</sub>)</td><td>20 Mbit/s</td></tr><tr><td>Static DL power (P<sub>static</sub>)</td><td>P<sub>3</sub> [11]</td></tr><tr><td>Dynamic power weight (A)</td><td>0.4 [11]</td></tr><tr><td>Sleep-mode transition times</td><td>As per 3GPP TR 38.864</td></tr><tr><td>Q-learning learning rate (α)</td><td>0.5</td></tr><tr><td>Discount factor (γ)</td><td>0.9</td></tr></tbody></table>

Relative power levels from TR 38.864 are converted into absolute values (Watts) by normalizing with respect to the active downlink power state $P_4 = 32$ (relative units), set to correspond to 40 W. The resulting scale factor of 1.25 is applied to all states, as shown in (11):

$$\text{Scale factor} = \frac{40 \text{ W}}{32} = 1.25 \Rightarrow P_i^{(\text{W})} = P_i^{(\text{rel})} \times 1.25. \quad (11)$$

The converted power values for the Category 2 BS are summarized in Table V.

<table><caption>TABLE V
CONVERTED POWER VALUES (CATEGORY 2 BS, WITH $P_4 = 40$ W)</caption><thead><tr><th>Power State</th><th>Relative Value</th><th>Power (W)</th></tr></thead><tbody><tr><td>Deep Sleep ($P_1$)</td><td>1</td><td>1.25</td></tr><tr><td>Light Sleep ($P_2$)</td><td>25</td><td>31.25</td></tr><tr><td>Micro Sleep ($P_3$)</td><td>5.5</td><td>6.88</td></tr><tr><td>Active DL ($P_4$)</td><td>32</td><td>40.0</td></tr><tr><td>Active UL ($P_5$)</td><td>6.5</td><td>8.13</td></tr></tbody></table>

Figure 1 shows the convergence behavior of the proposed joint antenna-time adaptation framework under three weight settings $w \in \{0.1, 0.5, 0.9\}$. In all cases, the sum of maximum Q-values increases rapidly during the initial episodes and then stabilizes, confirming the effective learning and convergence
<figure><img src="image_4.png" /><figcaption>Fig. 2. Normalized BS power consumption as a function of $w$ for the three adaptation strategies.</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>Fig. 3. Average user rate as a function of the weighting parameter $w$ for the three adaptation strategies.</figcaption></figure>

of the Q-table. Smaller $w$ values converge faster and reach higher plateaus, as the controller prioritizes throughput maximization and explores a wider range of actions. In contrast, larger $w$ values lead to lower steady-state levels and slower convergence, reflecting the stronger emphasis on power saving in the objective function formulation.

Figure 2 compares the normalized BS power consumption of the proposed joint antenna-time adaptation with antenna-only and time-only benchmarks as a function of the weighting factor $w$. As $w$ increases, the objective favors energy saving; consequently, the strategies that include time adaptation (joint and time-only) exhibit a pronounced reduction in power, whereas the antenna-only curve remains high. Across all $w$, the joint antenna-time scheme attains the lowest consumption, and the gap to the time-only baseline highlights the additional savings enabled by adaptive antenna activation. In contrast, the antenna-only strategy cannot exploit sleep opportunities: without time-domain micro/light/deep sleep, large static components remain powered, so muting antennas alone yields limited benefits. Overall, the results confirm that time adaptation is the primary driver of energy reduction, while combining it with antenna adaptation delivers the most favorable energy-performance trade-off under the reward in (8).

<figure><img src="image_6.png" /><figcaption>Fig. 4. Average percentage of base station modes as a function of $w$.</figcaption></figure>

Figure 3 compares the proposed joint antenna-time adaptation strategy with the antenna-only and time-only benchmarks in terms of the average normalized user rate as a function of the weighting factor $w$. The results clearly reveal the trade-off between throughput and energy efficiency. As $w$ increases, the controller gives greater priority to energy saving, which results in a noticeable reduction of the achievable rate for the schemes involving time adaptation. The joint antenna-time adaptation shows the steepest decline in throughput since both temporal activity and antenna usage are jointly reduced, whereas the time-only case presents a smoother degradation. In contrast, the antenna-only strategy achieves a comparatively higher throughput, since the BS stays active throughout the operation and only varies the fraction of active antennas. Comparing Fig. 2 and Fig. 3 highlights that the throughput

degradation observed for the proposed joint antenna-time adaptation, compared with the benchmark schemes, is a direct consequence of the controller's effort to minimize energy consumption. Nevertheless, this reduction remains acceptable, as the main objective of the proposed design is to enhance energy efficiency.

Figure 4 illustrates the distribution of base station activity among the different operating modes as a function of the weighting factor $w$ for both the proposed joint antenna-time adaptation and the time-only adaptation scheme. For both strategies, increasing $w$ progressively shifts the operation from the Active state toward deeper sleep levels, reflecting the controller's growing preference for energy saving. In the joint antenna-time adaptation, this transition is even more pro-
<figure><img src="image_7.png" /><figcaption>Fig. 5. Average antenna usage as a function of w.</figcaption></figure>

nounced: the share of Active mode decreases rapidly, while the portions of Light and Deep sleep modes increase substantially. Compared with the time-only case, the joint approach reaches deeper sleep more frequently and for longer durations, thanks to the additional flexibility offered by antenna deactivation. These results align with the strong power reduction observed in Fig. 2, confirming that combining antenna and time adaptation enables a more effective exploitation of the multi-level sleep hierarchy for enhanced energy efficiency.

Figure 5 depicts the average antenna utilization as a function of the weighting factor *w* for both the proposed joint antenna-time adaptation and the antenna-only benchmark scheme. The results show a clear downward trend for both strategies as *w* increases, confirming that the controller gradually limits antenna activation to reduce energy consumption. The proposed joint antenna-time adaptation achieves the lowest average antenna usage across all values of *w*, as it simultaneously exploits temporal inactivity and antenna deactivation to minimize power. In contrast, the antenna-only strategy maintains a higher level of antenna utilization since the BS remains continuously active, even when the traffic load decreases. These results demonstrate that combining antenna and time adaptation allows for a more flexible and energy-efficient control of radio resources by dynamically adjusting both transmission duration and the number of active antennas.

## VI. CONCLUSION AND FUTURE WORK

This paper proposed a Q-learning-based framework for energy-efficient BS control, leveraging the 3GPP TR 38.864 power model to account for multi-level sleep modes and antenna deactivation. The proposed approach jointly adapts the antenna configuration and the BS activity duration, allowing the controller to intelligently balance energy consumption and throughput. Simulation results demonstrated that antenna and time adaptation jointly enable significant energy savings, whereas single-domain adaptation (antenna-only or time-only) achieves limited performance improvements. As future work, we plan to extend this framework toward a joint

time-frequency-antenna adaptation strategy to further enhance flexibility under dynamic traffic conditions. This extension will allow us to analyze how frequency adaptation, combined with antenna and time control, can further improve the energy-rate trade-off and strengthen the overall efficiency of the BS management policy. Additionally, we aim to evaluate the proposed approach under other BS power models.

## ACKNOWLEDGMENT

This work has benefited from a government grant managed by the Agence Nationale de la Recherche under the France 2030 program, reference "ANR-22-PEFT-0008".

## REFERENCES

[1] Meysam Masoudi, Mohammad Galal Khafagy, et al. Green mobile networks for 5G and beyond. *Ieee Access*, 7, 2019.

[2] K. N. R. Surya Vara Prasad, Ekram Hossain, and Vijay K. Bhargava. Energy Efficiency in Massive MIMO-Based 5G Networks: Opportunities and Challenges. *IEEE Wireless Communications*, 24(3):86-94, 2017.

[3] Emil Björnson, Jakob Hoydis, and Luca Sanguinetti. *Massive MIMO Networks: Spectral, Energy, and Hardware Efficiency*. Now Publishers Inc., 2017.

[4] Ali El Amine, Paolo Dini, and Loutfi Nuaymi. Reinforcement Learning for Delay-Constrained Energy-Aware Small Cells with Multi-Sleeping Control. In *IEEE International Conference on Communications Workshops (ICC Workshops)*, pages 1-6, 2020.

[5] F. E. Salem et al. Optimal Policies of Advanced Sleep Modes for Energy-Efficient 5G Networks. In *IEEE NCA*, 2019.

[6] Fatima Salahdine, Johnson Opadere, Qiang Liu, Tao Han, Ning Zhang, and Shaohua Wu. A survey on sleep mode techniques for ultra-dense networks in 5G and beyond. *Computer Networks*, 201:108567, 2021.

[7] Jann Camilo Sánchez Huertas, Greta Vallero, Loutfi Nuaymi, and Anne-Cécile Orgerie. Review and classification of the use of SMs for energy saving in next-generation radio access networks. *Computer Networks*, 273:111745, 2025.

[8] Taekyung Kim, Dongyoon Kim, Jihwan Lee, and Jinho Choi. An Energy-Efficient Multi-Level Sleep Strategy for Periodic Uplink Transmission in Private 5G Networks. *Sensors*, 23(21):8928, 2023. https://pmc.ncbi.nlm.nih.gov/articles/PMC10674620/.

[9] D. Laselva, S. Hakimi, M. Lauridsen, B. Khan, D. Kumar, and P. Mogensen. On the Potential of Radio Adaptations for 6G Network Energy Saving. In *EuCNC/6G Summit*, 2024.

[10] T. Islam, D. Lee, and S. S. Lim. Enabling Network Power Savings in 5G-Advanced and Beyond. *IEEE JSAC*, 41(6):1888-1899, 2023.

[11] 3GPP. TR 38.864 v18.1.0: Study on network energy savings for nr. Technical report, 3rd Generation Partnership Project, March 2023.

[12] Juan Guillermo Borja, Philipp Bruhn, and Marina Petrova. Towards 6G: Leveraging Advanced Sleep Modes to Improve Energy Performance. In *IEEE International Conference on 6G Networking (6GNet)*, pages 112-116, 2024.

[13] Sana Elayoubi, Julien Monteil, and Anastasios Aravanis. A Queuing Model for Short Time-Scale Sleep Mode Evaluation in 5G Base Stations. In *IEEE Mediterranean Communication and Computer Networking Conference (MeditCom)*, 2025.

[14] Ali El Amine, Jean-Paul Chaiban, Hussein Al Haj Hassan, Paolo Dini, Loutfi Nuaymi, and Roger Achkar. Energy optimization with multi-sleeping control in 5G heterogeneous networks using reinforcement learning. *IEEE Transactions on Network and Service Management*, 19(4):4310-4322, 2022.

[15] R. Zheng, X. Chen, and L. Wang. Reinforcement Learning Driven Base-Station Sleep Scheduling for Green Networks. *IEEE Transactions on Wireless Communications*, 19(7):4633-4646, 2020.

[16] M. Choi et al. Cell On/Off Parameter Optimization for Saving Energy via Reinforcement Learning. In *IEEE GLOBECOM*, 2021.

[17] Hadi Sayed, Ali El Amine, Hussein Al Haj Hassan, Loutfi Nuaymi, and Roger Achkar. Reinforcement learning for radio resource management of hybrid-powered cellular networks. In *International Conference on Wireless and Mobile Computing, Networking and Communications (WiMob)*, pages 40-46, 2019.