# On the Potential of Dynamic RF Channel Configuration for Energy Efficiency Optimization

Antoine Dejonghe, Safaa Driouech, Zwi Altman, Francesco De Pellegrini

► To cite this version:

Antoine Dejonghe, Safaa Driouech, Zwi Altman, Francesco De Pellegrini. On the Potential of Dynamic RF Channel Configuration for Energy Efficiency Optimization. Annual International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC), IEEE, Sep 2024, Valencia, Spain, France. hal-04601455

HAL Id: hal-04601455

https://hal.science/hal-04601455

Submitted on 5 Jun 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# On the Potential of Dynamic RF Channel Configuration for Energy Efficiency Optimization

Antoine Dejonghe, Safaa Driouech, Zwi Altman, Francesco De Pellegrini

► To cite this version:

Antoine Dejonghe, Safaa Driouech, Zwi Altman, Francesco De Pellegrini. On the Potential of Dynamic RF Channel Configuration for Energy Efficiency Optimization. Annual International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC), IEEE, Sep 2024, Valencia, Spain, France. hal-04601455

HAL Id: hal-04601455

https://hal.science/hal-04601455v1

Submitted on 5 Jun 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# On the Potential of Dynamic RF Channel Configuration for Energy Efficiency Optimization

Antoine Dejonghe*†, Safaa Driouech*, Zwi Altman*, and Francesco de Pellegrini†

***Abstract***—The deployment of large antenna arrays in Massive Multiple Input Multiple Output (M-MIMO) systems substantially improves mobile networks' performance. Nevertheless, the increase in performance brought by additional antennas is often counterbalanced by an increase in Power Consumption (PC) caused by the use of supplemental hardware resources supporting additional Radio Frequency (RF) channels. In M-MIMO networks, switching on/off RF channels and muting the associated antennas according to load conditions is known to be an efficient Energy Saving (ES) mechanism. This paper introduces a new RF channels switch on/off solution to maximize the Energy Efficiency (EE) under Quality of Service (QoS) constraints. The proposed solution is based on a MAB algorithm, which appears simpler than state of the art approaches and can be easily implemented in real systems. The algorithm leverages the quasiconcave shape of the EE metric to sequentially select the optimal antenna array configuration — i.e., the number of RF channels in both azimuth and elevation — from a predefined set of configurations. Extensive system-level simulations demonstrate that the proposed algorithm achieves a significant EE gain over a baseline solution.

***Index Terms***—Massive MIMO, RF channel configuration, energy efficiency, energy saving, Multi-Armed Bandit

## I. INTRODUCTION

5G and beyond Radio Access Network (RAN) use M-MIMO technology as a mean to enhance User Equipments (UEs)' performance, leveraging on large two dimensional antenna arrays. M-MIMO systems perform beamforming in order to focus the overall transmitted power towards UEs via a suitable weighting of the signal sent by each radiating antenna [1]. In digital M-MIMO architecture, for example, beamforming is enabled by feeding each antenna using a dedicated RF channel, namely an electronic circuit comprising different components such as digital-to-analog converters and power amplifiers. Beamforming capabilities and hence UEs' performance improve with the number of antennas, at the cost of increased Energy Consumption (EC) due to the additional RF channels.

The reduction of EC is a key challenge for the sustainability of future networks [2]. It can be performed by introducing ES mechanisms, including switching off different components of the network. The associated time scale for on/off switching can range from hours and above in case of the entire on/off switch of a cell, and scale down to the range of tens of microseconds to tens of milliseconds in the case of Advanced Sleep Modes. At intermediate time-scales, one can consider on/off switching

of carriers or RF channels. A detailed overview covering different aspects of EE in 5G RAN is presented in [3].

In M-MIMO systems, under suitable traffic load conditions, significant EC savings are obtained by switching on/off certain RF channels and muting the radiating antennas they feed. This ES mechanism is addressed in various publications as an optimization problem: it consists in finding the number of RF channels that maximizes EE, namely the ratio between the transmitted data volume and the corresponding EC [4]. It is noted that in operational systems, one may need to solve a constrained EE optimization problem taking into account QoS constraints. Early contributions [5]–[7] address the EE optimization problem using EE analytic approximations in conjunction with heuristics. For example, [5] presents an alternating optimization algorithm enabling to sequentially optimize the transmit power and the numbers of active RF channels and scheduled UEs. The authors prove that the EE analytic approximation is a quasiconcave function of the optimization parameters. Hence, this guarantees the convergence of the algorithm's iterations to the optimal solution. Approaches based on analytic approximations usually assume a simplified channel model, namely Rayleigh fading, which neglects certain propagation characteristics such as Angular Spread (AS), and in turn masks the uneven contribution of RF channels to UEs' performance [8].

In order to enhance ESs, recent works propose to use Machine Learning (ML) for the RF channels switch on/off problem. The authors of [9] convert the RF channels switching into the selection of a suitable antenna array configuration from a predefined set of configurations. The EE is then optimized on a per slot basis by selecting the optimal configuration thanks to a neural network. Although this approach achieves significant ESs, it requires the collection of a large amount of data, specific to a given scenario, i.e., UEs' average channel vectors, and a long training period. To circumvent the data collection requirement, [10] proposes a Multi-Armed-Bandit (MAB) algorithm which sequentially selects the number of RF channels that maximizes EE from a set of high cardinality. The MAB is trained thanks to an intelligent database of geo-located data called a Radio Environment Map (REM). This approach assumes the ability to frequently switch on/off RF channels, namely every 60 ms, which may not be feasible due to the necessary adaptation of control signals such as, e.g., the synchronization signal blocks. It further assumes stationary conditions in the sense that for similar UEs' locations, a similar number of active RF channels is expected to maximize EE. Such assumption may not hold in networks supporting

<footnote>*Orange Labs, Chatillon, France; †University of Avignon, Avignon, France.
979-8-3503-6224-4/24/$31.00 ©2024 IEEE</footnote>
# On the Potential of Dynamic RF Channel Configuration for Energy Efficiency Optimization

Antoine Dejonghe*†, Safaa Driouech*, Zwi Altman*, and Francesco de Pellegrini†

***Abstract***—The deployment of large antenna arrays in Massive Multiple Input Multiple Output (M-MIMO) systems substantially improves mobile networks' performance. Nevertheless, the increase in performance brought by additional antennas is often counterbalanced by an increase in Power Consumption (PC) caused by the use of supplemental hardware resources supporting additional Radio Frequency (RF) channels. In M-MIMO networks, switching on/off RF channels and muting the associated antennas according to load conditions is known to be an efficient Energy Saving (ES) mechanism. This paper introduces a new RF channels switch on/off solution to maximize the Energy Efficiency (EE) under Quality of Service (QoS) constraints. The proposed solution is based on a MAB algorithm, which appears simpler than state of the art approaches and can be easily implemented in real systems. The algorithm leverages the quasiconcave shape of the EE metric to sequentially select the optimal antenna array configuration — i.e., the number of RF channels in both azimuth and elevation — from a predefined set of configurations. Extensive system-level simulations demonstrate that the proposed algorithm achieves a significant EE gain over a baseline solution.

***Index Terms***—Massive MIMO, RF channel configuration, energy efficiency, energy saving, Multi-Armed Bandit

## I. INTRODUCTION

5G and beyond Radio Access Network (RAN) use M-MIMO technology as a mean to enhance User Equipments (UEs)' performance, leveraging on large two dimensional antenna arrays. M-MIMO systems perform beamforming in order to focus the overall transmitted power towards UEs via a suitable weighting of the signal sent by each radiating antenna [1]. In digital M-MIMO architecture, for example, beamforming is enabled by feeding each antenna using a dedicated RF channel, namely an electronic circuit comprising different components such as digital-to-analog converters and power amplifiers. Beamforming capabilities and hence UEs' performance improve with the number of antennas, at the cost of increased Energy Consumption (EC) due to the additional RF channels.

The reduction of EC is a key challenge for the sustainability of future networks [2]. It can be performed by introducing ES mechanisms, including switching off different components of the network. The associated time scale for on/off switching can range from hours and above in case of the entire on/off switch of a cell, and scale down to the range of tens of microseconds to tens of milliseconds in the case of Advanced Sleep Modes. At intermediate time-scales, one can consider on/off switching

of carriers or RF channels. A detailed overview covering different aspects of EE in 5G RAN is presented in [3].

In M-MIMO systems, under suitable traffic load conditions, significant EC savings are obtained by switching on/off certain RF channels and muting the radiating antennas they feed. This ES mechanism is addressed in various publications as an optimization problem: it consists in finding the number of RF channels that maximizes EE, namely the ratio between the transmitted data volume and the corresponding EC [4]. It is noted that in operational systems, one may need to solve a constrained EE optimization problem taking into account QoS constraints. Early contributions [5]–[7] address the EE optimization problem using EE analytic approximations in conjunction with heuristics. For example, [5] presents an alternating optimization algorithm enabling to sequentially optimize the transmit power and the numbers of active RF channels and scheduled UEs. The authors prove that the EE analytic approximation is a quasiconcave function of the optimization parameters. Hence, this guarantees the convergence of the algorithm's iterations to the optimal solution. Approaches based on analytic approximations usually assume a simplified channel model, namely Rayleigh fading, which neglects certain propagation characteristics such as Angular Spread (AS), and in turn masks the uneven contribution of RF channels to UEs' performance [8].

In order to enhance ESs, recent works propose to use Machine Learning (ML) for the RF channels switch on/off problem. The authors of [9] convert the RF channels switching into the selection of a suitable antenna array configuration from a predefined set of configurations. The EE is then optimized on a per slot basis by selecting the optimal configuration thanks to a neural network. Although this approach achieves significant ESs, it requires the collection of a large amount of data, specific to a given scenario, i.e., UEs' average channel vectors, and a long training period. To circumvent the data collection requirement, [10] proposes a Multi-Armed-Bandit (MAB) algorithm which sequentially selects the number of RF channels that maximizes EE from a set of high cardinality. The MAB is trained thanks to an intelligent database of geo-located data called a Radio Environment Map (REM). This approach assumes the ability to frequently switch on/off RF channels, namely every 60 ms, which may not be feasible due to the necessary adaptation of control signals such as, e.g., the synchronization signal blocks. It further assumes stationary conditions in the sense that for similar UEs' locations, a similar number of active RF channels is expected to maximize EE. Such assumption may not hold in networks supporting

<footnote>*Orange Labs, Chatillon, France; †University of Avignon, Avignon, France.
979-8-3503-6224-4/24/$31.00 ©2024 IEEE</footnote>
<figure><img src="image_2.png" /><figcaption>Fig. 1: Network layout, $S = 7$, $C = 21$.</figcaption></figure>

different types of services.

This paper introduces an efficient low-complexity solution to perform the on/off switching of RF-channels. This solution can be easily integrated in a network controller of a 5G-3GPP network. It independently maximizes the EE of each network cell under QoS constraints. To this end, by leveraging on a priori knowledge related to the propagation environment, we first design an antenna array configuration set with reduced size and order it leading to a quasi-concave EE utility function over this set. Then, a MAB algorithm adapted to non-stationary environments is employed to sequentially select the optimal configuration. It is further shown that the MAB algorithm can be modified so as to leverage the quasi-concavity property, and hence significantly limit the selection of suboptimal configurations and further enhance EE.

The paper is organized as follows. Section II presents the system model. Section III covers the PC and EE models. Section IV proposes two MAB-based algorithms to switch on/off RF channels. Simulation results and conclusions are presented in sections V and VI.

## II. SYSTEM MODEL

Consider a 5G-Downlink (DL)-Orthogonal Frequency Division Multiplexing (OFDM)-Time Division Duplexing (TDD) network, with $S$ tri-sectorial macro-sites (i.e., $C = 3 \times S$ cells) in an urban environment. We define the set of cell indexes as $C = \{1, ..., C\}$. Each cell $c \in C$ serves a varying number of single-antenna UEs denoted by $U^c$. We denote by $B$ the total available bandwidth that can be used by a cell to serve UEs. This bandwidth is composed of $K = \frac{B}{\mu}$ OFDM subcarriers where $\mu$ is the Subcarrier Spacing (SCS).

**Massive MIMO Model.** Each cell is equipped with a Uniform Planar Array (UPA) composed of $M = M_h \times M_v$ single-polarized antennas, where $M_h$ and $M_v$ are the numbers of columns and rows of antennas, respectively. We consider the digital M-MIMO architecture, namely each RF channel feeds one antenna. For cell $c$, we denote by $N^c = N_h^c \times N_v^c \le M$ the number of active RF channels, where $N_h^c \le M_h$ and $N_v^c \le M_v$ are the numbers of active columns and rows of RF channels. In order the M-MIMO scheme to operate efficiently, we consider that $N_h^c \ge N_h^{min}$ and $N_v^c \ge N_v^{min}$, where $N_h^{min}$ and $N_v^{min}$ are the minimum numbers of active columns and

rows of RF channels. The couple ($N_h^c, N_v^c$) defines the active antenna array configuration of cell $c$. A network controller can periodically adapt such configuration by switching RF channels on or off. As further explained in the sequel, we assume here that ($N_h^c, N_v^c$) is selected from a predefined set of configurations denoted by $N_{conf}$.

**Channel Model.** In M-MIMO networks, channels are characterized among others by the dispersion of the multi-paths' azimuth and elevation angles (i.e. ASs). Such dispersion depends on the wireless environment's characteristics (e.g. scatterers' distribution). It is noted that the capability of M-MIMO to spatially separate UEs increases with the increase of azimuth and elevation ASs. For example, in a typical European city, the AS is much higher in the azimuthal than in the elevation direction. As a result, increasing the number of active antennas in the azimuthal direction permits to enhance UEs' performance more efficiently. Based on the above discussion, we write the Signal to Interference plus Noise Ratio (SINR) $\gamma_u^c$ and the rate $r_u^c$ of UE $u \in U^c = \{1, ..., U^c\}$ as functions of both $N_h^c$ and $N_v^c$: $r_u^c(N_h^c, N_v^c) = B \log_2(1 + \gamma_u^c(N_h^c, N_v^c))$. In this work, channels are modelled following the 3GPP Urban Macro (UMa) scenario [11]. This scenario considers high azimuth and low elevation ASs. Moreover, channels are assumed time-varying and frequency-dependent and remain constant in a coherence block of $T_{coh} \times B_{coh}$ symbols where $T_{coh}$ and $B_{coh}$ respectively denote the coherence time and bandwidth.

**Serving Model.** On each time slot, each cell $c$ schedules $U_{sch}^c \le U_{sch}^{max}$ UEs based on a standard Proportional Fair (PF) scheduler [12] adapted to multi-user scheduling. $U_{sch}^{max}$ denotes the maximum number of simultaneously scheduled UEs. Scheduled UEs are allocated a single data stream and communicate on all $K$ subcarriers. They are served thanks to the Zero Forcing (ZF) beamforming technique [1]. In order the ZF beamformer to operate efficiently, we consider here that $N_h^{min} = N_v^{min} = U_{sch}^{max}$. On each time slot, each scheduled UE is allocated a transmit power of $\frac{P_{tx}}{U_{sch}^{max}}$ which is equally distributed among the $K$ subcarriers.

**Traffic Model.** Consider elastic traffic of an enhanced Mobile Broadband (eMBB) service. UEs arrive according to a Poisson process, download a file of size $F = 150$ MB and leave the system upon download completion. A minimum Guaranteed Bit Rate (GBR) of $r_u^{gbr}$ for each UE $u \in U^c$ is assumed. In this work, the average rate of each UE $u \in U^c$, $\bar{r}_u^c$, is limited to its GBR via a proper resource allocation. It permits to favor the achievement by each UE of its GBR even under high load conditions. It is noted that the system model discussed in the rest of the paper does not depend on a specific traffic model.

## III. POWER CONSUMPTION AND ENERGY EFFICIENCY MODELS

We first detail a cell PC model adapted to M-MIMO networks. We then present the construction of the antenna array configuration set $N_{conf}$ used for the EE optimization.
<figure><img src="image_2.png" /><figcaption>Fig. 1: Network layout, $S = 7$, $C = 21$.</figcaption></figure>

different types of services.

This paper introduces an efficient low-complexity solution to perform the on/off switching of RF-channels. This solution can be easily integrated in a network controller of a 5G-3GPP network. It independently maximizes the EE of each network cell under QoS constraints. To this end, by leveraging on a priori knowledge related to the propagation environment, we first design an antenna array configuration set with reduced size and order it leading to a quasi-concave EE utility function over this set. Then, a MAB algorithm adapted to non-stationary environments is employed to sequentially select the optimal configuration. It is further shown that the MAB algorithm can be modified so as to leverage the quasi-concavity property, and hence significantly limit the selection of suboptimal configurations and further enhance EE.

The paper is organized as follows. Section II presents the system model. Section III covers the PC and EE models. Section IV proposes two MAB-based algorithms to switch on/off RF channels. Simulation results and conclusions are presented in sections V and VI.

## II. SYSTEM MODEL

Consider a 5G-Downlink (DL)-Orthogonal Frequency Division Multiplexing (OFDM)-Time Division Duplexing (TDD) network, with $S$ tri-sectorial macro-sites (i.e., $C = 3 \times S$ cells) in an urban environment. We define the set of cell indexes as $C = \{1, ..., C\}$. Each cell $c \in C$ serves a varying number of single-antenna UEs denoted by $U^c$. We denote by $B$ the total available bandwidth that can be used by a cell to serve UEs. This bandwidth is composed of $K = \frac{B}{\mu}$ OFDM subcarriers where $\mu$ is the Subcarrier Spacing (SCS).

**Massive MIMO Model.** Each cell is equipped with a Uniform Planar Array (UPA) composed of $M = M_h \times M_v$ single-polarized antennas, where $M_h$ and $M_v$ are the numbers of columns and rows of antennas, respectively. We consider the digital M-MIMO architecture, namely each RF channel feeds one antenna. For cell $c$, we denote by $N^c = N_h^c \times N_v^c \le M$ the number of active RF channels, where $N_h^c \le M_h$ and $N_v^c \le M_v$ are the numbers of active columns and rows of RF channels. In order the M-MIMO scheme to operate efficiently, we consider that $N_h^c \ge N_h^{min}$ and $N_v^c \ge N_v^{min}$, where $N_h^{min}$ and $N_v^{min}$ are the minimum numbers of active columns and

rows of RF channels. The couple $(N_h^c, N_v^c)$ defines the active antenna array configuration of cell $c$. A network controller can periodically adapt such configuration by switching RF channels on or off. As further explained in the sequel, we assume here that $(N_h^c, N_v^c)$ is selected from a predefined set of configurations denoted by $N_{conf}$.

**Channel Model.** In M-MIMO networks, channels are characterized among others by the dispersion of the multi-paths' azimuth and elevation angles (i.e. ASs). Such dispersion depends on the wireless environment's characteristics (e.g. scatterers' distribution). It is noted that the capability of M-MIMO to spatially separate UEs increases with the increase of azimuth and elevation ASs. For example, in a typical European city, the AS is much higher in the azimuthal than in the elevation direction. As a result, increasing the number of active antennas in the azimuthal direction permits to enhance UEs' performance more efficiently. Based on the above discussion, we write the Signal to Interference plus Noise Ratio (SINR) $\gamma_u^c$ and the rate $r_u^c$ of UE $u \in U^c = \{1, ..., U^c\}$ as functions of both $N_h^c$ and $N_v^c$: $r_u^c(N_h^c, N_v^c) = B \log_2(1 + \gamma_u^c(N_h^c, N_v^c))$. In this work, channels are modelled following the 3GPP Urban Macro (UMa) scenario [11]. This scenario considers high azimuth and low elevation ASs. Moreover, channels are assumed time-varying and frequency-dependent and remain constant in a coherence block of $T_{coh} \times B_{coh}$ symbols where $T_{coh}$ and $B_{coh}$ respectively denote the coherence time and bandwidth.

**Serving Model.** On each time slot, each cell $c$ schedules $U_{sch}^c \le U_{sch}^{max}$ UEs based on a standard Proportional Fair (PF) scheduler [12] adapted to multi-user scheduling. $U_{sch}^{max}$ denotes the maximum number of simultaneously scheduled UEs. Scheduled UEs are allocated a single data stream and communicate on all $K$ subcarriers. They are served thanks to the Zero Forcing (ZF) beamforming technique [1]. In order the ZF beamformer to operate efficiently, we consider here that $N_h^{min} = N_v^{min} = U_{sch}^{max}$. On each time slot, each scheduled UE is allocated a transmit power of $\frac{P_{tx}}{U_{sch}^{max}}$ which is equally distributed among the $K$ subcarriers.

**Traffic Model.** Consider elastic traffic of an enhanced Mobile Broadband (eMBB) service. UEs arrive according to a Poisson process, download a file of size $F = 150$ MB and leave the system upon download completion. A minimum Guaranteed Bit Rate (GBR) of $r_u^{gbr}$ for each UE $u \in U^c$ is assumed. In this work, the average rate of each UE $u \in U^c$, $\bar{r}_u^c$, is limited to its GBR via a proper resource allocation. It permits to favor the achievement by each UE of its GBR even under high load conditions. It is noted that the system model discussed in the rest of the paper does not depend on a specific traffic model.

## III. POWER CONSUMPTION AND ENERGY EFFICIENCY MODELS

We first detail a cell PC model adapted to M-MIMO networks. We then present the construction of the antenna array configuration set $N_{conf}$ used for the EE optimization.
A. Power consumption model

Following the approach proposed in [10], we simplify a model originally developed in [5] by only considering the components which significantly contribute to the cell PC. The overall PC of cell c can be expressed as a function of both $N_h^c$ and $N_v^c$ as follows

$$P_{tot}^{c}(N_h^{c}, N_v^{c}) = \frac{P_{tx}^{c}(N_h^{c}, N_v^{c})}{\eta} + P_{rf}^{c}(N_h^{c}, N_v^{c}) + P_{fix}. \quad (1)$$

In (1), $\frac{P_{tx}^{c}(N_h^{c}, N_v^{c})}{\eta}$ corresponds to the total power fed to the antennas in order to effectively transmit $P_{tx}^{c}(N_h^{c}, N_v^{c})$, taking into account the power amplifiers' efficiency $\eta$. Specifically, we consider that $P_{tx}^{c}(N_h^{c}, N_v^{c}) = \sum_{u=1}^{U^c} P_{tx}^{c,u}(N_h^{c}, N_v^{c})$ where $P_{tx}^{c,u}(N_h^{c}, N_v^{c})$ is the transmit power required by the UE $u \in U^c$ to achieve its GBR when $N_h^c$ columns and $N_v^c$ rows of RF channels are active. The dependency of $P_{tx}^{c,u}(\cdot)$ on both $N_h^c$ and $N_v^c$ is related to the dependency of $r_u^c(\cdot)$ on these two parameters. Indeed, $r_u^c(\cdot)$ impacts the necessary number of scheduling occasions that permit the UE $u$ to achieve its GBR. This in turn impacts the power consumed in transmission. In particular, $P_{tx}^{c,u}(\cdot)$ and $r_u^c(\cdot)$ vary inversely. $P_{rf}^{c}(N_h^{c}, N_v^{c}) = N_h^c \times N_v^c \times \hat{P}_{rf} + P_{lo}$ is the power consumed by all active RF channels where $\hat{P}_{rf}$ and $P_{lo}$ respectively denote the PC of a single RF channel and of the local oscillator. Finally, $P_{fix}$ denotes a constant PC accounting for site cooling and signaling.

B. Energy efficiency model

It is standard practice to compute the cell EE as the ratio of the cell average rate and the cell PC:

$$EE^c(N_h^c, N_v^c) = \frac{\sum_{u=1}^{U^c} \bar{r}_u^c(N_h^c, N_v^c)}{P_{tot}^c(N_h^c, N_v^c)} \quad (\text{bit/J}). \quad (2)$$

As mentioned previously, $(N_h^c, N_v^c)$ is assumed to take its value in $N_{conf}$. Using the properties of the AS described in Section II, one can construct a set $N_{conf}$ with reduced size. Indeed, $N_{conf}$ can be restricted to configurations with higher or equal number of RF channels in the highest AS direction with respect to the lowest one since they provide higher EE. Furthermore, the AS properties can also be used to order the reduced set $N_{conf}$ so as to render the EE utility function $EE^c: N_{conf} \mapsto \mathbb{R}$ quasiconcave. Interestingly, both the reduced size of $N_{conf}$ and the quasiconcavity property will be exploited by the EE optimization algorithm in order to effectively determine the most energy-efficient configuration in $N_{conf}$. The construction of $N_{conf}$ can be described as a three-step procedure:

Step 1. $N_{conf}$ is initialized with the configuration $(N_h^{min}, N_v^{min})$.

Step 2. The next configurations are obtained by successively increasing the number of lines (i.e. rows or columns) of active RF channels in the highest AS direction in steps of $\delta_N \in \mathbb{N}$ until its maximum value is achieved (while keeping the number of lines of active RF channels in the lowest AS direction to its minimum).

Step 3. $N_{conf}$ is completed by successively increasing the number of lines of RF channels in the lowest AS direction in steps of $\delta_N \in \mathbb{N}$ until its maximum value is achieved (while keeping the number of lines of active RF channels in the highest AS direction to its maximum).

The mathematical representation of steps 2 and 3 can be written as follows. Denote by $a \in \{h, v\}$ and $\bar{a} \in \{h, v\} \setminus \{a\}$ the directions of the highest and lowest ASs, respectively. Step 2 consists in successively increasing $N_a^c$ in steps of $\delta_N \in \mathbb{N}$ until $M_a$ while considering $N_a^c = N_a^{min}$. Analogously, step 3 consists in successively increasing $N_a^c$ in steps of $\delta_N \in \mathbb{N}$ until $M_a$ while considering $N_a^c = M_a$. Consider a higher AS in the azimuthal direction (i.e. $a = h$). The resulting configuration set can be written as in (3). A numerical example of such a set is provided in Fig. 2 for $N_h^{min} = N_v^{min} = 4$, $M_h = M_v = 8$ and $\delta_N = 2$.

$$N_{conf} = \left\{ (N_h^{min}, N_v^{min}), (N_h^{min} + \delta_N, N_v^{min}), \dots, (M_h, N_v^{min}), (M_h, N_v^{min} + \delta_N), \dots, (M_h, M_v) \right\}. \quad (3)$$

Fig. 2 shows the typical quasiconcave shape of $EE^c(\cdot)$ under realistic propagation conditions (i.e. 3GPP UMa scenario). We explain here this quasiconcavity property in the context of Rayleigh fading. The case of realistic propagation conditions is discussed next.

It is noted that for all configurations in $N_{conf}$, the cell average rate has small oscillations around $\sum_{u=1}^{U^c} r_u^{gb}$. Therefore, it is sufficient to show that $P_{tx}^{c}(\cdot) + P_{rf}^{c}(\cdot)$ is a discrete convex function to explain quasiconcavity. By construction of $N_{conf}$, $P_{rf}^{c}(\cdot)$ is defined on two joint subsets of $N_{conf}$ by linear functions of slopes $\delta_N N_a^{min} \hat{P}_{rf}$ and $\delta_N M_a \hat{P}_{rf}$, respectively. Since we consider $M_a \ge N_a^{min}$, $P_{rf}^{c}(\cdot)$ results in a discrete convex function. When considering Rayleigh fading and $N^c \ge U_{sch}^c + 1$, the SINR $\gamma_u^c(\cdot)$ of each UE $u$ is a strictly increasing discrete linear function of $N^c$ [5]. Hence, the rate $r_u^c(\cdot)$ of each UE $u$ is a discrete concave function of $N^c$. As mentioned in subsection III-A, $r_u^c(\cdot)$ and $P_{tx}^{c,u}(\cdot)$ have inverse variations so that $P_{tx}^{c,u}(\cdot)$ corresponds to a discrete convex function. Therefore, $P_{tx}^{c}(\cdot)$ and hence $P_{tx}^{c}(\cdot) + P_{rf}^{c}(\cdot)$ results in a discrete convex function.

For realistic propagation conditions, we observe from extensive simulations that $r_u^c(\cdot)$ and $EE^c(\cdot)$ are respectively discrete concave and discrete quasiconcave. Formally proving these shapes remains a challenging open problem. However, in the EE optimization solution, we incorporate an explicit quasiconcavity test before using this property.

It is noted that the average number of simultaneously scheduled UEs varies jointly with the network load and impacts both the number of scheduling occasions and the transmission power used for each UE. Therefore, a change in network load entails a change in the shape of both $P_{tx}^{c}(\cdot)$ and $EE^c(\cdot)$. In particular, continuous variations of the network load result in a continuous shift of the EE optimum from a configuration to an adjacent one in $N_{conf}$ as shown in Fig. 2. This property will be exploited by the EE optimization algorithm.
### A. Power consumption model

Following the approach proposed in [10], we simplify a model originally developed in [5] by only considering the components which significantly contribute to the cell PC. The overall PC of cell c can be expressed as a function of both $N_h^c$ and $N_v^c$ as follows

$$P_{tot}^{c}(N_h^{c}, N_v^{c}) = \frac{P_{tx}^{c}(N_h^{c}, N_v^{c})}{\eta} + P_{rf}^{c}(N_h^{c}, N_v^{c}) + P_{fix}. \quad (1)$$

In (1), $\frac{P_{tx}^{c}(N_h^{c}, N_v^{c})}{\eta}$ corresponds to the total power fed to the antennas in order to effectively transmit $P_{tx}^{c}(N_h^{c}, N_v^{c})$, taking into account the power amplifiers' efficiency $\eta$. Specifically, we consider that $P_{tx}^{c}(N_h^{c}, N_v^{c}) = \sum_{u=1}^{U^c} P_{tx}^{c,u}(N_h^{c}, N_v^{c})$ where $P_{tx}^{c,u}(N_h^{c}, N_v^{c})$ is the transmit power required by the UE $u \in U^c$ to achieve its GBR when $N_h^c$ columns and $N_v^c$ rows of RF channels are active. The dependency of $P_{tx}^{c,u}(\cdot)$ on both $N_h^c$ and $N_v^c$ is related to the dependency of $r_u^c(\cdot)$ on these two parameters. Indeed, $r_u^c(\cdot)$ impacts the necessary number of scheduling occasions that permit the UE $u$ to achieve its GBR. This in turn impacts the power consumed in transmission. In particular, $P_{tx}^{c,u}(\cdot)$ and $r_u^c(\cdot)$ vary inversely. $P_{rf}^{c}(N_h^{c}, N_v^{c}) = N_h^c \times N_v^c \times \hat{P}_{rf} + P_{lo}$ is the power consumed by all active RF channels where $\hat{P}_{rf}$ and $P_{lo}$ respectively denote the PC of a single RF channel and of the local oscillator. Finally, $P_{fix}$ denotes a constant PC accounting for site cooling and signaling.

### B. Energy efficiency model

It is standard practice to compute the cell EE as the ratio of the cell average rate and the cell PC:

$$EE^c(N_h^c, N_v^c) = \frac{\sum_{u=1}^{U^c} \bar{r}_u^c(N_h^c, N_v^c)}{P_{tot}^{c}(N_h^c, N_v^c)} \quad (\text{bit/J}). \quad (2)$$

As mentioned previously, $(N_h^c, N_v^c)$ is assumed to take its value in $N_{conf}$. Using the properties of the AS described in Section II, one can construct a set $N_{conf}$ with reduced size. Indeed, $N_{conf}$ can be restricted to configurations with higher or equal number of RF channels in the highest AS direction with respect to the lowest one since they provide higher EE. Furthermore, the AS properties can also be used to order the reduced set $N_{conf}$ so as to render the EE utility function $EE^c: N_{conf} \mapsto \mathbb{R}$ quasiconcave. Interestingly, both the reduced size of $N_{conf}$ and the quasiconcavity property will be exploited by the EE optimization algorithm in order to effectively determine the most energy-efficient configuration in $N_{conf}$. The construction of $N_{conf}$ can be described as a three-step procedure:

Step 1. $N_{conf}$ is initialized with the configuration $(N_h^{min}, N_v^{min})$.

Step 2. The next configurations are obtained by successively increasing the number of lines (i.e. rows or columns) of active RF channels in the highest AS direction in steps of $\delta_N \in \mathbb{N}$ until its maximum value is achieved (while keeping the number of lines of active RF channels in the lowest AS direction to its minimum).

Step 3. $N_{conf}$ is completed by successively increasing the number of lines of RF channels in the lowest AS direction in steps of $\delta_N \in \mathbb{N}$ until its maximum value is achieved (while keeping the number of lines of active RF channels in the highest AS direction to its maximum).

The mathematical representation of steps 2 and 3 can be written as follows. Denote by $a \in \{h, v\}$ and $\bar{a} \in \{h, v\} \setminus \{a\}$ the directions of the highest and lowest ASs, respectively. Step 2 consists in successively increasing $N_a^c$ in steps of $\delta_N \in \mathbb{N}$ until $M_a$ while considering $N_a^c = N_a^{min}$. Analogously, step 3 consists in successively increasing $N_a^c$ in steps of $\delta_N \in \mathbb{N}$ until $M_a$ while considering $N_a^c = M_a$. Consider a higher AS in the azimuthal direction (i.e. $a = h$). The resulting configuration set can be written as in (3). A numerical example of such a set is provided in Fig. 2 for $N_h^{min} = N_v^{min} = 4$, $M_h = M_v = 8$ and $\delta_N = 2$.

$$N_{conf} = \left\{ (N_h^{min}, N_v^{min}), (N_h^{min} + \delta_N, N_v^{min}), \dots, (M_h, N_v^{min}), (M_h, N_v^{min} + \delta_N), \dots, (M_h, M_v) \right\}. \quad (3)$$

Fig. 2 shows the typical quasiconcave shape of $EE^c(\cdot)$ under realistic propagation conditions (i.e. 3GPP UMa scenario). We explain here this quasiconcavity property in the context of Rayleigh fading. The case of realistic propagation conditions is discussed next.

It is noted that for all configurations in $N_{conf}$, the cell average rate has small oscillations around $\sum_{u=1}^{U^c} r_u^{gb}$. Therefore, it is sufficient to show that $P_{tx}^{c}(\cdot) + P_{rf}^{c}(\cdot)$ is a discrete convex function to explain quasiconcavity. By construction of $N_{conf}$, $P_{rf}^{c}(\cdot)$ is defined on two joint subsets of $N_{conf}$ by linear functions of slopes $\delta_N N_a^{min} \hat{P}_{rf}$ and $\delta_N M_a \hat{P}_{rf}$, respectively. Since we consider $M_a \ge N_a^{min}$, $P_{rf}^{c}(\cdot)$ results in a discrete convex function. When considering Rayleigh fading and $N^c \ge U_{sch}^c + 1$, the SINR $\gamma_u^c(\cdot)$ of each UE $u$ is a strictly increasing discrete linear function of $N^c$ [5]. Hence, the rate $r_u^c(\cdot)$ of each UE $u$ is a discrete concave function of $N^c$. As mentioned in subsection III-A, $r_u^c(\cdot)$ and $P_{tx}^{c,u}(\cdot)$ have inverse variations so that $P_{tx}^{c,u}(\cdot)$ corresponds to a discrete convex function. Therefore, $P_{tx}^{c}(\cdot)$ and hence $P_{tx}^{c}(\cdot) + P_{rf}^{c}(\cdot)$ results in a discrete convex function.

For realistic propagation conditions, we observe from extensive simulations that $r_u^c(\cdot)$ and $EE^c(\cdot)$ are respectively discrete concave and discrete quasiconcave. Formally proving these shapes remains a challenging open problem. However, in the EE optimization solution, we incorporate an explicit quasiconcavity test before using this property.

It is noted that the average number of simultaneously scheduled UEs varies jointly with the network load and impacts both the number of scheduling occasions and the transmission power used for each UE. Therefore, a change in network load entails a change in the shape of both $P_{tx}^{c}(\cdot)$ and $EE^c(\cdot)$. In particular, continuous variations of the network load result in a continuous shift of the EE optimum from a configuration to an adjacent one in $N_{conf}$ as shown in Fig. 2. This property will be exploited by the EE optimization algorithm.
<figure><img src="image_3.png" /><figcaption>Fig. 2: Average Energy Efficiency of the configurations in $\mathcal{N}_{conf}$ for different network load levels, ($\delta_N = 2$, $N_h^{min} = N_v^{min} = 4$, $\mathcal{N}_{conf} = \{(4, 4), (6, 4), (8, 4), (8, 6), (8, 8)\}$).</figcaption></figure>

## IV. MULTI-ARMED BANDIT-BASED ENERGY EFFICIENCY MAXIMIZATION

The proposed ES feature aims at independently maximizing the EE of each cell $c \in \mathcal{C}$ by switching RF channels on or off. As in [9], we convert the RF channels switch on/off problem into the selection of the configuration from $\mathcal{N}_{conf}$ that maximizes the cell EE under some QoS constraints (i.e. the achievement by each UE $u \in \mathcal{U}^c$ of its GBR). For cell $c$, the optimization problem is formulated as follows

$$ \max_{N_h^c, N_v^c} EE^c(N_h^c, N_v^c) \quad (4a) $$

$$ \text{subject to} \quad (N_h^c, N_v^c) \in \mathcal{N}_{conf} \quad (4b) $$

$$ \bar{r}_u^c(N_h^c, N_v^c) \geq r_u^{gbr}, u \in \mathcal{U}^c. \quad (4c) $$

In the proposed solution, a network controller independently and periodically chooses for each cell $c$ the configuration from $\mathcal{N}_{conf}$ to activate during a predefined period $T_{conf}$. The EE optimization is performed using a MAB-based scheme. Specifically, for each cell $c$ and at each time $t$ (i.e. every $T_{conf}$), the MAB selects a configuration from a set $\mathcal{N}_{conf}^{c,t} \subset \mathcal{N}_{conf}$ which gathers all the configurations from $\mathcal{N}_{conf}$ that up to now satisfied the constraint (4c). This selection is done based on the following decision rule

$$ i_c(t) = \arg \max_{j=1,...,|\mathcal{N}_{conf}^{c,t}|} I_{j,t}^c \quad (5) $$

where $I_{j,t}^c$ is a utility related to the average EE achieved by cell $c$ at time $t$ under the $j$-th configuration from $\mathcal{N}_{conf}^{c,t}$. During the $T_{conf}$ period following each configuration selection, each cell $c$ estimates the UEs' average rates and the cell EE provided by the active configuration. UEs' average rates are used to

update $\mathcal{N}_{conf}^{c,t}$, namely the tested configuration which does not satisfy the constraint (4c) is removed from $\mathcal{N}_{conf}^{c,t}$ for a duration $T_{bared}$. It is noted that the EE metric under the new set $\mathcal{N}_{conf}^{c,t}$ is still quasiconcave. The estimated EE is fed to the MAB as a reward so as to update the utility $I_{ic(t),t}^c$.

The MAB is a Reinforcement Learning (RL) framework which generally benefits from low complexity and fast convergence, at the price of continuous learning and can be adapted to non-stationary environments. In subsections IV-A and IV-B, we detail two MAB algorithms. The first one is the standard Sliding-Window-Upper Confidence Bound 1 (SW-UCB1) algorithm. The second corresponds to a modified version of SW-UCB1 which leverages the quasiconcave shape of the EE metric so as to accelerate the learning process.

### A. Sliding-Window-Upper Confidence Bound 1

Upper Confidence Bound 1 (UCB1) is a MAB algorithm based on the “optimism in the face of uncertainty” principle [13]. Such principle encourages the MAB to assign to each index $I_{j,t}^c, j \in \{1, ..., |\mathcal{N}_{conf}^{c,t}|\}, c \in \mathcal{C}$ the most optimistic value it could possibly get given the history (i.e. previous EEs achieved under the corresponding configuration). In compliance with this principle, UCB1 computes each index as follows

$$ I_{j,t}^c = \hat{\mu}_{j,t}^c + \sqrt{\frac{\alpha(b-a)^2 \log(t)}{Q_{j,t}^c}} \quad (6) $$

where $\hat{\mu}_{j,t}^c$ is the average of EEs achieved up to time $t$ under the $j$-th configuration from $\mathcal{N}_{conf}^{c,t}$. The second term in the right hand side of (6) corresponds to an Upper Confidence Bound (UCB) where $\alpha$ denotes the exploration rate, $[a, b]$ is the support for EE and $Q_{j,t}^c$ corresponds to the number of times the $j$-th configuration was selected up to time $t$.

It is noted that, under stationary conditions, the UCB estimate shrinks as $Q_{j,t}^c$ increases. In fact, the uncertainty on the sampling average EE achieved under the $j$-th configuration decreases (i.e. $\hat{\mu}_{j,t}^c$ approaches its true value $\bar{\mu}_j^c$). As a result, the index in (6) permits the MAB to test each configuration the minimum number of times so as to identify the optimal configuration with a high confidence.

The UCB1 algorithm performs well under the following assumption: successive selections of the $j$-th configuration for cell $c$ yield EEs $EE_{j,t}^c, l \in \mathbb{N}$ that are independently and identically distributed according to an unknown law with unknown expectation $\bar{\mu}_j^c$ (i.e. stationary distribution). In comparison, it usually fails to track the optimal configuration when considering non-stationary EEs' distributions. Such distributions are often encountered in real networks due to load variations. To deal with non-stationary distributions, [14] proposed the SW-UCB1 algorithm. It encourages a periodic retraining of the MAB by computing the index (6) over a fixed-size time horizon $\tau$. Specifically, in (6), $\log(t)$ is replaced by $\log(\min(t, \tau))$, and $\hat{\mu}_{j,t}^c$ and $Q_{j,t}^c$ are computed based on the last $\tau$ decisions as follows

$$ \hat{\mu}_{j,t}^c = \frac{1}{Q_{j,t}^c} \sum_{s=t-\tau+1}^{t} 1_{j,s} EE_{j,s}^c ; Q_{j,t}^c = \sum_{s=t-\tau+1}^{t} 1_{j,s}^c \quad (7) $$
<figure><img src="image_3.png" /><figcaption>Fig. 2: Average Energy Efficiency of the configurations in $\mathcal{N}_{conf}$ for different network load levels, ($\delta_N = 2$, $N_h^{min} = N_v^{min} = 4$, $\mathcal{N}_{conf} = \{(4, 4), (6, 4), (8, 4), (8, 6), (8, 8)\}$).</figcaption></figure>

## IV. MULTI-ARMED BANDIT-BASED ENERGY EFFICIENCY MAXIMIZATION

The proposed ES feature aims at independently maximizing the EE of each cell $c \in \mathcal{C}$ by switching RF channels on or off. As in [9], we convert the RF channels switch on/off problem into the selection of the configuration from $\mathcal{N}_{conf}$ that maximizes the cell EE under some QoS constraints (i.e. the achievement by each UE $u \in \mathcal{U}^c$ of its GBR). For cell $c$, the optimization problem is formulated as follows

$$ \max_{N_h^c, N_v^c} EE^c(N_h^c, N_v^c) \quad (4a) $$

$$ \text{subject to} \quad (N_h^c, N_v^c) \in \mathcal{N}_{conf} \quad (4b) $$

$$ \bar{r}_u^c(N_h^c, N_v^c) \geq r_u^{gbr}, u \in \mathcal{U}^c. \quad (4c) $$

In the proposed solution, a network controller independently and periodically chooses for each cell $c$ the configuration from $\mathcal{N}_{conf}$ to activate during a predefined period $T_{conf}$. The EE optimization is performed using a MAB-based scheme. Specifically, for each cell $c$ and at each time $t$ (i.e. every $T_{conf}$), the MAB selects a configuration from a set $\mathcal{N}_{conf}^{c,t} \subset \mathcal{N}_{conf}$ which gathers all the configurations from $\mathcal{N}_{conf}$ that up to now satisfied the constraint (4c). This selection is done based on the following decision rule

$$ i_c(t) = \arg \max_{j=1,...,|\mathcal{N}_{conf}^{c,t}|} I_{j,t}^c \quad (5) $$

where $I_{j,t}^c$ is a utility related to the average EE achieved by cell $c$ at time $t$ under the $j$-th configuration from $\mathcal{N}_{conf}^{c,t}$. During the $T_{conf}$ period following each configuration selection, each cell $c$ estimates the UEs' average rates and the cell EE provided by the active configuration. UEs' average rates are used to

update $\mathcal{N}_{conf}^{c,t}$, namely the tested configuration which does not satisfy the constraint (4c) is removed from $\mathcal{N}_{conf}^{c,t}$ for a duration $T_{bared}$. It is noted that the EE metric under the new set $\mathcal{N}_{conf}^{c,t}$ is still quasiconcave. The estimated EE is fed to the MAB as a reward so as to update the utility $I_{ic(t),t}^c$.

The MAB is a Reinforcement Learning (RL) framework which generally benefits from low complexity and fast convergence, at the price of continuous learning and can be adapted to non-stationary environments. In subsections IV-A and IV-B, we detail two MAB algorithms. The first one is the standard Sliding-Window-Upper Confidence Bound 1 (SW-UCB1) algorithm. The second corresponds to a modified version of SW-UCB1 which leverages the quasiconcave shape of the EE metric so as to accelerate the learning process.

### A. Sliding-Window-Upper Confidence Bound 1

Upper Confidence Bound 1 (UCB1) is a MAB algorithm based on the “optimism in the face of uncertainty” principle [13]. Such principle encourages the MAB to assign to each index $I_{j,t}^c, j \in \{1, ..., |\mathcal{N}_{conf}^{c,t}|\}, c \in \mathcal{C}$ the most optimistic value it could possibly get given the history (i.e. previous EEs achieved under the corresponding configuration). In compliance with this principle, UCB1 computes each index as follows

$$ I_{j,t}^c = \hat{\mu}_{j,t}^c + \sqrt{\frac{\alpha(b-a)^2 \log(t)}{Q_{j,t}^c}} \quad (6) $$

where $\hat{\mu}_{j,t}^c$ is the average of EEs achieved up to time $t$ under the $j$-th configuration from $\mathcal{N}_{conf}^{c,t}$. The second term in the right hand side of (6) corresponds to an Upper Confidence Bound (UCB) where $\alpha$ denotes the exploration rate, $[a, b]$ is the support for EE and $Q_{j,t}^c$ corresponds to the number of times the $j$-th configuration was selected up to time $t$.

It is noted that, under stationary conditions, the UCB estimate shrinks as $Q_{j,t}^c$ increases. In fact, the uncertainty on the sampling average EE achieved under the $j$-th configuration decreases (i.e. $\hat{\mu}_{j,t}^c$ approaches its true value $\bar{\mu}_j^c$). As a result, the index in (6) permits the MAB to test each configuration the minimum number of times so as to identify the optimal configuration with a high confidence.

The UCB1 algorithm performs well under the following assumption: successive selections of the $j$-th configuration for cell $c$ yield EEs $EE_{j,t}^c, l \in \mathbb{N}$ that are independently and identically distributed according to an unknown law with unknown expectation $\bar{\mu}_j^c$ (i.e. stationary distribution). In comparison, it usually fails to track the optimal configuration when considering non-stationary EEs' distributions. Such distributions are often encountered in real networks due to load variations. To deal with non-stationary distributions, [14] proposed the SW-UCB1 algorithm. It encourages a periodic retraining of the MAB by computing the index (6) over a fixed-size time horizon $\tau$. Specifically, in (6), $\log(t)$ is replaced by $\log(\min(t, \tau))$, and $\hat{\mu}_{j,t}^c$ and $Q_{j,t}^c$ are computed based on the last $\tau$ decisions as follows

$$ \hat{\mu}_{j,t}^c = \frac{1}{Q_{j,t}^c} \sum_{s=t-\tau+1}^{t} 1_{j,s} EE_{j,s}^c ; Q_{j,t}^c = \sum_{s=t-\tau+1}^{t} 1_{j,s}^c \quad (7) $$
where $\mathbb{1}_{j,s}^c$ is an indicator variable such that $\mathbb{1}_{j,s}^c = 1$ if the $j$-th configuration from $\mathcal{N}_{conf}^{c,t}$ is selected at time $s$ and 0 otherwise.

## B. Constrained Sliding-Window Upper Confidence Bound 1

To accelerate the MAB convergence, we use a modified version of SW-UCB1 [15] which leverages the quasiconcave shape of the EE metric. This version enables to limit the exploration of suboptimal configurations in order to further enhance ESs. Specifically, SW-UCB1 is restricted to the configurations in $\mathcal{N}_{conf}$ which are adjacent to the current optimal configuration. Given the quasiconcave shape of the EE metric, such restriction prevents selecting the less energy-efficient configurations. Furthermore, as explained in subsection III-B, continuous network load variations result in a continuous shift of the EE optimum from a configuration to an adjacent one in $\mathcal{N}_{conf}$. Therefore, this algorithm also permits to efficiently track the optimal configuration which always remains in the restricted version of $\mathcal{N}_{conf}$.

As explained in subsection III-B, proving the quasiconcavity property of the EE metric under realistic propagation conditions remains a challenging open problem. In a real system, this property can be periodically checked by running for some period $T_{check}$ the standard SW-UCB1 algorithm. If this property is validated for a cell, one can switch to the constrained algorithm.

# V. NUMERICAL RESULTS

The following results present the EE optimization achieved by the two MAB-based RF channels switch on/off solutions introduced in section IV. Simulations are carried out using a system simulator with time resolution of a slot (0.5 ms).

## A. Simulation scenario

We consider a hexagonal network composed of $S = 7$ tri-sectorial macro sites which are deployed with an inter-site distance of 300 meters (see Fig. 1). Each cell is equipped with a UPA of $M = M_h \times M_v$ antennas where $M_h = M_v = 8$. Furthermore, we consider $N_h^{min} = N_v^{min} = 4$. In the proposed scenario, the three cells of site 1 are used for performance assessment of the studied ES feature, i.e., they are the cells of interest. They have a controllable number of active RF channels (i.e. $N_h^{min} \le N_h^c \le M_h$, $N_v^{min} \le N_v^c \le M_v$, $c \in \{1, 2, 3\}$), while the rest of 18 cells have fixed M-MIMO setup (i.e. $N_h^c = N_h^{min}$, $N_v^c = N_v^{min}$, $\forall c \in C\{1, 2, 3\}$) and are used to generate realistic interference.

The central frequency and the bandwidth of the system are set to $f_c = 3.5$ GHz and $B = 100$ MHz. As the SCS is set to $\mu = 30$ kHz, we have $K = 3333$ subcarriers. The Base Station (BS) transmit power is equal to $P_{tx} = 100$ W. The maximum number of simultaneously scheduled UEs is set to $U_{sch}^{max} = 4$. We assume here the same GBR for all UEs, namely $\forall u \in \mathcal{U}^c$, $r_u^{gbr} = 25$ Mbps.

The PC of each cell is computed while considering $\eta = 0.5$, $\hat{P}_{rf} = 0.4$ W, $P_{lo} = 0.2$ W and $P_{fix} = 10$ W. We consider here $\delta_N = 2$ so that the set of available antenna

<figure><img src="image_4.png" /><figcaption>Fig. 3: Trajectory of the selection ratio produced by the constrained SW-UCB1 algorithm for each configuration from $\mathcal{N}_{conf}$ ($\chi_{j,t}^c, j \in \{1, ..., |\mathcal{N}_{conf}|\}$), network load: 10% from 0 to 400 seconds and 30% afterwards.</figcaption></figure>

array configurations (see subsection III-B) corresponds to $\mathcal{N}_{conf} = \{(4, 4), (6, 4), (8, 4), (8, 6), (8, 8)\}$. In the proposed ES feature, the MAB algorithm associated to each cell $c$ periodically selects the configuration from $\mathcal{N}_{conf}$ to apply during a predefined period $T_{conf} = 1$ second. Such period permits to perform significant ESs based on the computation of a robust EE metric, while leaving enough time for the adaptation of control signals. We respectively set the MAB learning rate and the sliding-window size to $\alpha = 0.05$ and $\tau = 175$. Such setting enables to track the optimal configuration, in a time interval of $\tau \times T_{conf} = 175$ seconds.

## B. Optimal configuration tracking

We consider a 800 seconds simulation with the following traffic profile. An average network load of 10% is observed during the first 400 seconds and then increases to 30% for the last 400 seconds. Let $\chi_{j,t}^c = \frac{Q_{j,t}^c}{\tau}$ denote the selection ratio of the $j$-th configuration from $\mathcal{N}_{conf}^{c,t}$. Fig. 3 shows the trajectory of the selection ratio produced by the constrained SW-UCB1 algorithm for each configuration from $\mathcal{N}_{conf}$ ($\chi_{j,t}^c, j \in \{1, ..., |\mathcal{N}_{conf}|\}$). These trajectories are correlated to the average network load. Indeed, for an average network load of 10% the algorithm selects more frequently the (4, 4) configuration which provides the highest average EE (see Fig. 2). Moreover, as the network load increases from 10% to 30%, the algorithm tends to choose more frequently the (6, 4) and (8, 4) configurations which offer similar levels of average EE (see Fig. 2). A delay between the network load increase and the selection ratios adaptation can be observed. This is due to the sliding-window size $\tau$ and the exploration rate $\alpha$ which influence the frequency at which all configurations are retested.
where $\mathbb{1}_{j,s}^c$ is an indicator variable such that $\mathbb{1}_{j,s}^c = 1$ if the $j$-th configuration from $\mathcal{N}_{conf}^{c,t}$ is selected at time $s$ and 0 otherwise.

## B. Constrained Sliding-Window Upper Confidence Bound 1

To accelerate the MAB convergence, we use a modified version of SW-UCB1 [15] which leverages the quasiconcave shape of the EE metric. This version enables to limit the exploration of suboptimal configurations in order to further enhance ESs. Specifically, SW-UCB1 is restricted to the configurations in $\mathcal{N}_{conf}$ which are adjacent to the current optimal configuration. Given the quasiconcave shape of the EE metric, such restriction prevents selecting the less energy-efficient configurations. Furthermore, as explained in subsection III-B, continuous network load variations result in a continuous shift of the EE optimum from a configuration to an adjacent one in $\mathcal{N}_{conf}$. Therefore, this algorithm also permits to efficiently track the optimal configuration which always remains in the restricted version of $\mathcal{N}_{conf}$.

As explained in subsection III-B, proving the quasiconcavity property of the EE metric under realistic propagation conditions remains a challenging open problem. In a real system, this property can be periodically checked by running for some period $T_{check}$ the standard SW-UCB1 algorithm. If this property is validated for a cell, one can switch to the constrained algorithm.

# V. NUMERICAL RESULTS

The following results present the EE optimization achieved by the two MAB-based RF channels switch on/off solutions introduced in section IV. Simulations are carried out using a system simulator with time resolution of a slot (0.5 ms).

## A. Simulation scenario

We consider a hexagonal network composed of $S = 7$ tri-sectorial macro sites which are deployed with an inter-site distance of 300 meters (see Fig. 1). Each cell is equipped with a UPA of $M = M_h \times M_v$ antennas where $M_h = M_v = 8$. Furthermore, we consider $N_h^{min} = N_v^{min} = 4$. In the proposed scenario, the three cells of site 1 are used for performance assessment of the studied ES feature, i.e., they are the cells of interest. They have a controllable number of active RF channels (i.e. $N_h^{min} \le N_h^c \le M_h$, $N_v^{min} \le N_v^c \le M_v$, $c \in \{1, 2, 3\}$), while the rest of 18 cells have fixed M-MIMO setup (i.e. $N_h^c = N_h^{min}$, $N_v^c = N_v^{min}$, $\forall c \in C\{1, 2, 3\}$) and are used to generate realistic interference.

The central frequency and the bandwidth of the system are set to $f_c = 3.5$ GHz and $B = 100$ MHz. As the SCS is set to $\mu = 30$ kHz, we have $K = 3333$ subcarriers. The Base Station (BS) transmit power is equal to $P_{tx} = 100$ W. The maximum number of simultaneously scheduled UEs is set to $U_{sch}^{max} = 4$. We assume here the same GBR for all UEs, namely $\forall u \in \mathcal{U}^c$, $r_u^{gbr} = 25$ Mbps.

The PC of each cell is computed while considering $\eta = 0.5$, $\hat{P}_{rf} = 0.4$ W, $P_{lo} = 0.2$ W and $P_{fix} = 10$ W. We consider here $\delta_N = 2$ so that the set of available antenna

<figure><img src="image_4.png" /><figcaption>Fig. 3: Trajectory of the selection ratio produced by the constrained SW-UCB1 algorithm for each configuration from $\mathcal{N}_{conf}$ ($\chi_{j,t}^c, j \in \{1, ..., |\mathcal{N}_{conf}|\}$), network load: 10% from 0 to 400 seconds and 30% afterwards.</figcaption></figure>

array configurations (see subsection III-B) corresponds to $\mathcal{N}_{conf} = \{(4, 4), (6, 4), (8, 4), (8, 6), (8, 8)\}$. In the proposed ES feature, the MAB algorithm associated to each cell $c$ periodically selects the configuration from $\mathcal{N}_{conf}$ to apply during a predefined period $T_{conf} = 1$ second. Such period permits to perform significant ESs based on the computation of a robust EE metric, while leaving enough time for the adaptation of control signals. We respectively set the MAB learning rate and the sliding-window size to $\alpha = 0.05$ and $\tau = 175$. Such setting enables to track the optimal configuration, in a time interval of $\tau \times T_{conf} = 175$ seconds.

## B. Optimal configuration tracking

We consider a 800 seconds simulation with the following traffic profile. An average network load of 10% is observed during the first 400 seconds and then increases to 30% for the last 400 seconds. Let $\chi_{j,t}^c = \frac{Q_{j,t}^c}{\tau}$ denote the selection ratio of the $j$-th configuration from $\mathcal{N}_{conf}^{c,t}$. Fig. 3 shows the trajectory of the selection ratio produced by the constrained SW-UCB1 algorithm for each configuration from $\mathcal{N}_{conf}$ ($\chi_{j,t}^c, j \in \{1, ..., |\mathcal{N}_{conf}|\}$). These trajectories are correlated to the average network load. Indeed, for an average network load of 10% the algorithm selects more frequently the (4, 4) configuration which provides the highest average EE (see Fig. 2). Moreover, as the network load increases from 10% to 30%, the algorithm tends to choose more frequently the (6, 4) and (8, 4) configurations which offer similar levels of average EE (see Fig. 2). A delay between the network load increase and the selection ratios adaptation can be observed. This is due to the sliding-window size $\tau$ and the exploration rate $\alpha$ which influence the frequency at which all configurations are retested.
<figure><img src="image_5.png" /><figcaption>Fig. 4: Average Energy Efficiency of the different RF channels switch on/off schemes as a function of network load conditions.</figcaption></figure>

### C. Performance evaluation

In Fig. 4 we assess the average EE offered by one baseline scheme and two RF channels switch on/off solutions for different average network loads. The baseline scheme is denoted by $(M_h, M_v)$ and represents a setting in which all RF channels are active for every transmission. The two RF channels switch on/off solutions correspond to the SW-UCB1 and the constrained SW-UCB1 algorithms presented in section IV. We observe that by adapting the antenna array configuration to the traffic conditions, the average EE is significantly improved. Specifically, the EE gains achieved by the two RF channels switch on/off solutions with respect to the baseline scheme decrease with network load since the $(M_h, M_v)$ configuration tends to become more optimal. For a 10% network load, the average EE is enhanced by respectively 28% and 33% with the SW-UCB1 and the constrained SW-UCB1 algorithms. The latter outperforms the former since it is able to limit the exploration of suboptimal configurations by exploiting the EE quasiconcave shape (see section IV). Finally, it is noted that both RF channels switch on/off solutions permit to significantly reduce network's PC without incurring any QoS degradation.

## VI. CONCLUSIONS

In this paper, we have presented two efficient RF channels switch on/off solutions to maximize EE under QoS constraints. They permit to sequentially select the optimal antenna array configuration from a predefined set of configurations. To this end, two MAB algorithms are adapted to non-stationary environments: SW-UCB1 and its constrained version leveraging the quasiconcave shape of the EE metric to limit the selection of suboptimal configurations. The numerical results show the capability of the proposed solutions to track the optimal configuration when considering non-stationary traffic conditions. The SW-UCB1 algorithm obtains significant EE gains with respect to a baseline solution at low and medium loads, while its constrained version further increases the gain by a few percent.

## REFERENCES

[1] E. Björnson, J. Hoydis, L. Sanguinetti, et al. "Massive MIMO networks: Spectral, energy, and hardware efficiency". Foundations and Trends in Signal Processing, 2017, vol. 11, no 3-4, p. 154-655.

[2] A. Gati, F. Salem, A. Serrano, et al. "Key technologies to accelerate the ICT Green evolution-An operator's point of view". arXiv preprint arXiv:1903.09627, 2019.

[3] D. López-Pérez, A. De Domenico, N. Piovesan, et al. "A survey on 5G radio access network energy efficiency: Massive MIMO, lean carrier design, sleep modes, and machine learning". IEEE Communications Surveys & Tutorials, 2022, vol. 24, no 1, p. 653-697.

[4] ETSI ES 203 228: "Environmental Engineering (EE); Assessment of mobile network energy efficiency".

[5] E. Björnson, L. Sanguinetti, J. Hoydis, et al. "Optimal design of energy-efficient multi-user MIMO systems: Is massive MIMO the answer?". IEEE Transactions on wireless communications, 2015, vol. 14, no 6, p. 3059-3075.

[6] Z. Xu, C. Yang, G. Li, et al. "Energy-efficient MIMO-OFDMA systems based on switching off RF chains". In: 2011 IEEE Vehicular Technology Conference (VTC Fall). IEEE, 2011. p. 1-5.

[7] H. Yu, L. Zhong, et A. Sabharwal. "Adaptive RF chain management for energy-efficient spatial-multiplexing MIMO transmission". In: Proceedings of the 2009 ACM/IEEE international symposium on Low power electronics and design. 2009. p. 401-406.

[8] H. Halbauer, A. Weber, D. Wiegner, et al. "Energy efficient massive MIMO array configurations". In: 2018 IEEE Globecom Workshops (GC Wkshps). IEEE, 2018. p. 1-6.

[9] N. Rajapaksha, J. Mohammadi, S. Wesemann, et al. "Minimizing Energy Consumption in MU-MIMO via Antenna Muting by Neural Networks with Asymmetric Loss". arXiv preprint arXiv:2306.05162, 2023.

[10] M. Hoffmann et P. Kryszkiewicz. "Reinforcement learning for energy-efficient 5G massive MIMO: Intelligent antenna switching". IEEE Access, 2021, vol. 9, p. 130329-130339.

[11] "Channel model(s) for 0.5-100 GHz," 3GPP. Sophia Antipolis Cedex, France, TR 38.901., Jun. 2018. [Online]. Available: www.3gpp.org.

[12] R. Combes et al., "Scheduling gain for frequency-selective Rayleigh fading channels with application to self-organizing packet scheduling," Elsevier Performance Evaluation, Vol. 68, Issue 8, Aug 2011.

[13] P. Auer, N. Cesa-Bianchi, et P. Fischer. "Finite-time analysis of the multiarmed bandit problem". Machine learning, 2002, vol. 47, p. 235-256.

[14] A. Garivier et E. Moulines. "On upper-confidence bound policies for switching bandit problems". In: International Conference on Algorithmic Learning Theory. Berlin, Heidelberg: Springer Berlin Heidelberg, 2011. p. 174-188.

[15] R. Combes et A. Proutiere. "Unimodal bandits: Regret lower bounds and optimal algorithms". In: International Conference on Machine Learning. PMLR, 2014. p. 521-529.
<figure><img src="image_5.png" /><figcaption>Fig. 4: Average Energy Efficiency of the different RF channels switch on/off schemes as a function of network load conditions.</figcaption></figure>

### C. Performance evaluation

In Fig. 4 we assess the average EE offered by one baseline scheme and two RF channels switch on/off solutions for different average network loads. The baseline scheme is denoted by $(M_h, M_v)$ and represents a setting in which all RF channels are active for every transmission. The two RF channels switch on/off solutions correspond to the SW-UCB1 and the constrained SW-UCB1 algorithms presented in section IV. We observe that by adapting the antenna array configuration to the traffic conditions, the average EE is significantly improved. Specifically, the EE gains achieved by the two RF channels switch on/off solutions with respect to the baseline scheme decrease with network load since the $(M_h, M_v)$ configuration tends to become more optimal. For a 10% network load, the average EE is enhanced by respectively 28% and 33% with the SW-UCB1 and the constrained SW-UCB1 algorithms. The latter outperforms the former since it is able to limit the exploration of suboptimal configurations by exploiting the EE quasiconcave shape (see section IV). Finally, it is noted that both RF channels switch on/off solutions permit to significantly reduce network's PC without incurring any QoS degradation.

## VI. CONCLUSIONS

In this paper, we have presented two efficient RF channels switch on/off solutions to maximize EE under QoS constraints. They permit to sequentially select the optimal antenna array configuration from a predefined set of configurations. To this end, two MAB algorithms are adapted to non-stationary environments: SW-UCB1 and its constrained version leveraging the quasiconcave shape of the EE metric to limit the selection of suboptimal configurations. The numerical results show the capability of the proposed solutions to track the optimal configuration when considering non-stationary traffic conditions. The SW-UCB1 algorithm obtains significant EE gains with respect to a baseline solution at low and medium loads, while its constrained version further increases the gain by a few percent.

## REFERENCES

[1] E. Björnson, J. Hoydis, L. Sanguinetti, et al. "Massive MIMO networks: Spectral, energy, and hardware efficiency". Foundations and Trends in Signal Processing, 2017, vol. 11, no 3-4, p. 154-655.

[2] A. Gati, F. Salem, A. Serrano, et al. "Key technologies to accelerate the ICT Green evolution-An operator's point of view". arXiv preprint arXiv:1903.09627, 2019.

[3] D. López-Pérez, A. De Domenico, N. Piovesan, et al. "A survey on 5G radio access network energy efficiency: Massive MIMO, lean carrier design, sleep modes, and machine learning". IEEE Communications Surveys & Tutorials, 2022, vol. 24, no 1, p. 653-697.

[4] ETSI ES 203 228: "Environmental Engineering (EE); Assessment of mobile network energy efficiency".

[5] E. Björnson, L. Sanguinetti, J. Hoydis, et al. "Optimal design of energy-efficient multi-user MIMO systems: Is massive MIMO the answer?". IEEE Transactions on wireless communications, 2015, vol. 14, no 6, p. 3059-3075.

[6] Z. Xu, C. Yang, G. Li, et al. "Energy-efficient MIMO-OFDMA systems based on switching off RF chains". In: 2011 IEEE Vehicular Technology Conference (VTC Fall). IEEE, 2011. p. 1-5.

[7] H. Yu, L. Zhong, et A. Sabharwal. "Adaptive RF chain management for energy-efficient spatial-multiplexing MIMO transmission". In: Proceedings of the 2009 ACM/IEEE international symposium on Low power electronics and design. 2009. p. 401-406.

[8] H. Halbauer, A. Weber, D. Wiegner, et al. "Energy efficient massive MIMO array configurations". In: 2018 IEEE Globecom Workshops (GC Wkshps). IEEE, 2018. p. 1-6.

[9] N. Rajapaksha, J. Mohammadi, S. Wesemann, et al. "Minimizing Energy Consumption in MU-MIMO via Antenna Muting by Neural Networks with Asymmetric Loss". arXiv preprint arXiv:2306.05162, 2023.

[10] M. Hoffmann et P. Kryszkiewicz. "Reinforcement learning for energy-efficient 5G massive MIMO: Intelligent antenna switching". IEEE Access, 2021, vol. 9, p. 130329-130339.

[11] "Channel model(s) for 0.5-100 GHz," 3GPP. Sophia Antipolis Cedex, France, TR 38.901., Jun. 2018. [Online]. Available: www.3gpp.org.

[12] R. Combes et al., "Scheduling gain for frequency-selective Rayleigh fading channels with application to self-organizing packet scheduling," Elsevier Performance Evaluation, Vol. 68, Issue 8, Aug 2011.

[13] P. Auer, N. Cesa-Bianchi, et P. Fischer. "Finite-time analysis of the multiarmed bandit problem". Machine learning, 2002, vol. 47, p. 235-256.

[14] A. Garivier et E. Moulines. "On upper-confidence bound policies for switching bandit problems". In: International Conference on Algorithmic Learning Theory. Berlin, Heidelberg: Springer Berlin Heidelberg, 2011. p. 174-188.

[15] R. Combes et A. Proutiere. "Unimodal bandits: Regret lower bounds and optimal algorithms". In: International Conference on Machine Learning. PMLR, 2014. p. 521-529.