# Green AI for 6G: Quantization of Neural Networks for Predictive Handover from Theory to Practice

Lucas Taucoory, Yassine Hadjadj-Aoul, Chaima Ghribi

► To cite this version:

Lucas Taucoory, Yassine Hadjadj-Aoul, Chaima Ghribi. Green AI for 6G: Quantization of Neural Networks for Predictive Handover from Theory to Practice. GLOBECOM 2025 - Annual IEEE Global Communications Conference, IEEE, Dec 2025, Taipe, Taiwan. hal-05455210

HAL Id: hal-05455210

https://inria.hal.science/hal-05455210v1

Submitted on 13 Jan 2026

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
# Green AI for 6G: Quantization of Neural Networks for Predictive Handover from Theory to Practice

Lucas Taucoory*, Yassine Hadjadj-Aoul†, and Chaima Ghribi*

*Nokia, Massy, France

†Univ Rennes, Inria, CNRS, IRISA, France

Lucas.Taucoory@nokia.com, yhadjadj@irisa.fr, Chaima.ghribi@nokia.com

***Abstract***—Quantization is widely regarded as a key technique for reducing inference latency and energy consumption in AI models, especially as 6G networks move toward AI-native architectures. However, its practical effectiveness varies significantly depending on the model and hardware platform. This paper investigates the impact of post-training quantization on LSTM, CNN and Transformer models used for predictive handover, a representative 6G telecom use case. We benchmark quantized and full-precision models on both CPU and GPU platforms, evaluating performance in terms of accuracy, latency and energy consumption. Our results show that while quantization can offer clear benefits, its gains are inconsistent particularly on many GPU hardware, due to limited INT8 support and suboptimal kernel execution. We identify current limitations and provide practical guidance for deploying quantized models effectively in telecom environments.

***Index Terms***—Post-Training Quantization, Predictive Handover, 6G Networks, Energy Efficiency

## I. INTRODUCTION

With the advent of 6G networks, the integration of AI and machine learning (ML) as native components within telecom infrastructure is expected to revolutionize network management and service delivery. These intelligent networks will rely heavily on ML models embedded in real-time functions such as predictive handover, where the goal is to anticipate user mobility and proactively switch connections to optimize quality of service (QoS) and reduce latency. However, the increasing computational demands of these ML models raise critical concerns regarding energy efficiency and the broader environmental sustainability of next-generation telecom systems.

Predictive handover is a challenging use case that requires processing sequential and spatiotemporal data to accurately forecast user movement patterns. To this end, models like Long Short-Term Memory networks (LSTM) are widely used for their ability to capture temporal dependencies, Convolutional Neural Networks (CNNs) for spatial feature extraction and Transformers for their superior modeling of long-range dependencies and parallelism. These architectures have demonstrated strong performance in related telecom tasks such as traffic prediction, anomaly detection and resource allocation. However, the wider adoption of these models in 6G raises critical questions and concerns regarding the sector's commitment to net-zero energy goals, particularly regarding their significant computational and energy demands.

Quantization is one of the most powerful techniques to decrease the computational time and memory consumption of neural network. It reduces numerical precision typically from FP32 to lower-bit formats such as INT8 offers a promising solution to reduce inference latency, memory footprint and energy consumption while maintaining model accuracy. When moving from 32-bit to 8-bit precision, the memory overhead of storing tensors decreases by a factor of four, while the computational cost for matrix multiplications can reduce quadratically by a factor of sixteen. Low-bit fixed-point representations such as INT8 further reduce energy consumption because fixed-point operations are inherently more efficient than floating-point computations [3]. Despite these theoretical advantages, exact latency improvements and energy savings are highly dependent on the model, the hardware platform and its support for low-precision operations.

Recent studies [2], [5], [9] have demonstrated the effectiveness of quantization in computer vision and natural language processing tasks. For example, Han et al. [2] showed significant energy savings using quantized CNNs on embedded devices, while Shafiee et al. [9] reported accuracy retention in quantized Transformer models for language tasks. In the telecom domain, prior research has applied ML models for handover prediction [6], [11], but comprehensive studies evaluating the impact of quantization on these models' accuracy, latency and energy consumption especially across diverse hardware platforms such as CPUs and GPUs remain limited. Moreover, while post-training quantization (PTQ) is popular for its simplicity [1], its effectiveness is often constrained by hardware support and model architecture. For example, many modern GPUs, frequently used for telecom ML inference, lack fully optimized INT8 kernels for sequence models like LSTMs and Transformers, limiting practical gains in energy and latency.

This paper addresses these gaps by systematically evaluating quantization's impact on LSTM, CNN, and Transformer models applied to predictive handover. We benchmark these models on CPU and GPU platforms, analyzing accuracy, runtime and energy consumption to understand the interplay between quantization, model type, and hardware capabilities. Our results identify key challenges and provide practical guidance for deploying quantized AI models efficiently in 6G telecom systems, thereby bridging the gap between theoretical potential and real-world constraints.
The remainder of this paper is structured as follows. Section II discusses PTQ energy efficiency challenges. Section III presents a brief overview of the related work. Section IV describes our predictive Handover use case and Model Architectures. Section V shows details of our experimental tested and results followed by conclusions in Section VII.

## II. CHALLENGES IN ENERGY-EFFICIENT POST-TRAINING QUANTIZATION

PTQ has been demonstrated in the literature to significantly accelerate inference [1]. Theoretically, INT8 quantization can achieve up to a 4 times speedup, while FP16 quantization typically provides around a 2 times improvement. However, the realized gains in inference speed and energy efficiency depend heavily on several factors, including model architecture, hardware capabilities, deployment frameworks, software library support and hyperparameter configurations. Key challenges to fully exploiting the benefits of PTQ include:

- **Model Architecture Sensitivity:** The benefits of quantization is highly dependent on the model's size and internal structure. Smaller models often exhibit limited speedup, while larger models offer greater potential for acceleration but may include layers that are less amenable to quantization.

- **Hyperparameter Sensitivity:** The performance of quantized models can be strongly influenced by calibration hyperparameters, such as batch size or calibration dataset size. Inappropriate settings may lead to unstable quantization scales.

- **Hardware Dependence:** The speedup and energy gains from PTQ depend strongly on the target hardware, with CPUs, edge AI accelerators, and high-end GPUs offering efficient INT8 and FP16 inference, whereas many general-purpose GPUs lack optimized INT8 kernel support.

- **Quantization Tools and Libraries:** Quantization support across deployment frameworks (e.g., TensorRT, ONNX Runtime, PyTorch) varies in maturity and optimization level. Some libraries lack efficient INT8 implementations for complex operations, and in certain cases, the provided quantization schemes are limited to fake quantization for calibration or simulation, offering little to no runtime benefit on GPU.

## III. RELATED WORK

Quantization is gaining interest in telco as it promises to reduce the energy overhead of AI workloads. Authors in [10] and [7] explore the use of quantization to improve the efficiency of federated learning (FL) in 5G+. Recent efforts, such as in [4], have evaluated the practical benefits of LLM quantization at the edge. While existing studies primarily focus on FL and LLMs, our work extends the investigation to a broader set of neural network architectures including CNNs, LSTMs and Transformers within the predictive handover use case. We perform a practical and comprehensive evaluation of post-training quantization across CPU and GPU platforms, taking

into account various deployment frameworks and quantization libraries. In addition to traditional metrics like accuracy, we explicitly measure both the energy cost and energy savings introduced by quantization, a perspective that, to the best of our knowledge, has not been thoroughly explored in prior work. This allows us to assess the efficiency trade-offs of quantization in telecom environments and identify deployment scenarios where it provides meaningful gains.

## IV. PREDICTIVE HANDOVER: USE CASE DESCRIPTION AND MODEL ARCHITECTURES

### A. Predictive Handover

We consider the predictive handover use case described in [8], which leverages machine learning to optimize mobility decisions based on UE measurements, rather than relying on predefined mobility procedures. This approach allows the mobility to be dynamically optimized for each UE and situation while considering the predicted radio conditions. The predictive handover task involves determining, at each time step, the cell providing the highest connection quality. The input dataset is proprietary and derived from real network measurements, comprising data from 10 UEs connected to 7 cells over 450,000 time steps. It is organized into sequences of 50 time steps, where each step contains the Signal-to-Interference-plus-Noise Ratio (SINR) values of a UE with respect to the seven cells. Correspondingly, the output labels identify the optimal serving cell at each time step. This task can thus be formulated as a sequence classification problem, where the model infers the best serving cell over time from the observed SINR patterns.

### B. Model Architectures

In this paper, we propose three types of models for the predictive handover use case: Long Short-Term Memory (LSTM), Convolutional Neural Network (CNN) and Transformer architectures.

* **LSTM:** A two-layer LSTM network with 128 hidden units, processing sequences of 7 features, followed by dropout ($p = 0.2$) and a fully connected layer for 7-class classification.

* **CNN:** 1D CNN with two convolutional layers (64 and 128 filters, kernel size 3), each followed by ReLU activation and max pooling, then a fully connected block (256 units, dropout $p = 0.5$) for 7-class classification.

* **Transformer:** Transformer encoder model with 4 layers, 8 attention heads, and an embedding dimension of 128, using input projection, learned positional encoding, and a two-layer feedforward classification head for 7-class prediction.

### C. Post-Training Quantization for Handover Prediction

To optimize the handover prediction models for energy efficiency, we applied post-training quantization techniques across the different architectures. Post-training quantization is a model compression method that reduces the numerical precision of weights and activations—typically from 32-bit floating
point to lower bit-width formats such as 8-bit integers. This approach is known to significantly decrease the model size and computational complexity, enabling faster inference and lower power consumption, which are critical for deployment in resource-constrained environments.

<figure><img src="image_2.png" /><figcaption>Fig. 1: Post-Training Quantization for Handover Prediction</figcaption></figure>

## V. EXPERIMENTAL RESULTS

In this section, we first describe the experimental setup, including model training configurations, hardware platforms and the libraries used. We then present a detailed analysis of the results organized by model architecture. Our study focuses on inference speedup, variations in energy consumption due to quantization, and the energy cost of the quantization process itself. Finally, we summarize the key findings.

### A. Experimental Setup

**- Training:** All models were trained using the Cross-Entropy Loss function and the Adam optimizer. For the LSTM and CNN models, we used a learning rate of $1 \times 10^{-3}$ without weight decay. For the Transformer model, we used a learning rate of $1 \times 10^{-4}$ with a weight decay of $1 \times 10^{-5}$. Training was performed on an Intel® Xeon® Gold 5218 CPU @ 2.30 GHz and two NVIDIA A100 80GB GPUs.

**- GPU pipeline:** FP32 models are exported from PyTorch to ONNX and run with NVIDIA TensorRT. FP16 and BF16 cast all weights and activations to the target precision using IEEE-754 rounding rules. INT8 uses uniform symmetric per-tensor quantization, calibrated with 2048 random training samples. For LSTM and Transformer models, unsupported INT8 layers default to FP32; for comparison purposes, we define an INT8 with FP16 fallback (I8FP16) mixed precision type where these layers run in FP16.

**- CPU pipeline:** ONNX models are executed in ONNX Runtime with the OpenVINO provider for Intel CPU optimizations. FP16/BF16 casting and INT8 calibration match the GPU setup, but precision fallback for unsupported LSTM/Transformer layers is not straightforward in ONNX Runtime, so no fallback adjustment was applied.

**- Energy measurement:** Energy consumption was measured on both CPU and GPU using the CodeCarbon library. Calibration and quantization for INT8 models consumed approximately $6 \times 10^{-5}$ kWh on CPU and $8 \times 10^{-4}$ kWh on GPU, while precision casting for FP16 and BF16 models required

approximately $2 \times 10^{-5}$ kWh on CPU and $4 \times 10^{-4}$ kWh on GPU. These costs correspond to the most energy-efficient settings obtained (8 batches of size 256 for INT8 calibration) and are negligible compared to the cumulative inference energy in our predictive handover scenario, where models operate online at high frequency.

**- Benchmarking setup:** The benchmark was conducted by running inference with both baseline and quantized models on 65,000 test samples, recording accuracy and energy consumption. All reported differences in energy consumption were assessed for statistical significance using a two-tailed Student's t-test at the 5% significance level.

### B. CNN: Result Analysis

<figure><img src="image_3.png" /><figcaption>Fig. 2: Average CNN energy consumption on CPU for different precisions and batch sizes $\le 16$.</figcaption></figure>

<figure><img src="image_4.png" /><figcaption>Fig. 3: Relative energy variation (%) vs. FP32 baseline for CNN on GPU across precisions and batch sizes $\le 16$.</figcaption></figure>

As depicted in Figure 2 and 3, on the CPU server, BF16 shows significant energy reductions compared to FP32 at batch sizes 1, 2 and 16, with the largest reduction of **26.6%** at batch size 1. FP16 resulted in a significant reduction only at batch size 1, with a **25.3%** improvement. INT8 achieved a statistically significant energy saving of **6.3%** at batch size 1, but consumed **6.3% more energy** than FP32 at batch size
2, also with statistical significance. For larger batch sizes, none of the quantized formats showed statistically significant differences from FP32.

<figure><img src="image_5.png" /><figcaption>Fig. 4: Average CNN energy consumption on GPU for different precisions and batch sizes ≤ 16</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>Fig. 5: Relative energy variation (%) vs. FP32 baseline for CNN on GPU across precisions and batch sizes ≤ 16.</figcaption></figure>

Figures 4 and 5 illustrate that, on the GPU, at batch size 1, both BF16 and FP16 consumed significantly more energy than FP32, with increases of **7.1%** and **8.4%**, respectively, contrasting with the CPU results where BF16 and FP16 offered notable energy savings at this batch size. At batch size 2, all quantized formats (BF16, FP16, INT8) showed significant energy reductions, between **8.6%** and **10.6%**. At batch size 4, BF16 and INT8 used significantly more energy than FP32, while FP16 did not bring any significant change. At batch size 8, BF16 and FP16 again consumed more energy; INT8 showed no significant difference. At batch size 16, only BF16 showed a significant reduction of **6.1%**, while the other formats did not result in any significant change compared to FP32.

### C. LSTM: Result Analysis

As shown in Figures 6 and 7, at batch size 1, both BF16 and FP16 consumed significantly more energy than FP32, with increases of **9.2%** and **3.2%**, respectively, while INT8

showed no significant change. At batch size 2, BF16, FP16, and INT8 all consumed significantly more energy than FP32, with increases ranging from **1.6%** to **17.3%**.

<figure><img src="image_7.png" /><figcaption>Fig. 6: Average LSTM energy consumption on CPU for different precisions and batch sizes ≤ 16</figcaption></figure>

Figures 8 and 9 reveal that, at batch size 4, BF16 and INT8 delivered significant reductions in energy consumption of **10.9%** and **14.7%**, respectively, while FP16 showed no significant change. At batch size 8, all quantized formats consumed significantly more energy than FP32, with increases between **3.1%** and **9.4%**. At batch size 16, FP16 and INT8 both showed significant energy reductions of approximately **14%**, while BF16 consumed significantly more energy than FP32 (**15.6%**).

<figure><img src="image_8.png" /><figcaption>Fig. 7: Relative energy variation (%) vs. FP32 baseline for LSTM on CPU across precisions and batch sizes ≤ 16.</figcaption></figure>

As presented in Figures 8 and 9, at all batch sizes, BF16, FP16, and the INT8 with FP16 fallback configuration (INT8+FP16FB) consistently consumed significantly less energy than FP32, with improvements ranging from **5.8%** to **11%**. INT8, in contrast, consistently consumed slightly more energy than FP32 across all batch sizes, with statistically significant differences, though the margins were small (around **1–3%**). This behavior is likely related to recurring fallbacks to FP32 precision, due to limited support for LSTM operations in TensorRT.
<figure><img src="image_9.png" /><figcaption>Fig. 8: Average LSTM energy consumption on GPU for different precisions and batch sizes ≤ 16</figcaption></figure>

<figure><img src="image_10.png" /><figcaption>Fig. 9: Relative energy variation (%) vs. FP32 baseline for LSTM on GPU across precisions and batch sizes ≤ 16.</figcaption></figure>

## D. Transformer: Result Analysis

<figure><img src="image_11.png" /><figcaption>Fig. 10: Average Transformer energy consumption on CPU for different precisions and batch sizes ≤ 16</figcaption></figure>

Figures 10 and 11 reveal that, at batch size 1, BF16 showed a significant energy reduction of **5.2%**, while FP16 and INT8 both consumed significantly more energy than FP32, with increases of **1.8%** and **78.8%**, respectively. At batch size 2,

FP16 showed a significant energy reduction of **6.1%**, whereas BF16 and INT8 both consumed significantly more energy. At batch size 4, INT8 and BF16 again showed significantly higher energy consumption, while FP16 did not result in any significant change. At batch size 8, only FP16 showed a significant improvement (**14.2%** reduction); BF16 and INT8 showed no significant change. At batch size 16, BF16 showed a small but significant reduction of **2.4%**, while FP16 and INT8 consumed significantly more energy than FP32.

<figure><img src="image_12.png" /><figcaption>Fig. 11: Relative energy variation (%) vs. FP32 baseline for Transformer on CPU across precisions and batch sizes ≤ 16.</figcaption></figure>

<figure><img src="image_13.png" /><figcaption>Fig. 12: Average Transformer energy consumption on GPU for different precisions and batch sizes ≤ 16</figcaption></figure>

Finally, as evidenced by Figures 12 and 13, at batch size 1, FP16, BF16, and INT8 with FP16 fallback all showed significant energy reductions between **6.4%** and **7.6%**, while INT8 showed no significant change. At batch size 2, FP16 and INT8 with FP16 fallback again showed significant reductions, while BF16 did not show any significant difference. INT8 consumed slightly more energy, though the increase was not statistically significant. At batch size 4, all quantized formats consumed significantly less energy than FP32, with reductions ranging from **5.0%** (INT8) to **14.8%** (BF16). At batch size 8, all formats again showed significant reductions, with INT8+FP16FB, FP16, and BF16 achieving **8–13%** savings.
At batch size 16, all quantized formats showed significant
differences: INT8 consumed slightly more energy, while all
others reduced energy consumption, with FP16 achieving the
largest reduction at **15.9%**.

<figure><img src="image_14.png" /><figcaption>Fig. 13: Relative energy variation (%) vs. FP32 baseline for Transformer on GPU across precisions and batch sizes ≤ 16.</figcaption></figure>

VI. SUMMARY

6G will enable a hybrid AI deployment, where AI workloads dynamically shift between device, edge and cloud. Our results indicate that quantization on CPU delivers notable energy and latency improvements, particularly at small batch sizes and for CNN architectures. This makes post-training quantization a promising technique for real-time, latency-sensitive inference in telecom use cases, especially where GPU acceleration is unavailable or energy is constrained. On GPU platforms, the benefits of post-training quantization are less consistent. While CNNs may benefit from partial INT8 acceleration, many GPUs are primarily optimized for floating-point operations, with limited INT8 support for sequence-based models like LSTMs and Transformers. Incomplete kernel coverage, fallback to FP32 operations and suboptimal memory usage often result in marginal or unpredictable gains. In our experiments, these limitations make GPU-based quantized inference less suitable for real-time or low-power telecom applications, especially for models with complex attention or recurrent structures. Importantly, accuracy was unaffected for FP16 and BF16 compared to FP32, and INT8 quantization resulted in at most a 1 % decrease across all models and hardware, indicating that energy and latency trade-offs observed do not come at the cost of significant predictive performance loss on this specific use case. Furthermore, the one-time energy cost of quantization can be large relative to per-inference consumption for small workloads, but in high-volume or continuous deployments it is quickly amortized. Quantization strategies should therefore be adapted to the expected usage patterns and deployment context to ensure the energy overhead is justified.

VII. CONCLUSION AND FUTURE WORK

This study examined the impact of post-training quantiza-
tion on CNN, LSTM, and Transformer models for predictive

handover in 6G telecom environments. Experimental results
demonstrate that while quantization can offer notable infer-
ence speedups and energy savings, these benefits are highly
dependent on the model architecture, batch size and hardware
platform, with GPU deployments showing more variability.
Our analysis highlights the practical challenges of quantization
in telecom use cases and provides guidance for effectively
leveraging quantization to enhance energy efficiency without
compromising accuracy. In future work, we plan to extend this
study to edge devices, where energy constraints are even more
critical, and to evaluate quantization-aware training (QAT) and
quantization on LLMs tailored for telecom tasks, where the
balance between efficiency and accuracy will be increasingly
vital for real-world deployment in 6G networks.

REFERENCES

[1] Amir Gholami, Sehoon Kim, Zhen Dong, Zhewei Yao, Michael W. Mahoney, and Kurt Keutzer. A survey of quantization methods for efficient neural network inference, 2021.

[2] Song Han et al. Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. In *International Conference on Learning Representations (ICLR)*, 2016.

[3] Mark Horowitz. Computing's energy problem (and what we can do about it). *IEEE International Solid-State Circuits Conference Digest of Technical Papers (ISSCC)*, pages 10–14, 2014.

[4] Erik Johannes Husom et al. Sustainable llm inference for edge ai: Evaluating quantized llms for energy efficiency, output accuracy, and inference latency, 2025.

[5] Benoit Jacob, Skirmantas Kligys, Bo Chen, Menglong Zhu, Matthew Tang, Andrew Howard, Hartwig Adam, and Dmitry Kalenichenko. Quantization and training of neural networks for efficient integer- arithmetic-only inference. In *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)*, pages 2704–2713, 2018.

[6] Jongmin Kim et al. Machine learning-based handover optimization in 5g networks. *IEEE Transactions on Vehicular Technology*, 2021.

[7] Yutao Liu, Chenyu Fan, and Xiaoning Zhang. Accelerating federated learning with adaptive test-driven quantization at edge network. In *2024 IEEE 24th International Conference on Communication Technology (ICCT)*, pages 174–179, 2024.

[8] Ahmed Masri, Teemu Veijalainen, Henrik Martikainen, Stephen Mwanje, Janne Ali-Tolppa, and Márton Kajó. Machine-learning-based predictive handover. In *2021 IFIP/IEEE International Symposium on Integrated Network Management (IM)*, pages 648–652, 2021.

[9] Mohammad Javad Shafiee et al. Quantizing transformers for efficient inference. In *NeurIPS Workshop on Efficient Methods for Deep Neural Networks*, 2020.

[10] Dian Shi et al. Toward energy-efficient federated learning over 5g+ mobile devices. *IEEE Wireless Communications*, 29(5):44–51, 2022.

[11] Haoyu Zhang et al. Predictive mobility management for 6g networks: A deep learning approach. *IEEE Communications Magazine*, 2022.

<figure><img src="image_15.png" /></figure>

6G SNS

**Acknowledgment:**

This work is Co-funded by the European
Union under Grant Agreement 101191936.
Views and opinions expressed are however
those of the author(s) only and do not nec-
essarily reflect those of all SUSTAIN-6G
consortium parties nor those of the European
Union or the SNS JU (granting authority).
Neither the European Union nor the granting
authority can be held responsible for them.