# Predictive Resource Management in the Computing Continuum: Transfer Learning from Virtual Machines to Containers using Transformers

Danny de Novi, Lorenzo Carnevale, Daniel Balouek-Thomert, Manish Parashar, Massimo Villari

<figure><img src="image_1.png" /></figure>

## ► To cite this version:

Danny de Novi, Lorenzo Carnevale, Daniel Balouek-Thomert, Manish Parashar, Massimo Villari. Predictive Resource Management in the Computing Continuum: Transfer Learning from Virtual Machines to Containers using Transformers. UCC 2025 - IEEE/ACM 18th International Conference on Utility and Cloud Computing, Dec 2025, Nantes, France. pp.1-8, 10.1145/3773274.3774856. hal-05468841

HAL Id: hal-05468841

https://hal.science/hal-05468841v1

Submitted on 21 Jan 2026

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_2.png" /></figure>
# Predictive Resource Management in the Computing Continuum:
## Transfer Learning from Virtual Machines to Containers using
## Transformers

Danny De Novi*

Lorenzo Carnevale*

Daniel Balouek*

University of Messina

University of Messina

IMT Atlantique, Inria, LS2N

Messina, Italy

Messina, Italy

Nantes, France

danny.denovi@unime.it

lcarnevale@unime.it

daniel.balouek@inria.fr

Manish Parashar*

Massimo Villari*

Scientific Computing and Imaging

University of Messina

Institute, University of Utah

Messina, Italy

Salt Lake City, USA

mvillari@unime.it

manish.parashar@utah.edu

## Abstract

Efficient workload forecasting is a key requirement for optimizing resource management across the computing continuum, spanning from edge deployments to cloud infrastructures. In this paper, we propose a Temporal Transformer architecture for predicting CPU utilization, designed to capture both short-term fluctuations and long-range dependencies in workload dynamics. The model is first pretrained on a large-scale Microsoft Azure VM dataset and then fine-tuned on the Alibaba container dataset, enabling effective transfer learning across different virtualization environments. Experimental results show that the proposed approach consistently achieves high accuracy while maintaining a small model size and inference times suitable for real-time operation. Qualitative analysis further confirms the model's ability to reproduce workload patterns with high fidelity. These findings demonstrate that the proposed Temporal Transformer is both accurate and efficient, making it suitable not only for cloud platforms but also for edge intelligence scenarios.

## CCS Concepts

* **Do Not Use This Code → Generate the Correct Terms for Your Paper; Generate the Correct Terms for Your Paper;** Generate the Correct Terms for Your Paper; Generate the Correct Terms for Your Paper.

## Keywords

Cloud-Edge Continuum, Time-Series Forecasting, Artificial Intelligence Operations, Temporal Transformer

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org.

UCC 2025, Nantes, France

© 2025 Copyright held by the owner/author(s). Publication rights licensed to ACM.

ACM ISBN 978-1-4503-XXXX-X/2018/06

https://doi.org/XXXXXXX.XXXXXXX

## ACM Reference Format:

Danny De Novi, Lorenzo Carnevale, Daniel Balouek, Manish Parashar, and Massimo Villari. 2025. Predictive Resource Management in the Computing Continuum: Transfer Learning from Virtual Machines to Containers using Transformers. In *Proceedings of The 18th IEEE/ACM International Conference on Utility and Cloud Computing (UCC 2025)*. ACM, New York, NY, USA, 10 pages. https://doi.org/XXXXXXX.XXXXXXX

## 1 Introduction

The accelerated deployment of cloud, fog, and edge infrastructures has brought the computing continuum paradigm into the spotlight. This continuum promises latency-sensitive applications, real-time analytics, and pervasive artificial intelligence by leveraging heterogeneous resources heterogeneous and dynamically shared resources [8].

However, the heterogeneity of computational resources and uncertainty of workloads across this continuum introduce significant challenges for resource management. Ensuring that the right computational resources are selected and provisioned efficiently requires accurate workload prediction [11, 15]. Effective prediction mechanisms allow systems to anticipate demand, improve scheduling decisions, and maintain performance and energy efficiency at scale [22].

Recent advances in machine learning have significantly improved workload forecasting for resource management in cloud infrastructures, where virtualization technologies are predominant. With the growing adoption of containerization, resource allocation must now adapt to finer-grained and more dynamic execution environments. While containers offer better elasticity and faster orchestration compared to VMs, their volatile workload pose additional challenges for prediction.

Bridging the predictive models for CPU utilization developed for VMs with the requirements of containers thus presents an important feature within the continuum. Traditional methods provide partial solutions but often struggle to generalize across heterogeneous environments and to capture both short-term variability and long-range dependencies (Section 3). These limitations hinder their use in real-world deployment of services where workloads are dynamic and diverse.

<footnote>*Both authors contributed equally to this research.</footnote>
Transformers, originally from the natural language processing community, have recently shown strong performance in time-series forecasting, due to their ability to model temporal dependencies without the constraints of recurrent architectures [23]. Temporal transformers enable the modeling of complex workload dynamics, making them an encouraging method for resource usage prediction [12]. However, their application to resource utilization, in particular across different virtualization platforms, remains underexplored.

In this paper, we propose a Temporal Transformer architecture for CPU forecasting across the continuum. We investigate how predictive models trained on VM workload traces can be adapted to containerized environments. Evaluation is performed over real-world datasets using a pre-training on Microsoft Azure virtual machine traces and fine-tuning on Alibaba container traces, enabling knowledge transfer across virtualization boundaries. Results demonstrate that our approach combines high accuracy with low-latency inference and a compact model size, making it suitable for both cloud and edge scenarios.

The remainder of the paper is organized as follows. Section 2 details the motivation behind this work. Section 3 introduces related work on resource management and forecasting. Section 4 describes the selected datasets, formulates the problem, and presents the proposed model. The environmental setup, metrics, and results are discussed in Section 5. Section 6 draws conclusions and shines a light on future work.

## 2 Motivation

The continuous growth of heterogeneous computing resources—including cloud data centers, edge and fog nodes, and billions of IoT devices—has fundamentally reshaped the way we provision resources for geo-distributed applications. The cloud offers virtually unlimited scalability and centralized management, the edge ensures proximity to end-users with low-latency processing, fog nodes enable intermediate aggregation and coordination, and IoT devices act as pervasive data producers. The computing continuum has been proposed as a paradigm to unify these diverse resources into a cohesive, end-to-end infrastructure, where computation, storage, and networking can be dynamically allocated and migrated across layers.

A compelling example of a dynamic application that relies on the computing continuum is disaster management. During natural disasters such as earthquakes, floods, or wildfires, IoT devices—including sensors, drones, and cameras—collect critical real-time data about environmental conditions and human activity [1, 3]. Meanwhile, the cloud enables large-scale data fusion and predictive analytics, but its integration must be carefully managed to account for fluctuating bandwidth and latency constraints during emergencies. This dynamic interplay highlights the need for advanced resource management strategies that can anticipate demand, reallocate workloads across layers in real time, and guarantee resilience under highly uncertain and evolving conditions.

One concrete resource management strategy in disaster scenarios is the use of predictive workload offloading with priority-aware scheduling. In this approach, edge nodes employ lightweight predictive models such as LSTM to forecast short-term workload surges

from IoT devices such as drones or mobile sensors. When a surge is detected, latency-critical tasks (e.g., image recognition for victim detection) are prioritized and retained at the edge, while less time-sensitive tasks (e.g. global pattern analysis) are proactively offloaded to fog or cloud resources.

In this context, fog or cloud nodes dynamically balance tasks among neighboring edges based on resource availability and connectivity status. This decision-making process is heavily depended on the knowledge of neighboring edge capacities, connectivity status, and latency budgets to effectively redistribute tasks. Maintaining a global state of the system is costly and not feasible in real-time or at scale. Our motivation for the continuum is to explore workload prediction based on historical workload traces to proactively reserve resources, preemptively offload non-critical tasks, and minimize task migration overhead.

In this work, we propose a temporal transformer to build a CPU forecasting model for the computing continuum. CPU readings from different virtualization systems, such as VMs and containers, have been considered. Therefore, data are selected from the Microsoft Azure and Alibaba Trace datasets. In particular, transfer learning plays a central role in enabling models trained in the cloud to be rapidly adapted to heterogeneous and resource-constrained environments at the edge and fog.

Our proposed approach focuses on realizing the following design goals:

* Low-latency, lightsize, non-intensive inference and control

* Accurate workload prediction for short-term predictions

* Robustness of model performance under novel conditions

## 3 Related Work

Resource management plays an important role in optimizing large and micro-data centers [9, 10]. In this regard, we are interested in the role of CPU utilization for the prediction of the workload. The state of the art is an exploration of the literature mixing the keywords CPU, usage, utilization, prediction and forecasting. Generally speaking, the most popular models adopted by the scientific communities are the ARIMA and LSTM [17, 19] models. Not many solutions adopt Transformers to solve the task. In the following, we reference the most relevant ones.

Duggan et al. [7] proposed a study which focused on the comparison between prediction models that were one and many steps ahead. The model was a RNN with a Back-Propagation-Through-Time (BPTT) algorithm. The input size was two, considering the current and previous samples. Three hidden neurons were used in the hidden layer. PlanetLab was the dataset, out of 800 cloud hosts machines simulated with the CloudSim simulator. CPU utilization values were measured every 5 minutes. The model converges to a good solution in a short time, outperforming the compared models (e.g., Random Walk, Moving Avg, Backpropagation). The analysis on multi-steps highlights a slow degradation of the model while the steps increase. This means that the model may be used to train more steps ahead (e.g., 20, 30 minutes), but this is inconvenient to extend the time.

Janardhanan et al. [14] investigated CPU workload forecasting by comparing the LSTM and ARIMA models. Their study used Google Cluster Data Trace, which includes resource monitoring
for 29 consecutive days. The dataset is over 300 GB and consists of scheduling information of over 12000 data center machines. Data for a single machine were extracted, creating a new dataset of about 8300 readings. The ARIMA model correctly predicts near-term predictions, but failed to predict long-term values. The LSTM forecasted both near-term and long-term values better. The paper lacked reproducibility, such as input shape and size, accurate details on the LSTM architecture, or any source link for codebase and dataset.

Mason et al. [16] investigated the problem of CPU prediction for VMs by applying bioinspired AI methods to optimize a Recurrent Neural Network (RNN). The input size was 2. The output was one, representing the next predicted CPU reading. The Particle Swarm Optimization (PSO), the Covariance Matrix Adaptation Evolutionary Strategy (CMA-ES) and the Differential Evolution (DE) were used as optimization algorithms for fine tuning the RNN model. The dataset was the PlanetLab. It includes CPU utilization samples measured every 5 minutes in a 24h day.

Bauer et al. [2] introduced UtilML, a resource prediction system based on LSTM for the computing continuum. The Rectified Linear Unit (ReLU) activation function was replaced with the LeakyReLU, an extension that is beneficial when there is a large amount of negative values. Batch normalization was proposed to mitigate internal covariate shifts. The dataset was the Alibaba Cloud GPU trace. The model was compared with baseline variants considering RMSE, MAPE and sMAPE.

Daraghmeh et al. [6] proposed a multilevel learning model composed of an Isolation Forest to perform anomaly detection, a k-means to cluster data and discern recurrent CPU usage patterns, a pool of models (e.g., Gradient Boosting Regressor, Random Forest Regressor, Extreme Gradient Boosting, AdaBoost Regressor, more) ens ambled in a number of the best 4. The input size was fixed to 60 minutes with samples of 5 minutes each. During the anomaly detection phase, an anomaly score is assigned to each data point. Points that exceed a threshold are classified as anomalies. During the data clustering phase, the Elbow method and the knee point detection algorithm were used to determine the right number of clusters. During the ensamble phase, models are integrated using stacking or voting regression methods. Many metrics were used to validate the three phases. The Gradient Boosting Regressor outperforms the others in terms of R-squared, MAE and MSE.

Wang et al. [22] highlights the need to rethink IT operations with the advent of artificial intelligence (AI). This new trend is called AIOps. The proposed approach includes more phases. The first to detect outliers with the 4-sigma algorithm. Data were then filtered by applying the Butterworth filter, a low-pass filter that cuts off the outliers. Data were collected any 30 minutes for 13 days in a custom dataset. Preprocessed data are input of more statistical (e.g., SARIMAX, Prophet, Holt-Winters), machine learning (e.g., XGBoost) and deep learning (e.g., LSTM) methods. Finally, an appropriate estimator is chosen as the metamodel, taking in input the results of the previous predictions. Therefore, a final prediction is generated. Models were evaluated in terms of MAE, MSE and MAPE. The XGBoost model performed better than the others, whereas the SARIMAX was the worst.

Roque et al. [20] proposed a resource allocation approach for Network Function Virtualization (NFV) based on the prediction of

CPU utilization. In this regard, they built a Support Vector Regression (SVR) model optimized with the Particle Swarm Optimization (PSO) algorithm. The prediction is computed one step ahead. The results were compared with other algorithms (e.g., Artificial Neural Network, ARIMA).

Wang et al. [21] discussed the importance of building effective models for the prediction of extreme load conditions in CPU utilization. More specifically, they proposed ExtremoNet, which incorporates isolated forest to identify the outliers and a regression problem based on timeseries to predict 1 step in the future. The timeseries included k-dimensions, representing not only the CPU utilization, but also, e.g., memory and network. The features were selected using the Pearson correlation, but any verification of the suitability of the methodology was provided. The dataset was the AliCloud and the authors selected the container trace. The model was evaluated with MAE, MSE and R-squared. The model was compared with three models from the literature.

Carnevale et al. [4] discussed a distributed solution based on the federated learning paradigm. A bidirectional LSTM was developed and distributed over a number of local zone clients. The weights of the neural network were, therefore, aggregated on a regional server, using the Federated Average aggregation algorithm. The methodology preserves data privacy and reduces latency by moving the computation to the edge of the network, where data are generated. Performance analysis, in terms of R², MSE, RMSE and MAE, prove the quality of the methodology.

The discussed works cover a broad range of knowledge about the prediction of CPU utilization. LSTM [18] was often used as a reference model to predict one step ahead (e.g., 5 minutes) with an input size of 2. Only the work [2] mentions the possibility of using such a model for the computing continuum, but the dataset was trained on a GPU trace dataset, which is typically, not the hardware architecture of edge devices. We aim to build a compute continuum model, training in the phases. During the first phase, the model is trained on the Microsoft Azure Trace dataset to build a pretrained based on VMs. During the second phase, the model is re-trained on the Alibaba Trace dataset to build a finetuning based on containers.

# 4 Material and Methods

## 4.1 Dataset

Considering the target of this paper is building a model for the Computing Continuum paradigm, that is, including cloud and edge resources, we selected two datasets, one for each kind of resources. Indeed, the cloud dataset is the Microsoft Azure Trace, while the edge dataset is the Alibaba Cloud Trace.

**Microsoft Azure Trace Dataset:** The Microsoft Azure trace dataset<fnref n="1" /> was released in 2019 [5]. Dataset includes metrics from more than 2,5 million VMs. The uncompressed dataset has a size of 235 GiB divided into 198 files. VM CPU utilization was sampled every 5 minutes for 30 consecutive days. The total of samples recorded more than 100 million hours and almost 2 billion readings. Dataset schema contains many attributes divided into multiple files (e.g., deployment size, VM virtual core count bucket VM memory bucket). However, we focus on the attributes reported in the table

<footnote n="1">¹https://github.com/Azure/AzurePublicDataset</footnote>
1. Services deployed in the VMs are first-parties (internal services, i.e., research, development, testing) and third-parties (costumers services, i.e., web service, e-commerce). Services are balanced between Infrastructure as a Service (IaaS) and Platform as a Service (PaaS). Most of the VMs have a few virtual cores and no more than 4 GB.

<table><caption>**Table 1: Features description of the Microsoft Azure Trace Dataset**</caption><thead><tr><td>metric</td><td>description</td></tr></thead><tbody><tr><td>Timestamp</td><td>The time when the sample was collected</td></tr><tr><td>VM ID</td><td>The unique identifier of the virtual machine</td></tr><tr><td>Minimum CPU</td><td>The minimum CPU sampled at that time</td></tr><tr><td>Maximum CPU</td><td>The maximum CPU sampled at that time</td></tr><tr><td>Average CPU</td><td>The average CPU sampled at that time</td></tr></tbody></table>

**Alibaba Cloud Trace Dataset:** The Alibaba Cloud Trace dataset<fnref n="2" /> was released in 2022 [24]. Dataset includes metrics from more than 40 thousand bare-metal nodes during 13 days, including more than 470 thousand containers for more than 28 thousand microservices. The uncompressed dataset has a size of more than 2TB divided into multiple files. The CPU utilization of the containers was sampled every minute for 13 consecutive days. Data schema contains many attributes divided into multiple files (e.g., bare-metal node resources, container resources, microservice call rate and response time). However, we focus on the attributes reported in the table 2. There is no information about the type of services deployed in the containers.

<table><caption>**Table 2: Features description of the Alibaba Cloud Trace Dataset**</caption><thead><tr><td>metric</td><td>description</td></tr></thead><tbody><tr><td>Timestamp</td><td>The time when the sample was collected</td></tr><tr><td>Microservice ID</td><td>The unique identifier of the microservice</td></tr><tr><td>CPU Utilization</td><td>Normalized CPU sampled at that time</td></tr></tbody></table>

## 4.2 Problem Definition

We consider a k-dimensional time-series with:

$$X_t = [x_t^i, x_t^{ii}, x_t^{iii}, \dots, x_t^k] \qquad (1)$$

In the proposed approach, focused on CPU utilization, it includes the CPU utilization value ($x_t^i$). The size k of the input is 1.

The model uses historical observations ($X_{t-n+1}, \dots, X_t$) to estimate CPU utilization $X_{t+1}$ at one or more next points in time $t+1$, $t+2$ and $t+3$. Therefore,

$$X_{t+1}, X_{t+2}, X_{t+3} = f(X_{t-n+1}, \dots, X_t) \qquad (2)$$

where f is the relationship between the historical data and the predicted CPU utilization. It represents the models that we propose in the following. The time t represents the timestamp recorded

when the sample is read. This can vary according to the need of the application. According to equation 1 and considering $k=1$, the equation 2 becomes:

$$[x_{t+1}^i], [x_{t+2}^i], [x_{t+3}^i] = f([x_{t-n+1}^i], \dots, [x_t^i]) \qquad (3)$$

The choice of predicting a few step in the future is two-fold. On the one hand, this is explained in [21]. Indeed, this avoids accumulation of errors in the long-term forecast if it is present. Our choice minimizes errors and guarantees the reliability of the forecasted results. On the other hand, considering a $t=300$s, it is enough time to be informed about the future behavior of the infrastructure resource, plan and act the migration from the source to a target destination [13]. The choice is therefore confirmed by the literature.

## 4.3 Preprocessing

Both datasets contain cloud and edge CPU readings sampled, respectively, every 5 and 1 minutes from different VMs and containers. It is therefore required to extract the readings from one single VM or container to create coherent sequences of samples.

We downloaded all 195 files from the Microsoft Azure Trace and 47 files from the Alibaba Cloud Trace. In both cases, we selected the first file and collected a set of 100 ids. These are the candidates for analysis. At this point, the remaining files were opened to extract and add the readings of VMs and containers to Pandas DataFrames. The final dataset for a single VM has a size of approximately 1 MB and 8600 readings, while for a single container it has a size of approximately 110 KB and 1440 readings. The features are the same as explained in Tables 1 and 2.

An additional challenge was represented by the different sampling frequencies of the two datasets. Azure traces are natively sampled every 300 s, while Alibaba traces are sampled every 60 s. To align the two datasets and obtain comparable sequences, the Alibaba container readings were aggregated by computing the mean value over non-overlapping 300 s windows. This resampling step produced a time grid coherent with Azure, but inevitably reduced the number of available points by a factor of five, leading to information loss in the Alibaba traces.

As a result of preprocessing, Figure 1 shows the shape of the CPU utilization samples for a subset of the selected VMs and containers. The values change frequently over time, with relevant spike-peak phenomena. This indicates that the system was experiencing sudden increases in resource demand and the running applications had irregular behavior. The dataset shows different trends, pointing out the variability of user behavior with the deployed services. The dataset now represents a good fit for testing the proposed approach, covering the case when the services stress the VMs.

All data were normalized to the range [0, 1] using the min-max normalization formula as follows:

$$X_{\text{norm}} = \frac{X - X_{\text{min}}}{X_{\text{max}} - X_{\text{min}}} \qquad (4)$$

$X_{\text{max}}$ and $X_{\text{min}}$ are, respectively, the maximum and minimum values considering all datasets.

Normalized data were used for both training and evaluation in order to assess the quality of the learning process. The dataset was partitioned into 70% for training, 15% for validation, and the

<footnote n="2">²https://github.com/alibaba/clusterdata</footnote>
<figure><img src="image_3.png" /><figcaption>Figure 1: VM Samples from the Microsoft Azure Trace Dataset and Alibaba Cloud Trace Dataset</figcaption></figure>

remaining 15% for testing. The splitting procedure is defined as follows:

$$t = \lfloor r_{\text{train}}N \rfloor, \quad v = \lfloor r_{\text{val}}N \rfloor, \quad g = \max(1, s + p - 1),$$

$$\text{Train} = [0, t),$$

$$\text{Validation} = [\min(t+g, N), \min(t+g+v, N)),$$

$$\text{Test} = [\min(t+g+v, N), N).$$

where N is the total sequence length, r$_{train}$ and r$_{val}$ are the training and validation ratios, s is the input sequence length, p is the prediction length, g is the applied gap, ⌊x⌋ denotes the floor operator, t and v are the computed training and validation lengths. A gap g was introduced in order to avoid information leakage from future windows that could bias the training process.

## 4.4 Predictive Model

To address the problem of multi-step forecasting of CPU utilization, we designed a Temporal Transformer architecture, illustrated in Figure 2. The model is specifically tailored to capture both short-term fluctuations and long-range dependencies in workload dynamics, such as variations in average CPU consumption.

The model input is a four-dimensional tensor:

$$\text{Input} \in \mathbb{R}^{B \times T \times N \times F} \qquad (5)$$

where B denotes the batch size, T the length of the input sequence (e.g., the lookback window), N the number of VMs, and F the number of features, that is the average CPU utilization. In this work, we set N = 100. Each univariate CPU usage value is first projected into a d$_{model}$ = 192-dimensional latent space through a linear transformation, producing richer internal representations:

$$h \in \mathbb{R}^{B \times T \times N \times d_{\text{model}}}$$

<figure><img src="image_4.png" /><figcaption>Figure 2: Architecture of the proposed temporal transformer for CPU utilization forecasting.</figcaption></figure>

This projection is initialized with Xavier uniform initialization to stabilize training and facilitate convergence.

Two complementary embeddings are incorporated into the model: i) the VM Embedding, which assigns a unique vector to each VM, enabling the model to disambiguate between different machines and capture VM-specific behavior; and ii) the Time Embedding, which encodes the temporal position of each timestep, thereby compensating for the permutation invariance of the attention mechanism.

The core of the model consists of three stacked temporal transformer layers. Each layer contains:

$$\text{Attention}(Q, K, V) = \text{softmax}\left(\frac{QK^T}{\sqrt{d_{\text{head}}}} + M\right) V$$

where M is a causal mask that ensures that predictions at time t cannot access future information. Each attention block employs eight heads, with d$_{head}$ = d$_{model}$/8 = 24. The attention mechanism is followed by a position-wise feed-forward network with hidden dimension d$_{ff}$ = 256 and a Gaussian Error Linear Unit (GELU) activation, which expands and contracts the feature space to capture
complex non-linear patterns. Residual connections, dropout (p = 0.15), and layer normalization are applied throughout the network to stabilize optimization and reduce overfitting.

Finally, a decoder maps the transformer outputs back to the original feature space, yielding predictions with the following shape:

$$ \text{Output} \in \mathbb{R}^{B \times \text{pred\_len} \times N \times 1} \qquad (6) $$

which corresponds to the forecasted CPU utilization for each VM at each future timestep.

Overall, the model comprises approximately 8.0 × 10⁵ trainable parameters, corresponding to a memory footprint of about 3.2 MB in single-precision representation. This balance between expressiveness and efficiency makes the architecture suitable not only for cloud applications but also for edge intelligence scenarios.

## 4.5 Training Phase

The training phase, as shown in Figure 3, was conducted separately for each dataset. After splitting the timeseries into training, validation, and test sets with appropriate gaps, as previously discussed, the training set was normalized using the min-max scaler, fitted exclusively on the training data to avoid information leakage. The same transformation was then applied to the validation and test sets.

<figure><img src="image_5.png" /><figcaption>Figure 3: Pipeline for transfer learning between datasets. The model is first pretrained on the Microsoft Azure dataset (left branch), including splitting, scaling, and training. The resulting pretrained model is then fine-tuned on the Alibaba dataset (right branch) using its own splits and scaling, followed by evaluation on both Alibaba and Microsoft Azure testsets.</figcaption></figure>

The model was trained on the Microsoft Azure dataset using the PyTorch framework with early stopping based on the validation loss

to prevent overfitting. The optimization process relied on the Adam optimizer with a learning rate tuned to 0.001, and the loss function considered was the Mean Squared Error (MSE). Each training run involved multiple epochs until convergence, which often happened around 256 epochs, with the validation set continuously monitored to ensure generalization.

During training, the model learned temporal dependencies from the input sequences through its multi-head attention mechanism, as well as representations of different VMs via embedding layers. Longer lookback windows provided more historical context, while multi-step forecasting required the model to capture patterns that extended across several future timesteps.

After training on the Microsoft Azure dataset, the model weights were saved and subsequently reused in the transfer learning phase, where fine-tuning was performed on the Alibaba dataset. This two-step process allowed the model to take advantage of the knowledge acquired from Azure while adapting to the characteristics of Alibaba.

## 4.6 Post-Processing

After training and evaluating the models, the output must be de-normalized using the formula:

$$ X = X_{\text{norm}} \cdot (X_{\text{max}} - X_{\text{min}}) + X_{\text{min}} \qquad (7) $$

With $X_{max}$ and $X_{min}$ the same values used during data normalization.

# 5 Results and Discussion

## 5.1 Environmental Setup

The experiments were executed on a dedicated VM provisioned with 8 GB of RAM and powered by an AMD Ryzen 9 7950X processor with 16 cores, a high-performance CPU designed for parallel workloads. For accelerated training and inference, the system was equipped with an NVIDIA RTX 4090 Super GPU, featuring 24 GB of dedicated VRAM, which provided sufficient memory to accommodate large Transformer models and multi-step forecasting experiments without requiring aggressive batch size reductions or gradient checkpointing trade-offs.

The software stack was centered on Python 3 as the main programming language, with the PyTorch deep learning framework serving as the backbone for model implementation. PyTorch v2.6.0 was chosen for its efficient memory management, flexibility for custom architecture design, and strong integration with the CUDA framework, which enabled direct exploitation of the GPU's parallel processing capabilities. Other modules used are Numpy v2.3.1 for the array management, Pandas v2.3.2 for the .csv files processing, Matplotlib v3.10.0 for data visualization and Scikit-learn v1.7.2 and Scipy v1.16.1 for metrics calculations, such as $R^2$, MSE, MAE, RMSE and confidence intervals in inference time calculations.

## 5.2 Metrics

The quality of the methodology is expressed in terms of:

* $R^2$: it measures the proportion of the variance in the dependent variable that is predictable from the independent variables, providing an indication of the model's accuracy
and is defined as:

$$R^2 = 1 - \frac{\sum_{i=1}^{n} (y_i - \hat{y}_i)^2}{\sum_{i=1}^{n} (y_i - \bar{y})^2} \qquad (8)$$

where $y_i$ represents the real value, $\hat{y}$ represents the average of the real values, and $\hat{y}$ the predicted value. The calculated value $R^2$ must lie between 0 and 1: a value close to 1 indicates a higher predictive capacity. A negative value $R^2$ implies that the model is not usable.

* **Mean Squared Error (MSE):** measures the average of squared errors between predicted values and actual values. It emphasizes larger errors more than MAE and is commonly used for regression tasks:

$$MSE = \frac{1}{n} \sum_{i=1}^{n} (y_i - \hat{y}_i)^2 \qquad (9)$$

* **Root Mean Squared Error (RMSE):** measure the standard deviation between predicted values and actual values. It is useful for understanding the absolute error when the errors are squared to prevent positive and negative values from canceling each other out.

$$RMSE = \sqrt{MSE} \qquad (10)$$

* **Mean Absolute Error (MAE):** measures the average of the absolute errors between predicted values and actual values. It is useful for understanding the accuracy of a model's predictions:

$$MAE = \frac{1}{n} \sum_{i=1}^{n} |y_i - \hat{y}_i| \qquad (11)$$

* **Average inference time:** measures the average time in seconds for making an inference.

* **Model size:** measures the size of the model in MB.

## 5.3 Results Discussion

The obtained results underline specific common trends across the different experimental scenarios as follows: i) Azure, ii) Alibaba and iii) Azure after Transfer Learning.

Experiments were setup with 6, 12 and 24 steps as lookback window size. The accuracy metrics in Table 3 show that for all the lookback window sizes considered, the $R^2$ score remains high (above 0.76), with few slight decreases as the forecasting horizon increases. This pattern is expected in multi-step forecasting, where uncertainty naturally grows with longer horizons. Performance on the Alibaba dataset is generally higher than on Azure, suggesting either more regular behavior in the data or better generalization of the model in that setting. Transfer learning from Azure to Alibaba yields results that are comparable, and in some cases slightly better, than direct training on Azure, showing the benefit of reusing learned representations.

The MSE, MAE, and RMSE scores indicate stable behavior in different scenarios. Performance does not deteriorate dramatically when increasing the lookback from 6 to 24 steps, which suggests that the architecture can effectively exploit longer input windows without clear signs of overfitting.

As reported in Table 5, inference times remain low and consistent. Using the Microsoft Azure dataset, predictions take about

0.6–0.8 seconds, with narrow confidence intervals, while using the Alibaba dataset they are much faster than 0.04 seconds, likely due to differences in dataset size or complexity. Transfer learning does not add any noticeable overhead, keeping inference times essentially unchanged. This is an important property for real-time or near real-time applications, where fast response is essential.

The size of the model remains constant at 3.2MB in all lookback windows and forecasting steps as shown in Table 4. This behavior is expected because the underlying architecture does not change with these settings. As a result, the observed differences in performance are entirely due to the data dynamics rather than to variations in the model complexity. The compact and fixed size of the model also makes it suitable for deployment in environments with limited memory resources, such as edge devices.

When it comes to qualitative analysis, referring to the Azure dataset, Figure 4 clearly shows how the predicted curve closely follows the testing values, often matching the shape of the actual timeseries. This qualitative evidence is consistent with the quantitative metrics previously discussed, which confirm that the transfer learning process improved the ability of the model to capture relevant patterns. Such improvements can also be observed in Figures 4a, 4b, and 4c, and are further highlighted in the corresponding transfer learning results reported in Figures 4d, 4e, and 4f.

Regarding the predictions on the Alibaba dataset as shown in Figure 5, the test curves exhibit a good alignment with the actual values, showing only minimal deviations. This result highlights the ability of the Temporal Transformer, once fine-tuned through transfer learning, to generalize effectively across different environments. The predicted series not only follows the overall trend of the ground truth, but also captures local fluctuations and short-term variations, which are typically harder to reproduce. This behavior further confirms the robustness of the approach and supports the conclusions drawn from the quantitative metrics. Qualitative evidence can be clearly observed in Figures 5a, 5b, and 5c, where the predicted trajectories are nearly indistinguishable from the corresponding actual curves.

Table 6 compares the Temporal Transformer trained with transfer learning with other models available in the literature. The first difference is that the other models are trained with one dataset. Unfortunately, this is often neither the Microsoft Azure Trace or the Alibaba Trace dataset. PlanetLab is an old dataset, while the custom dataset is not accessible. The proposed model is comparable in terms of MSE, RMSE and MAE when the lookback window size is 6 and 12 steps long. In the first case, all variants of the proposed model have better results than PSE-RNN, DE-RNN, CMA-ES-RNN and BPTT. The experiment with a window 12 steps long is comparable with the Gradient Boosting, but better results are obtained by the model that predicts 3 steps ahead.

## 6 Conclusion and Future Works

In this paper, we proposed a Temporal Transformer architecture to forecast CPU workloads in the computing continuum, with a focus on leveraging transfer learning from VMs to containers. By pretraining the model on the Microsoft Azure dataset and subsequently fine-tuning it on the Alibaba dataset, we demonstrated that
<table><caption>Table 3: Forecasting metrics by lookback window (Window) and steps ahead. The acronyms PT and FT respectively indicate the pre-trained and the fine-tuned model.</caption><thead><tr><th rowspan="2">Window</th><th rowspan="2">Steps</th><th colspan="3">R<sup>2</sup></th><th colspan="3">MSE</th><th colspan="3">MAE</th><th colspan="3">RMSE</th></tr><tr><th>PT (Azure Test)</th><th>FT (Alibaba Test)</th><th>FT (Azure Test)</th><th>PT (Azure Test)</th><th>FT (Alibaba Test)</th><th>FT (Azure Test)</th><th>PT (Azure Test)</th><th>FT (Alibaba Test)</th><th>FT (Azure Test)</th><th>PT (Azure Test)</th><th>FT (Alibaba Test)</th><th>FT (Azure Test)</th></tr></thead><tbody><tr><td rowspan="3">6</td><td>1</td><td>0.8425</td><td>0.8658</td><td>0.8525</td><td>0.0047</td><td>0.0063</td><td>0.0044</td><td>0.0304</td><td>0.0523</td><td>0.0354</td><td>0.0686</td><td>0.0794</td><td>0.0664</td></tr><tr><td>2</td><td>0.8203</td><td>0.8405</td><td>0.8194</td><td>0.0054</td><td>0.0075</td><td>0.0054</td><td>0.0326</td><td>0.0589</td><td>0.0428</td><td>0.0733</td><td>0.0867</td><td>0.0735</td></tr><tr><td>3</td><td>0.7644</td><td>0.8302</td><td>0.7980</td><td>0.0070</td><td>0.0080</td><td>0.0060</td><td>0.0363</td><td>0.0619</td><td>0.0416</td><td>0.0839</td><td>0.0894</td><td>0.0777</td></tr><tr><td rowspan="3">12</td><td>1</td><td>0.8468</td><td>0.8674</td><td>0.8504</td><td>0.0046</td><td>0.0062</td><td>0.0045</td><td>0.0292</td><td>0.0516</td><td>0.0367</td><td>0.0677</td><td>0.0788</td><td>0.0669</td></tr><tr><td>2</td><td>0.8163</td><td>0.8404</td><td>0.8196</td><td>0.0055</td><td>0.0075</td><td>0.0056</td><td>0.0313</td><td>0.0590</td><td>0.0425</td><td>0.0741</td><td>0.0866</td><td>0.0749</td></tr><tr><td>3</td><td>0.7826</td><td>0.8262</td><td>0.8034</td><td>0.0064</td><td>0.0081</td><td>0.0061</td><td>0.0342</td><td>0.0618</td><td>0.0432</td><td>0.0801</td><td>0.0898</td><td>0.0781</td></tr><tr><td rowspan="3">24</td><td>1</td><td>0.8489</td><td>0.8701</td><td>0.8513</td><td>0.0045</td><td>0.0061</td><td>0.0046</td><td>0.0288</td><td>0.0508</td><td>0.0375</td><td>0.0670</td><td>0.0781</td><td>0.0677</td></tr><tr><td>2</td><td>0.8231</td><td>0.8428</td><td>0.8220</td><td>0.0053</td><td>0.0074</td><td>0.0055</td><td>0.0309</td><td>0.0579</td><td>0.0437</td><td>0.0729</td><td>0.0860</td><td>0.0745</td></tr><tr><td>3</td><td>0.7915</td><td>0.8295</td><td>0.8078</td><td>0.0062</td><td>0.0080</td><td>0.0063</td><td>0.0337</td><td>0.0609</td><td>0.0446</td><td>0.0787</td><td>0.0894</td><td>0.0793</td></tr></tbody></table>

<figure><img src="image_6.png" /><figcaption>(a) Model=VMs (Pre-Trained) | Dataset=Azure | VM 2</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>(b) Model=VMs (Pre-Trained) | Dataset=Azure | VM 56</figcaption></figure>

<figure><img src="image_8.png" /><figcaption>(c) Model=VMs (Pre-Trained) | Dataset=Azure | VM 92</figcaption></figure>

<figure><img src="image_9.png" /><figcaption>(d) Model=VMs+Cont. (Fine-Tuned) | Dataset=Azure | VM 2</figcaption></figure>

<figure><img src="image_10.png" /><figcaption>(e) Model=VMs+Cont. (Fine-Tuned) | Dataset=Azure | VM 56</figcaption></figure>

<figure><img src="image_11.png" /><figcaption>(f) Model=VMs+Cont. (Fine-Tuned) | Dataset=Azure | VM 92</figcaption></figure>

<figure><img src="image_12.png" /><figcaption>(a) Model=VMs+Cont. (Fine-Tuned) | Dataset=Alibaba | Cont. 2</figcaption></figure>

<figure><img src="image_13.png" /><figcaption>(b) Model=VMs+Cont. (Fine-Tuned) | Dataset=Alibaba | Cont. 56</figcaption></figure>

<figure><img src="image_14.png" /><figcaption>(c) Model=VMs+Cont. (Fine-Tuned) | Dataset=Alibaba | Cont. 92</figcaption></figure>

<caption>Figure 5: Comparison of prediction results on the Alibaba dataset on Training-Validation-Test data. Subfigures 5a-5b-5c show the corresponding model trained with transfer learning.</caption>

knowledge can be effectively transferred across different virtualization environments. The model achieved consistently high $R^2$ values (above 0.80) with low MSE, MAE, and RMSE, while preserving a compact size of 3.2 MB and inference times suitable for real-time operation. These results highlight both the accuracy and efficiency

of the approach, making it viable not only for cloud platforms but also for edge intelligence deployments. From a qualitative perspective, the predicted CPU utilization curves closely follow the actual values, capturing both global trends and local fluctuations. This
<table><caption>**Table 4: Model size (MB) by lookback window and steps ahead.**</caption><thead><tr><td>Window</td><td>Steps</td><td>Pre-Trained (Azure)</td><td>Fine-Tuned (Alibaba)</td></tr></thead><tbody><tr><td>6, 12, 24</td><td>1, 2, 3</td><td>3.2</td><td>3.2</td></tr></tbody></table>

confirms the ability of the Temporal Transformer to model workload dynamics and adapt to heterogeneous environments through transfer learning.

Future work will focus on extending the methodology to multivariate forecasting by incorporating additional metrics such as memory and network utilization. Another promising direction is the integration of federated and distributed learning strategies, enabling workload prediction directly at the edge while preserving data privacy. Furthermore, we envision the development of a predictive scheduling framework to manage the deployment of microservices, specifically containers and VMs, in order to assess efficiency in real-world scenarios. Finally, deploying and benchmarking the proposed approach in operational continuum infrastructures will provide further validation of its robustness and practical impact.

## References

[1] Daniel Balouek-Thomert, Ivan Rodero, and Manish Parashar. 2020. Harnessing the Computing Continuum for Urgent Science. ACM SIGMETRICS Performance Evaluation Review 48 (11 2020), 41–46. Issue 2. doi:10.1145/3439602.3439618

[2] Christian Bauer, Narges Mehran, Radu Prodan, and Dragi Kimovski. 2023. Machine Learning Based Resource Utilization Prediction in the Computing Continuum. In 2023 IEEE 28th International Workshop on Computer Aided Modeling and Design of Communication Links and Networks (CAMAD). IEEE, Edinburgh, United Kingdom, 219–224. doi:10.1109/CAMAD59638.2023.10478387

[3] Pete Beckman, Jack Dongarra, Nicola Ferrier, Geoffrey Fox, Terry Moore, Dan Reed, and Micah Beck. 2020. Harnessing the computing continuum for programming our world. Fog Computing: Theory and Practice (2020), 215–230.

[4] Lorenzo Carnevale, Daniel Balouek, Serena Sebbio, Manish Parashar, and Massimo Villari. 2025. Private Distributed Resource Management Data: Predicting CPU Utilization with Bi-LSTM and Federated Learning. In 2025 IEEE 25th International Symposium on Cluster, Cloud and Internet Computing (CCGrid). 266–275. doi:10.1109/CCGRID64434.2025.00048

[5] Eli Cortez, Anand Bonde, Alexandre Muzio, Mark Russinovich, Marcus Fontoura, and Ricardo Bianchini. 2017. Resource Central: Understanding and Predicting Workloads for Improved Resource Management in Large Cloud Platforms. In Proceedings of the 26th Symposium on Operating Systems Principles. ACM, Shanghai China, 153–167. doi:10.1145/3132747.3132772

[6] Mustafa Daraghmeh, Anjali Agarwal, and Yaser Jararweh. 2023. A Multilevel Learning Model for Predicting CPU Utilization in Cloud Data Centers. In 2023 IEEE Intl Conf on Dependable, Autonomic and Secure Computing, Intl Conf on Pervasive Intelligence and Computing, Intl Conf on Cloud and Big Data Computing, Intl Conf on Cyber Science and Technology Congress (DASC/PiCom/CBDCom/CyberSciTech). IEEE, Abu Dhabi, United Arab Emirates, 1016–1023. doi:10.1109/DASC/PiCom/CBDCom/Cy59711.2023.10361305

[7] Martin Duggan, Karl Mason, Jim Duggan, Enda Howley, and Enda Barrett. 2017. Predicting host CPU utilization in cloud computing using recurrent neural networks. In 2017 12th International Conference for Internet Technology and Secured Transactions (ICTST). IEEE, Cambridge, 67–72. doi:10.23919/ICTST.2017.8356348

[8] Schahram Dustdar, Victor Casamayor Pujol, and Praveen Kumar Donta. 2022. On distributed computing continuum systems. IEEE Transactions on Knowledge and Data Engineering 35, 4 (2022), 4092–4105.

[9] Rebeca Estrada, Irving Valeriano, and Xavier Aizaga. 2023. CPU Usage Prediction Model: A Simplified VM Clustering Approach. In Complex, Intelligent and Software Intensive Systems, Leonard Barolli (Ed.). Vol. 176. Springer Nature Switzerland, Cham, 210–221. doi:10.1007/978-3-031-35734-3_21 Series Title: Lecture Notes on Data Engineering and Communications Technologies.

[10] Muhammad Fahimullah, Rohit Gupta, Shohreh Ahvar, and Maria Trocan. 2022. Explaining Predictive Scheduling in Cloud. In Recent Challenges in Intelligent Information and Database Systems, Edward Szczerbicki, Krystian Wojtkiewicz, Sinh Van Nguyen, Marcin Pietranik, and Marek Krótkiewicz (Eds.). Vol. 1716. Springer Nature Singapore, Singapore, 81–91. doi:10.1007/978-981-19-8234-7_7 Series Title: Communications in Computer and Information Science.

[11] Jiechao Gao, Haoyu Wang, and Haiying Shen. 2020. Machine learning based workload prediction in cloud computing. In 2020 29th international conference on computer communications and networks (ICCCN). IEEE, 1–9.

[12] Aroosa Hameed, John Violos, Aris Leivadeas, Nina Santi, Rémy Grünblatt, and Nathalie Mitton. 2022. Toward QoS prediction based on temporal transformers for IoT applications. IEEE Transactions on Network and Service Management 19, 4 (2022), 4010–4027.

[13] Red Hat. 2024. How to create and scale 6,000 virtual machines in 7 hours with Red Hat OpenShift Virtualization. https://cloud.redhat.com/learning/learn: how-create-and-scale-6000-virtual-machines-7-hours-red-hat-openshift-virtualization/resource/resources:virtual-machine-migration-workflows.

[14] Deepak Janardhanan and Enda Barrett. 2017. CPU workload forecasting of machines in data centers using LSTM recurrent neural networks and ARIMA models. In 2017 12th International Conference for Internet Technology and Secured Transactions (ICTST). IEEE, Cambridge, 55–60. doi:10.23919/ICTST.2017.8356346

[15] Mohammad Masdari and Afsane Khoshnevis. 2020. A survey and classification of the workload forecasting methods in cloud computing. Cluster Computing 23, 4 (2020), 2399–2424.

[16] Karl Mason, Martin Duggan, Enda Barrett, Jim Duggan, and Enda Howley. 2018. Predicting host CPU utilization in the cloud using evolutionary neural networks. Future Generation Computer Systems 86 (Sept. 2018), 162–173. doi:10.1016/j.futu re.2018.03.040

[17] Seon-cheol Park and Young-han Kim. 2022. An Improvement of Multi-Cluster Stability of Private Cloud Systems through LSTM-Based CPU Usage Prediction. The Journal of Korean Institute of Communications and Information Sciences 47, 8 (Aug. 2022), 1081–1095. doi:10.7840/kics.2022.47.8.1081

[18] Eva Patel and Dharmender Singh Kushwaha. 2023. An Integrated Deep Learning Prediction Approach for Efficient Modelling of Host Load Patterns in Cloud Computing. Journal of Grid Computing 21, 1 (March 2023), 5. doi:10.1007/s10723- 022-09639-6

[19] Khanh Nguyen Quoc, Van Tong, Cuong Dao, Tuyen Ngoc Le, and Duc Tran. 2024. Boosted regression for predicting CPU utilization in the cloud with periodicity. The Journal of Supercomputing (Aug. 2024). doi:10.1007/s11227-024-06451-9

[20] Rafael Roque, Rodrigo Paula, Daniel Bezerra, Iago Rodrigues, Robson Lins, Michelle Tenca, and Gabriel Almeida. 2023. A PSO-SVR Based Resource Allocation Approach for Network Function Virtualization. In 2023 IEEE Latin American Conference on Computational Intelligence (LA-CCI). IEEE, Recife-Pe, Brazil, 1–6. doi:10.1109/LA-CCI58595.2023.10409340

[21] Chaoxue Wang and Zhenbang Wang. 2024. Isolated Forest-Based Prediction of Container Resource Load Extremes. Applied Sciences 14, 7 (March 2024), 2911. doi:10.3390/app14072911

[22] Zihao Wang, Huan Zhao, and Rong Dai. 2023. Research on Stacked Model for Cluster CPU Utilization Prediction Based on Butterworth Filter. In 2023 4th International Conference on Big Data, Artificial Intelligence and Internet of Things Engineering (ICBAIE). IEEE, Hangzhou, China, 191–197. doi:10.1109/ICBAIE5971 4.2023.10281293

[23] Qingsong Wen, Tian Zhou, Chaoli Zhang, Weiqi Chen, Ziqing Ma, Junchi Yan, and Liang Sun. 2023. Transformers in Time Series: A Survey. arXiv:2202.07125 [cs.LG] https://arxiv.org/abs/2202.07125

[24] Xiantao Zhang, Xiao Zheng, Zhi Wang, Hang Yang, Yibin Shen, and Xin Long. 2020. High-density Multi-tenant Bare-metal Cloud. In Proceedings of the Twenty-Fifth International Conference on Architectural Support for Programming Languages and Operating Systems (Lausanne, Switzerland) (ASPLOS '20). Association for Computing Machinery, New York, NY, USA, 483–495. doi:10.1145/3373376.3378 507
<table><caption>Table 5: Average inference time (seconds) with 95% confidence interval by lookback window (Window) and steps ahead.</caption><thead><tr><th>Window</th><th>Steps</th><th>Pre-Trained (Azure Test)</th><th>Fine-Tuned (Alibaba Test)</th><th>Fine-Tuned (Azure Test)</th></tr></thead><tbody><tr><td rowspan="3">6</td><td>1</td><td>0.5993 [0.5961-0.6025]</td><td>0.0323 [0.0322-0.0323]</td><td>0.5967 [0.5947-0.5988]</td></tr><tr><td>2</td><td>0.6760 [0.6741-0.6780]</td><td>0.0379 [0.0378-0.0379]</td><td>0.6835 [0.6779-0.6891]</td></tr><tr><td>3</td><td>0.7866 [0.7861-0.7870]</td><td>0.0428 [0.0427-0.0428]</td><td>0.7772 [0.7723-0.7822]</td></tr><tr><td rowspan="3">12</td><td>1</td><td>0.6175 [0.6170-0.6180]</td><td>0.0320 [0.0319-0.0320]</td><td>0.6187 [0.6149-0.6226]</td></tr><tr><td>2</td><td>0.7065 [0.7002-0.7127]</td><td>0.0371 [0.0368-0.0373]</td><td>0.7121 [0.7088-0.7153]</td></tr><tr><td>3</td><td>0.7871 [0.7813-0.7928]</td><td>0.0405 [0.0401-0.0408]</td><td>0.7963 [0.7957-0.7969]</td></tr><tr><td rowspan="3">24</td><td>1</td><td>0.6235 [0.6232-0.6238]</td><td>0.0317 [0.0317-0.0318]</td><td>0.6128 [0.6125-0.6131]</td></tr><tr><td>2</td><td>0.7240 [0.7231-0.7250]</td><td>0.0344 [0.0344-0.0344]</td><td>0.7114 [0.7078-0.7149]</td></tr><tr><td>3</td><td>0.7923 [0.7916-0.7930]</td><td>0.0383 [0.0383-0.0384]</td><td>0.7926 [0.7920-0.7931]</td></tr></tbody></table>

<table><caption>Table 6: Experimental results for the testset. Classical models are evaluated on 1-step forecasting with different input windows. The proposed Temporal Transformer is evaluated after transfer learning on the Alibaba Cloud Trace Dataset and tested on the Microsoft Azure Trace Dataset (Fine-Tuned (Azure Test)) on multiple prediction horizons (1, 2, and 3 steps ahead) with window sizes of 6, 12, and 24. All other models predict 1 step. All datasets are normalized with min-max scaling between 0 and 1.</caption><thead><tr><th>model</th><th>metric</th><th>dataset</th><th>Window=6</th><th>Window=12</th><th>Window=24</th></tr></thead><tbody><tr><td rowspan="2">PSE-RNN [16]</td><td>MSE</td><td>PlanetLab</td><td>0.0483</td><td>-</td><td>-</td></tr><tr><td>MAE</td><td>PlanetLab</td><td>0.1564</td><td>-</td><td>-</td></tr><tr><td rowspan="2">DE-RNN [16]</td><td>MSE</td><td>PlanetLab</td><td>0.0465</td><td>-</td><td>-</td></tr><tr><td>MAE</td><td>PlanetLab</td><td>0.1495</td><td>-</td><td>-</td></tr><tr><td rowspan="2">CMA-ES-RNN [16]</td><td>MSE</td><td>PlanetLab</td><td>0.0468</td><td>-</td><td>-</td></tr><tr><td>MAE</td><td>PlanetLab</td><td>0.1498</td><td>-</td><td>-</td></tr><tr><td rowspan="2">BPTT [7]</td><td>MSE</td><td>PlanetLab</td><td>0.038</td><td>-</td><td>-</td></tr><tr><td>MAE</td><td>PlanetLab</td><td>0.153</td><td>-</td><td>-</td></tr><tr><td rowspan="12">Temporal Transformer</td><td>R<sup>2</sup> (1 step)</td><td>Fine-Tuned (Azure Test)</td><td>0.8525</td><td>0.8504</td><td>0.8513</td></tr><tr><td>MSE (1 step)</td><td>Fine-Tuned (Azure Test)</td><td>0.0044</td><td>0.0045</td><td>0.0046</td></tr><tr><td>RMSE (1 step)</td><td>Fine-Tuned (Azure Test)</td><td>0.0664</td><td>0.0669</td><td>0.0677</td></tr><tr><td>MAE (1 step)</td><td>Fine-Tuned (Azure Test)</td><td>0.0354</td><td>0.0367</td><td>0.0375</td></tr><tr><td>R<sup>2</sup> (2 steps)</td><td>Fine-Tuned (Azure Test)</td><td>0.8194</td><td>0.8196</td><td>0.8220</td></tr><tr><td>MSE (2 steps)</td><td>Fine-Tuned (Azure Test)</td><td>0.0054</td><td>0.0056</td><td>0.0055</td></tr><tr><td>RMSE (2 steps)</td><td>Fine-Tuned (Azure Test)</td><td>0.0735</td><td>0.0749</td><td>0.0745</td></tr><tr><td>MAE (2 steps)</td><td>Fine-Tuned (Azure Test)</td><td>0.0428</td><td>0.0425</td><td>0.0437</td></tr><tr><td>R<sup>2</sup> (3 steps)</td><td>Fine-Tuned (Azure Test)</td><td>0.7980</td><td>0.8034</td><td>0.8078</td></tr><tr><td>MSE (3 steps)</td><td>Fine-Tuned (Azure Test)</td><td>0.0060</td><td>0.0061</td><td>0.0063</td></tr><tr><td>RMSE (3 steps)</td><td>Fine-Tuned (Azure Test)</td><td>0.0777</td><td>0.0781</td><td>0.0793</td></tr><tr><td>MAE (3 steps)</td><td>Fine-Tuned (Azure Test)</td><td>0.0416</td><td>0.0432</td><td>0.0446</td></tr><tr><td rowspan="3">Gradient Boosting [6]</td><td>MSE</td><td>Custom</td><td>-</td><td>0.0058</td><td>-</td></tr><tr><td>RMSE</td><td>Custom</td><td>-</td><td>0.0758</td><td>-</td></tr><tr><td>MAE</td><td>Custom</td><td>-</td><td>0.0509</td><td>-</td></tr></tbody></table>