# Using machine learning for timing analysis: where do we stand?

Abderaouf Nassim Amalou, Isabelle Puaut

<figure><img src="image_1.png" /></figure>

## To cite this version:

Abderaouf Nassim Amalou, Isabelle Puaut. Using machine learning for timing analysis: where do we stand?. Real-Time Systems, 2025, 61 (2), pp.300-305. 10.1007/s11241-025-09442-y. hal-05092607

HAL Id: hal-05092607

https://hal.science/hal-05092607v1

Submitted on 2 Jun 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Using machine learning for timing analysis: where do we stand?

Abderaouf Nassim Amalou<fnref n="1" />,<fnref n="2" /> · Isabelle Puaut<fnref n="1" />,<fnref n="2" />

Accepted: 9 May 2025

© The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature
2025

## Abstract

This paper presents our experience using Machine Learning (ML) to predict the Worst-Case Execution Time (WCET) of small code snippets on single-core platforms. We provide a concise overview of our work, highlight key observations made throughout our study, and advocate for further exploration of this topic.

**Keywords** Worst-case execution time (WCET) estimation · Machine learning (ML)

## 1 Introduction

Demonstrating that tasks in real-time systems meet their deadlines requires estimating the Worst-Case Execution Time (WCET) of each task. WCET estimates must be both *safe* (greater than any possible execution time) and as *accurate* as possible (as close as possible to the actual WCET, which is generally unknown). Static WCET estimation techniques Wilhelm et al. (2008) require detailed knowledge of the micro-architecture of the processor under study. Designing safe yet precise timing models of processors is not only time-consuming but also increasingly difficult due to intellectual property constraints and the growing complexity of micro-architectures.

In the last few years, we have explored the use of Machine Learning (ML) techniques to predict the WCET of small code snippets (basic blocks, BB) for use in WCET estimation techniques Amalou (2023). The motivations for using ML to estimate the WCET of BBs are the following:

<footnote>☒ Abderaouf Nassim Amalou
amalou-a@univ-nantes.fr</footnote>

<footnote>☒ Isabelle Puaut
Isabelle.Puaut@irisa.fr</footnote>

<footnote n="1">¹ Nantes Université, École Centrale Nantes, CNRS, LS2N, CNRS UMR 6004, F-44000 Nantes, France</footnote>

<footnote n="2">² Univ Rennes, Inria, CNRS, IRISA, Rennes, France</footnote>
(i) ML-based timing estimation does not require the design of a hardware abstraction layer, as is the case with static techniques. Moreover, the time-consuming training phase is performed only once per architecture.

(ii) Although the training phase may be time-consuming, prediction is fast and does not require measurements for each BB.

(iii) ML-based techniques can process large volumes of execution scenarios for BBs and identify patterns during the training phase, enabling accurate predictions.

This paper makes an honest attempt to evaluate the usefulness of ML for WCET estimation across a wide spectrum of processor architecture complexities, datasets, and ML techniques. The rest of this paper provides a concise overview of our research (Section 2), reports key observations based on our experience (Section 3), and suggests directions for future work (Section 4).

## 2 Overview of research

Our work estimates an upper bound on the WCET of basic blocks (BBs) using *supervised* machine learning techniques in two classical steps:

* **Training:** an ML model is trained using a dataset of pairs (BB, WCET).

* **Prediction:** the WCET of BBs from the analyzed program is estimated and then integrated into a WCET analysis tool to derive program-level WCET estimates.

Our approach is *hybrid*, in the sense that it combines static analysis and learning-based prediction. It inherits the structural safety guarantees of classical static WCET estimation techniques by identifying the longest execution path in the program's control flow graph via Implicit Path Enumeration Technique (IPET) Wilhelm et al. (2008). It relies on ML models to predict the timing behavior of BBs instead of using hardware models or exhaustive measurements. We have explored a broad spectrum of variations in processors, datasets, measurement processes, and ML models:

* **Processors:** we have studied a range of processors, from very simple ones with no cache and limited instruction level parallelism (e.g., TI MSP430, Cortex-M4) Reymond et al. (2024); Amalou et al. (2023) to more complex processors with deeper instruction pipelines and a memory hierarchy (e.g., Cortex-M7, Cortex-A53) Amalou et al. (2021, 2023).

* **Datasets and features:** different types of datasets have been used, including synthetic code and real code from benchmarks. Various classes of features have also been explored for training, ranging from simple static features (proportions of instructions per BB) Amalou et al. (2021) to more complex representations (assembly code representing the sequences of instructions with operand information, including addresses) Amalou et al. (2023).

* **Measurement process:** measurements can be obtained using non-intrusive methods (JTAG probes) or more intrusive software instrumentation. Beyond

<figure><img src="image_3.png" /></figure>
how timing is measured, a key challenge is ensuring that measurements are taken
in the worst-case execution scenario. We explored two approaches: (i) introduc-
ing new code for each isolated basic block to create worst-case conditions (code
padding to maximize instruction cache misses, cache pollution to maximize data
cache misses Amalou et al. (2021); Reymond et al. (2024)); (ii) measuring a
basic block's execution time by replaying the trace of preceding BBs to obtain
less pessimistic WCET values for training, yet in the worst scenario Amalou
et al. (2023).

* ML techniques. We have experimented with a wide range of ML techniques:

- Classic models available in open-source libraries such as Scikit-learn Amalou et al. (2021); Reymond et al. (2024): our exploration started with simple tabular-based models such as linear regression, decision trees, and multi-layer perceptrons, which rely on manually crafted features extracted from basic blocks.

- Advanced methods used in Natural Language Processing (NLP) Amalou et al. (2023): we transitioned to sequence-aware, textual-based approaches that treat machine instructions as language-like tokens. These included Long-Short Term Model (LSTM) Amalou et al. (2022), followed by self-attention-based architectures (Transformer), and finally to Transformer-XL.

**3 Findings**

Our experience with ML for WCET estimation provided key insights, notably that *all estimates exceeded the maximum observed execution time*. Additional findings are outlined below. The model accuracy in the following is measured using Mean Percentage Error (MPE), where lower values indicate better performance: $MPE = \frac{1}{n} \sum_{i=1}^{n} \frac{\hat{y}_i - y_i}{y_i} \times 100$, where $y_i$ is the actual value, $\hat{y}_i$ is the predicted value, and *n* is the number of observations (number of BBs from the TACLe benchmarks).

**3.1 ML achieves high accuracy on simple processors**

On simple microcontrollers like the TI MSP430 (2-stage pipeline, no cache) and Cortex-M4 (3-stage pipeline, no cache), ML models demonstrate high accuracy. On the MSP430, a Multi-Layer Perceptron (MLP) achieves a MPE of 8%, outperforming linear regression (57%) and gradient boosting (42%). Similarly, on the Cortex-M4, the MLP remains the most precise (44%) among classical ML models. Prediction accuracy improves further with sequential NLP-based models. On the Cortex-M4, a Transformer-based approach achieves a MPE of 24% Amalou et al. (2023), outperforming MLP. Since our models are trained on actual execution measurements, they naturally reflect the observed behavior of the platform. Although we did not compare our WCET estimates to those produced by static analysis techniques, we believe that our approach yields less pessimistic results.
## 3.2 Predictions get more pessimistic for complex architectures

Our experiments show that as processor complexity increases, ML models tend to overestimate WCET. This is evident on the Cortex-M7 (6-stage pipeline, L1 cache), where a Transformer-based approach Amalou et al. (2023) sees its MPE increase from 24% (for a Cortex-M4) to 102% (for a Cortex-M7). A similar trend is observed for MLP, with an MPE increasing from 44% to 130% when changing from a Cortex-M4 to a Cortex M7. This increased pessimism suggests that ML models struggle to capture long-range timing dependencies in processors, and also struggle to predict cache effects under a random replacement policy.

### 3.2.1 Average-case execution time is easier to predict than WCET

Predicting WCET differs from predicting Average-Case Execution Time (ACET) because the objective is not just accuracy, but also ensuring an upper bound on execution time. Standard loss functions like Mean Squared Error (MSE) minimize the prediction error, leading to good ACET estimates with an MPE of 11% with Transformers on the Cortex-M7. However, for WCET, MSE does not enforce a systematic overapproximation, which is necessary for safety Amalou (2023). To address this, we replaced MSE with a Root Mean Squared Logarithmic Error loss, which penalizes underestimations more heavily Amalou et al. (2023). This adjustment, while enforcing safety, increases the pessimism of the estimates, resulting in a MPE of 126% with Transformers on the Cortex-M7.

### 3.2.2 Feature engineering significantly impacts model performance

Selecting the right input features is crucial for ML-based WCET prediction, as it directly affects accuracy. On the Cortex-M7, using only basic static features (e.g., instruction type proportions) results in high errors, with MLP reaching a 311% MPE. Incorporating details such as access types (e.g., direct vs. indirect memory accesses) lowers this to a 181% MPE. Transformers further improve accuracy, as they better capture instruction dependencies, pipeline effects, branching, and some cache behaviors. For instance, Transformers reduce the MPE to 102%. However, encoding memory addresses and operand values remains a challenge: deep learning models struggle to learn precise timing variations from long code sequences, a limitation often linked to the *gradient vanishing problem* or *limited memory capacity*.

### 3.2.3 Variable timing instructions introduce large errors

In real code from benchmarks, the occurrence of floating-point instructions within a basic block was rare, and their execution time varies significantly due to data-dependent latency for floating-point operations. This combination leads to substantial prediction errors as the model struggles to generalize from limited timing information and variable latencies. On the Cortex-M7, removing floating-point instructions reduces the MPE by nearly 200%, highlighting their impact. Addressing this issue requires more detailed modeling or explicit handling of these rare
yet influential timing outliers. This finding is not specific to ML-based approaches. Classical static timing analysis techniques also face difficulties when dealing with instructions with data-dependent timing, as the precise timing depends on the input data and may not be captured by generic hardware models. In both cases, over-approximation or conservative assumptions are needed, which can lead to increased pessimism.

# 4 Open research directions

ML-based WCET estimation, especially with transformer-based solutions, has proven to be effective. Future research should aim to extend these methods to *multicore systems*, where contention delays from accessing shared resources (caches, interconnects, memory) must also be predicted. Additionally, enhancing *explainability* is crucial for trust and debugging. EXplainable AI (XAI) techniques can help identify key input features influencing WCET predictions, but more research is needed to tailor explanations to the constraints of real-time systems. Finally, *certification* remains a critical challenge, requiring methodologies that ensure safety, reproducibility, and robustness. A promising approach is to integrate confidence measures to limit predictions in situations when they are shown to be reliable, paving the way for ML-based WCET estimation in safety-critical applications.

**Author contributions** The authors contributed equally to this work.

**Data availability** No datasets were generated or analysed during the current study.

**Declarations**

**Competing interests** The authors declare no competing interests.

# References

Wilhelm R, Engblom J, Ermedahl A, Holsti N, Thesing S, Whalley D, Bernat G, Ferdinand C, Heckmann R, Mitra T, Mueller F, Puaut I, Puschner P, Staschulat J, Stenström P (2008) The worst-case execution-time problem–overview of methods and survey of tools. ACM Trans. Embed. Comput. Syst. 7(3)

Amalou AN (2023) Machine learning for timing estimation. PhD thesis, University of Rennes 1, France. https://tel.archives-ouvertes.fr/tel-04406029

Reymond H, Amalou AN, Puaut I (2024) Wortex: Worst-case execution time and energy estimation in low-power microprocessors using explainable ml. In: 22nd Int. Workshop on Worst-Case Execution Time Analysis

Amalou AN, Fromont E, Puaut I (2023) Cawet: Context-aware worst-case execution time estimation using transformers. In: 35th Euromicro Conf. on Real-Time Systems

Amalou AN, Puaut I, Muller G (2021) We-hml: hybrid wcet estimation using machine learning for architectures with caches. In: IEEE 27th Int. Conf. on Embedded and Real-Time Computing Systems and Applications

Amalou AN, Fromont É, Puaut I (2022) CATREEN: context-aware code timing estimation with stacked recurrent networks. In: Reformat, M.Z., Zhang, D., Bourbakis, N.G. (eds.) 34th IEEE Int. Conf. on
Tools with Artificial Intelligence, ICTAI 2022, Macao, China, October 31 - November 2, 2022, pp. 571–576. IEEE, ???

**Publisher's Note** Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

<figure><img src="image_4.png" /></figure>

**Abderaouf Nassim Amalou** received his Master's degree in Embedded Systems at Université Toulouse III - Paul Sabatier, France, followed by a Ph.D. in Computer Science at Université de Rennes. He is currently an Assistant Professor at Nantes Université, where he conducts research in real-time systems, and machine learning for performance analysis.

<figure><img src="image_5.png" /></figure>

**Isabelle Puaut** received a Ph.D. degree in computer science from the University of Rennes in 1993. After 10 years as an assistant professor at INSA (Institut National des Sciences Appliquées, engineering school), she is since 2004 a professor in computer science at the university of Rennes. Her research interests include real-time systems, computer architecture, programming languages, and operating systems. Her current research focuses on timing analysis of real-time software (worst case execution times estimation), more precisely on the impact of hardware on worst-case performance. She has published more than 130 papers in journal publications or peer-reviewed conference papers.