<figure><img src="image_1.png" /></figure>

HAL
open science

# Can We Spot Energy Regressions Using Developers' Tests? An Industrial Replication

Louay Khrouf, Anas Shatnawi, Romain Rouvoy

► To cite this version:

Louay Khrouf, Anas Shatnawi, Romain Rouvoy. Can We Spot Energy Regressions Using Developers' Tests? An Industrial Replication. ICPC 2026 - International Conference on Program Comprehension, Apr 2026, Rio de Janeiro, Brazil. 10.1145/3794763.3798169. hal-05527295

HAL Id: hal-05527295

https://hal.science/hal-05527295v1

Submitted on 25 Feb 2026

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
# Can We Spot Energy Regressions Using Developers' Tests?
## An Industrial Replication

Louay Khrouf

Berger-Levrault / Inria / Univ. Lille /
CNRS - UMR CRISTAL 9189

Lille, France

Anas Shatnawi

Berger-Levrault
Montpellier, France

Romain Rouvoy

Univ. Lille / Inria /
CNRS - UMR CRISTAL 9189

Lille, France

## Abstract

Producing energy-efficient software is gaining increasing attention in both industry and research communities. In this context, the company Berger-Levrault, an international software publisher, aims to better understand and monitor the energy behaviour of its software systems to support the development of more energy-efficient products. However, achieving this objective remains challenging because developers lack feedback mechanisms that show how their implementation and testing choices affect the software system's energy consumption.

To address this limitation, the literature highlights a notable study that could serve as a foundation for Berger-Levrault to enhance developers' energy awareness. This study investigated whether developers' test suites can detect energy regressions in *Continuous Integration* (CI) environments. Our goal is to evaluate if its findings hold in a real-world setting and to provide further evidence on the feasibility and reliability of detecting energy regressions using developers' test suites in practical CI workflows. The results confirm that developers' tests can reliably identify energy regressions while maintaining stable performance and energy metrics. Finally, the approach enabled the classification of changes across commits and the manual validation of detected regressions, reinforcing its feasibility and reliability in an industrial context.

## CCS Concepts

* Software and its engineering → Software creation and management;

## Keywords

Energy consumption, Energy regression, Continuous integration, Regression detection, Energy-aware software engineering

## ACM Reference Format:

Louay Khrouf, Anas Shatnawi, and Romain Rouvoy. 2026. Can We Spot Energy Regressions Using Developers' Tests? An Industrial Replication. In *34th IEEE/ACM International Conference on Program Comprehension (ICPC'26), April 12-13, 2026, Rio de Janeiro, Brazil*. ACM, New York, NY, USA, 9 pages. https://doi.org/10.1145/3794763.3798169

## 1 Introduction

With the growing reliance on software systems and the ongoing digitization of services, it has become crucial to understand the energy cost associated with these systems. Recent results from the Shift Project [8, 9] indicate that the energy footprint of digital technologies increased by 67% between 2019 and 2022. In 2022, this footprint accounted for approximately 10% of global electricity consumption.

With global digital energy consumption rising by an estimated 6% per year [9], addressing the energy behaviour of software systems has become an urgent priority for industry to develop more energy-efficient software. This priority also applies to Berger-Levrault, a large industrial software publisher maintaining several actively evolving systems, where this study was conducted. As part of its ecological commitment and as an actor in the French public sector, Berger-Levrault must align with French and European regulations [18] requiring more energy-efficient digital services. To achieve this objective, the company aims to integrate energy-efficient practices into its daily development process by acquiring a deeper understanding of the energy consumption of its software systems at the source-code level. Although Berger-Levrault relies on *Continuous Integration* (CI) pipelines to automate the building and testing of new functionalities, these pipelines provide no visibility into the energy impact of code changes. As a result, developers lack the necessary feedback to assess the energy consequences of their modifications and to guide the production of more energy-efficient code. To address this challenge, Berger-Levrault seeks to develop approaches that can identify energy hotspots at the source-code level. Such approaches would support the adoption of energy-efficient development practices and provide CI-level visibility into the energy impact introduced by new functionalities or code modifications.

Providing such approaches is a key concern for practitioners. Consequently, the literature includes multiple studies that propose methodologies to move toward greener software products and to provide developers with deeper insights into energy-aware development practices.

One line of research focuses on detecting energy regressions through unit-test executions, proposing an approach called *Energy Regression Testing* (ERT) [4]. Other works examine how specific factors influence energy usage, such as source-code structure [13], testing-related tasks [5], or design patterns [10]. Additional studies seek to identify energy-intensive code regions using spectrum-based leak localization [16], while others introduce source-code-level monitoring tools to support energy analysis and optimization [15]. Finally, recent work has analysed energy-usage patterns

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org.

ICPC'26, Rio de Janeiro, Brazil

© 2026 Copyright held by the owner/author(s). Publication rights licensed to ACM.
ACM ISBN 979-8-4007-2482-4/2026/04
https://doi.org/10.1145/3794763.3798169
in CI workflows [5]. While these approaches provide valuable in-
sights into software energy behaviour, most have been applied un-
der controlled conditions or on open-source systems that do not
reflect the more complex, heterogeneous environments of real in-
dustrial settings. As a result, it remains unclear whether the find-
ings of these studies hold in large-scale industrial settings such as
those at Berger-Levrault. This gap motivates the need to evaluate
these methodologies in a real industrial context to determine their
feasibility and reliability in such settings. In addition, prior work
has highlighted practical challenges in industrial energy measure-
ment, including hardware variability, environmental noise, and dif-
ficulties in obtaining stable and reproducible measurements [11].
Among these studies, a particularly relevant one that aligns with
the approach Berger-Levrault aims to adopt is the work of Dan-
glot et al. on *Energy Regression Testing* (ERT) [4]. ERT leverages
developers’ unit tests to detect energy regressions introduced by
code changes. However, this approach has been validated only on
small-scale open-source projects executed under controlled envi-
ronments. In this study, we replicate and adapt ERT in an indus-
trial setting and apply it to a larger software system with a rich
evolution history.

We, therefore, investigate the following research questions to
assess the applicability of this approach in an industrial environ-
ment:

* *To what extent can the energy consumption of developers' tests be measured stably in an industrial CI environment?* This question aims to examine whether energy measurements collected in our industrial setup are stable and whether a performance proxy can be identified to approximate energy consumption.

* Can developers' tests be used to detect potential energy regressions introduced by code changes in an industrial CI environment? This question investigates whether energy regressions can be detected based on the changes introduced by developers across successive versions.

To answer these research questions, we adapt the approach of
the original study [4] to our industrial setup to overcome compati-
bility challenges. We therefore design a five-step process that con-
sists of extracting the commit history of the software under study,
executing the full test suite for each commit and recording energy
and performance measurements, analysing the collected metrics,
detecting regressions to label each commit as passing or breaking
in terms of energy consumption, and finally performing a code-
change analysis to interpret the causes of the detected regressions.

This replication reveals impactful insights. First, it confirms the
original study's findings by showing that energy and performance
measurements can be collected stably even in an industrial envi-
ronment. The analysis shows that cycles are highly correlated with
energy consumption, with a correlation coefficient of 0.75, indi-
cating that cycles can serve as a reliable proxy for understanding
energy behaviour. Execution duration also shows a correlation of
0.75 with energy, suggesting that longer executions consume more
energy. This strong correlation is expected because test executions
in our environment are mostly sequential, so longer runs naturally
consume more energy. Second, our results confirm that, in an in-
dustrial setting, developers' tests can be used to assess the impact

of code changes on software energy consumption. This conclusion
is supported by a manual assessment of the code modifications in-
troduced in each commit classified as a regression.

The remainder of this paper is organized as follows. Section 2 presents the industrial case study, describing the use case and the software system under investigation. Section 3 details the methodology adopted in this work. Section 4 reports the experimental results answering each research question. Section 5 discusses the threats to validity and the lessons learned, while Section 6 reviews related work. The paper concludes with a general conclusion and outlines future research directions in Section 7.

## 2 Industrial Case Study

To evaluate the proposed approach, we applied it to the commit
history of an industrial Java application managed with Maven. The
backend is built with Spring Boot 3.5.4 and Java 21. It consists of
894,029 lines of code, 7,909 classes, 88,200 methods, and 40,594
attributes. This application represents a realistic, large-scale finan-
cial management system used in production. It shows a rich devel-
opment history with 3,562 commits and includes more than 200
test cases, making the application complex and actively maintained
with frequent daily modifications.

The application was selected in collaboration with the develop-
ment team because it is representative of an actively developed
and tested software system within the company's CI infrastructure,
making it well-suited for evaluating the feasibility and relevance
of energy regression detection in a real industrial context. These
characteristics make it an appropriate candidate for studying the
energy behaviour of an evolving industrial software system.

## 3 Methodology

To evaluate whether developers' tests can be used to detect energy regressions across different versions of the same application in an industrial context, we propose a five-step process illustrated in Figure 1. This process is designed to analyse multiple consecutive versions of the target application. The analysis was performed by running all tests in an identical environment, enabling a systematic evaluation of the impact of code changes on the software's energy behaviour.

### 3.1 RQ1: To what extent can the energy consumption of developers' tests be measured stably in an industrial CI environment?

To answer RQ1, we consider two hypotheses related to the stability of the measurements. The first hypothesis (H1) states that the energy measurements obtained from developers' tests are stable and reproducible. The second hypothesis (H2) states that performance metrics, such as executed instructions and CPU cycles, can be used as proxies to provide stable approximations of energy consumption. The evaluation of these hypotheses relies on the first three steps of our process illustrated in Figure 1.

***Extract the commit history:*** Commit extraction consists of selecting a sequence of consecutive commits from the repository, where each commit represents a distinct version of the application.
<figure><img src="image_3.png" /><figcaption>Figure 1: Process measuring the impact of code changes on energy and performance.</figcaption></figure>

This step is necessary to obtain multiple versions of the software and evaluate its energy behaviour before and after code changes. The commit history is extracted from the main development branch, reflecting realistic development activity and capturing the natural evolution of the software under study. We retrieve this history using standard Git commands, which provide a complete list of commits and allow us to navigate to each version for analysis. Since each version must be buildable, only commits that can be correctly built are retained. In our industrial setup, some older versions rely on dependencies that have been removed from the company repositories, making them impossible to build. However, the changes introduced in these unbuildable commits are not lost, as they are incorporated into the subsequent buildable versions. As a result, the extracted history consists of the subset of commits that can be built and executed successfully, while still capturing the full evolution of the software system.

**Execute the full test suite and record energy and performance metrics:** Unit tests are automated tests designed to verify the behaviour of individual components within the application. To understand the impact of a code change on software energy consumption, it is necessary to analyse the behaviour of the unit tests that exercise the modified portions of the codebase. This analysis requires collecting energy and performance measurements during test executions. To support this, existing tools, such as Diff-Joules [3], which estimate the energy impact of code changes; JoularJX [15], which monitors energy usage at the source-code level; PowerJoular [15], which tracks real-time hardware power consumption; and Perf [17], which counts hardware-level events, can be used. Due to compatibility constraints in our industrial setup, it was not possible to rely on Diff-Joules, the tool proposed in the original study. Since Diff-Joules itself relies on low-level measurement sources such as Perf for performance counters and RAPL for energy readings, using these sources directly allows us to obtain equivalent measurements even without the tool. To overcome this limitation, we combine Perf, which provides performance metrics such as instructions, CPU cycles, and execution time, with direct readings from the CPU's RAPL interface exposed via /sysfs. The energy values are extracted by reading the counter available at /sys/class/powercap/rapl:0/energy_uj. However, the RAPL counter is cumulative, so to compute the energy consumption of a test execution, we record its value immediately before running the test suite, execute the tests, and then read the counter again after completion. The energy consumed by the tests is computed as the difference between the two readings. To obtain more reliable, statistically significant results, we run the full test suite 100 times per commit and record

performance metrics and energy measurements for each execution. The data collected during this step consist of performance metrics and energy measurements used in the subsequent phases of the study. In particular, the recorded energy values are used to compute the average power consumption using the following formula: The collected energy measurements also require a cleaning step to address negative energy values caused by the RAPL counter's behaviour. This counter is a monotonically increasing hardware register with a fixed maximum range. When this maximum is reached, the counter wraps back to zero. As a result, if this wrap-around occurs between the “before” and “after” readings, the simple difference between these two values may appear negative, even though the actual energy consumption is strictly positive.

**Analysis and visualization of collected metrics:** This step has two objectives. First, it aims to assess the stability of energy and performance metrics across repeated executions. Second, it seeks to determine whether certain performance metrics can serve as reliable proxies for energy consumption.

To do so, we analyse the energy and performance metrics obtained in the previous step using standard deviation and coefficient of variation.

The standard deviation (σ) [7] quantifies the absolute variability of the metrics across repeated runs. A low standard deviation indicates that the values are close to the mean (μ) of the distribution. The standard deviation is computed using the formula:

$$ \sigma = \sqrt{\frac{\sum_{i \in N(x_i - \mu)^2}}{N}} $$

The Coefficient of Variation (CV) [1] measures the dispersion of a probability distribution. In other words, the CV provides a normalized measure of variability that accounts for differences in metric scale. A low CV indicates that the metric is stable across executions. The CV is computed using the formula:

$$ CV = \frac{\sigma}{\mu} $$

To represent measurement stability, we summarize the CV value using a histogram and a Kernel Density Estimate (KDE), which highlight the distribution and spread of variability across executions.

The second part of this step aims to determine whether performance metrics can serve as proxies to estimate energy consumption in software systems, thereby reducing the noise associated with direct energy measurements and obtaining more reliable results. To this end, we adopt the Pearson correlation coefficient [2],
as it is appropriate for evaluating linear relationships between measured metrics. Pearson correlation measures the strength and direction of the linear relationship between performance metrics and energy consumption. A high correlation value indicates that a performance metric is strongly linked to energy usage, suggesting that it may serve as a potential proxy for energy behaviour. To analyse these correlations, we summarize the results using a correlation heatmap, which facilitates the comparison of correlation strengths across multiple metrics.

## 3.2 RQ2: Can developers' tests be used to detect potential energy regressions introduced by code changes in an industrial CI environment?

To answer RQ2, we rely on the outputs of the first three steps and extend the process with additional steps, as illustrated in Figure 1.

**Decision and regression detection:** Energy regression refers to an increase in energy consumption introduced by a code change, analogous to a performance regression but focused on energy-related metrics. Identifying such regressions is essential for determining whether a change negatively or positively affects the software's energy efficiency. In our approach, regression detection relies on the output from the previous step, which provides a baseline for comparing metric values before and after each change. To achieve this, we rely on the collected metrics for each successive pair of commits, $C_i$ and $C_{i+1}$. For each pair, we compute the metric deltas by subtracting the measurements of $C_{i+1}$ from those of $C_i$, which produces a set of difference values for both performance and energy metrics. We then organise these results into four categories: (1) pairs in which all performance metrics show a regression, meaning that all deltas are positive; (2) pairs in which only the energy metric exhibits a regression; (3) pairs in which all metrics show an improvement in both performance and energy; and (4) pairs in which only the energy metric shows an improvement. Pairs for which no meaningful change is observed across metrics are not considered further, as they do not contribute to regression analysis.

**Code-change analysis of the breaking commits :** This step complements the regression classification by analysing the code changes associated with commits identified as energy regressions. This analysis is necessary to ensure that the detected regressions correspond to meaningful software modifications rather than to measurement variability or environmental noise.

To achieve this, we perform a qualitative analysis of each commit that is classified as exhibiting energy regression. This analysis examines the source-code changes introduced by the commit, including modifications to existing logic, additions or removals of functionality, and changes to dependencies or configurations. By analysing these elements, this approach provides the contextual information required to relate observed energy increases to concrete modifications in the software. It further supports the interpretation of the detected regressions by assessing whether the magnitude and direction of the energy impact are consistent with the nature and scope of the changes introduced by the developers.

<figure><img src="image_4.png" /><figcaption>Figure 2: Distribution of the Coefficient of Variation (CV) for energy, cycles, and instructions across commits.</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>Figure 3: Clustered correlation heatmap of energy consumption and performance metrics (CPU, cycles, instructions, duration, and power) across commits.</figcaption></figure>
# 4 Results

## 4.1 RQ1: To what extent can the energy consumption of developers' tests be measured stably in an industrial CI environment?

In terms of measurement stability and the correlation between performance metrics and energy consumption, our experiments yielded valuable results.

Figure 2 shows the distribution of the *Coefficient of Variation* (CV) for energy, cycles, and instructions across all commits. This figure allows us to evaluate whether the measurements remain stable and reproducible across repeated executions. The CV values for cycles and instructions generally range from 0.01 to 0.045, whereas the CV for energy reaches 0.06, indicating greater variability. Most instruction CV values cluster around 0.015, showing consistent behavior. Cycles exhibit slightly lower dispersion than instructions, suggesting that they are the most stable of the three metrics.

Overall, the figure shows that cycles and instructions remain highly stable across executions, while energy displays a higher CV at certain points. This higher variability is expected, as RAPL-based energy measurements are more sensitive to hardware noise and system-level interactions. Nevertheless, most energy CV values remain below 0.02, indicating that energy consumption can still be measured with reasonable stability in this industrial context.

These observations support H1 by confirming that energy measurements obtained from developers' tests are stable and reproducible. They also show that performance metrics associated with test executions exhibit strong stability across repeated runs.

Even though RAPL-based energy measurements show reasonable stability, it remains important to determine whether performance metrics can serve as reliable proxies for energy consumption. Reliable proxies would enable developers to reason about energy behavior without relying solely on direct energy measurements. To investigate this aspect, we analyze the correlation between performance metrics and energy consumption. Figure 3 depicts a clustered correlation heatmap showing the relationships among these metrics.

The heatmap shows how each performance metric relates to energy consumption. Cycles exhibit a correlation of 0.75 with energy, while instructions show a correlation of 0.68. These values indicate a moderate relationship between performance metrics and energy usage. Cycles show a slightly stronger correlation, suggesting they may be a more suitable proxy for approximating energy consumption. However, the correlation remains moderate, meaning that performance metrics can capture part of the energy behavior but cannot fully replace direct energy measurements.

CPU utilization shows a strong negative correlation with energy consumption, with a value of -0.45. This means that tests running with lower CPU usage tend to consume more energy. CPU utilization also correlates strongly and negatively with execution time, with a value of -0.73. This indicates that tests running at higher CPU load generally complete faster. The correlations between CPU utilization and the number of instructions and cycles remain weak and negative, with values not exceeding -0.24. These weak correlations suggest that CPU utilization reflects execution intensity

rather than the amount of executed operations. Based on these observations, CPU utilization alone cannot serve as a suitable proxy for energy behavior.

To further investigate the energy-execution time correlation, the heatmap shows a strong, positive correlation ($ρ = 0.75$). This correlation indicates that longer test executions tend to consume more energy. The duration also shows a strong correlation with instructions and cycles, with values of 0.80 and 0.83, respectively. These correlations indicate that tests that perform more operations take longer to complete. However, time exhibits a strong negative correlation with CPU usage, with a value of -0.73. This suggests that tests running with a high CPU load finish faster. Considering the execution duration helps understand the software's behaviour, which in turn supports identifying its energy behaviour.

To further examine how power relates to energy behaviour, the heatmap shows that power exhibits only a very weak correlation with total energy ($ρ = 0.10$). This indicates that variations in power do not reflect the overall energy consumed during test execution. Power correlates moderately and positively with CPU utilization ($ρ = 0.69$), suggesting that it is more sensitive to processor activity than to the cumulative energy used. In contrast, power shows a moderate negative correlation with execution time ($ρ = -0.52$) and very weak correlations with cycles ($ρ = 0.11$) and instructions ($ρ = 0.08$), indicating that neither longer executions nor higher operational workloads are consistently associated with changes in power. Overall, these results show that power captures a different aspect of test execution behaviour and cannot be considered a reliable proxy for understanding software energy consumption.

The clustering structure reinforces these observations by grouping energy, cycles, instructions, and execution time, while placing power and CPU utilization in separate branches, confirming that they follow different behavioural patterns.

These observations support H2, as the heatmap indicates that performance metrics can serve as proxies for understanding the software system's energy behaviour after a specific change. Cycles show the highest suitability as a proxy, and execution time also provides a useful indication of energy behaviour.

Our results show that it is possible to measure the energy consumed by software tests in an industrial context. By validating H1 and H2, we demonstrate that performance metrics—mainly instructions and cycles—are more stable than energy measurements, and that cycles correlate more strongly with energy than instructions, making them a suitable proxy for approximating energy consumption. Execution time also provides complementary insight into the system's energy behaviour.
**Answer to RQ1:** In an industrial context, executing unit tests across different versions of the software system yields stable energy consumption measurements. We also observe that performance metrics, such as CPU cycles and instructions, are more stable than direct energy measurements. These results validate Hypothesis H1, showing that both energy consumption and performance can be measured through developers' unit test executions. Furthermore, CPU cycles and execution time correlate strongly with energy consumption ($\rho$ = 0.75), while instructions show a slightly weaker correlation ($\rho$ = 0.68). This confirms that performance metrics—especially CPU cycles—can serve as useful proxies for understanding energy behaviour, although they cannot fully replace direct energy measurements.

## 4.2 RQ2: Can developers' tests be used to detect potential energy regressions introduced by code changes in an industrial CI environment?

<table><caption>**Table 1:** Classification of commit pairs based on metric deltas.</caption><thead><tr><td>Category</td><td>increased</td><td>decreased</td></tr></thead><tbody><tr><td>All metrics</td><td>7</td><td>4</td></tr><tr><td>Only energy</td><td>2</td><td>7</td></tr></tbody></table>

Building on the observed stability of energy measurements, our results show that developers' tests can be used to distinguish commits by energy impact and to detect energy regressions introduced by code changes.

Table 1 summarizes the resulting classification based on metric deltas. The first column reports the category assigned to each consecutive commit pair, while the remaining columns indicate the number of pairs exhibiting increased or decreased values. Based on the deltas, we identify four main categories. The first corresponds to cases where all metric deltas are positive, indicating that the change leads to a regression in both performance and energy. The second category contains pairs in which all deltas are negative, indicating that the change improves the energy behavior of the software. The last two categories correspond to cases in which the energy delta is either positive or negative, indicating an increase or decrease in energy while performance metrics remain stable. These cases can indicate hidden regressions or improvements that are not reflected in performance metrics.

Based on these results, we observe that 7 commit pairs out of the 92 analyzed exhibit regressions not only in energy consumption but also in performance metrics. This finding shows that our approach can detect regressions introduced by specific code changes. It also indicates that developers' tests are sensitive to such changes and can expose performance and energy degradations across versions.

4 commit pairs lead to an improvement in energy consumption, indicating that certain code changes can have a positive impact on energy behavior. These cases also show improvements in performance metrics, suggesting that the corresponding modifications

reduce the system's overall workload. The execution of the test suite in these commits can therefore detect such improvements.

Our analysis also categorizes 7 commit pairs as energy improvements while the performance metrics remain stable or deteriorate. This indicates that, although performance metrics and energy consumption are correlated, there are cases where performance metrics alone cannot detect a regression or an improvement triggered by a code change.

Finally, the two commit pairs in which only energy increases reveal hidden regressions that are not reflected in performance metrics, further confirming that direct energy measurement is necessary to identify certain energy-related issues.

These findings indicate that developers' tests can reveal energy regressions introduced by code changes, including both clear regressions and hidden ones that are not reflected in performance metrics. The presence of improvements, regressions, and mixed behaviors suggests that energy measurements provide complementary information that cannot be fully inferred from performance counters alone. While these results support developers' tests' ability to detect energy-related issues in our industrial setting, they should be considered preliminary evidence and require further validation across additional projects and environments.

## Code-Change Analysis of The Breaking Commits

To understand the root causes of the regressions observed across the 7 commit pairs, this section examines the code changes introduced between the commits that triggered them.

As we are working in an industrial setting and due to security and confidentiality constraints, we report each regression case using the generic label "Regression Case R".

* **Regression Case R1 – Mapper modification.** In this case, the changes occurred at the level of the mapper that converts domain objects into resource representations. The modification introduced new fields into the mapper, along with several new getters and setters integrated into the mapping logic, and new header declarations. Handling these additional fields and invoking the new accessors requires more object manipulations and method dispatches. This increase in operations raises the overall workload and processing demand during test execution. The resulting higher computational effort leads to increased performance and energy consumption, which explains the observed regression across all metrics.

* **Regression Case R2 – Java version upgrade.** In this case, the observed regression is due to an upgrade of the configuration environment, without any change to the application logic. This commit applies a transition from Java 17 to Java 21 and includes a configuration modification to enable virtual threads. The commit does not change the application logic, but it introduces a runtime upgrade that affects the JVM. These changes can include updates to JIT compilation strategies that affect how the program is optimized
and executed at runtime. Such modifications impact the performance metrics and contribute to the regression. In addition, the introduction of virtual threads adds extra scheduling and continuation-management overhead in short-lived workloads, such as developer test suites. This overhead increases execution time and, in turn, energy consumption, which explains the observed energy regression.

* **Regression Case R3 - Parent project upgrade or downgrade.** Here, we find that three commits introduce either an upgrade or a downgrade in the POM file, indicating that these changes are related to the libraries of the parent project of the software itself. Such upgrades or downgrades can alter the configurations, methods, and API calls inherited from the parent project. These modifications can introduce new or more complex internal logic, additional class loading, or different runtime behavior. These changes lead to increased performance metrics, which explains the regressions. The older parent-project version involved in the downgrade appears to be more energy-consuming, while the newer parent-project version involved in the upgrade also showed a higher energy regression.

* **Regression Case R4 - Imports & path updates.** In this commit, we observe that the developers only reorganized the project structure, updating the imports. These changes did not impact the functional logic, which should not affect runtime behavior, and any observed regression is therefore likely attributable to measurement noise or minor class-loading differences.

* **Regression Case R5 -DTO & test cases update.** In this commit, developers introduced new test cases that can increase execution time and lead to higher energy consumption. Besides this, they introduced new *Data Transfer Object* (DTO) and helper classes, which result in additional object allocations, more getters and setters, and the handling of extra values. The commit also includes new methods and multiple conditional statements, increasing the amount of runtime logic. These additions can lead to higher performance costs, which explains the energy regression observed in this commit.

This manual assessment validates that the energy measurements of the developers' test suites can detect regressions, and that these regressions are explained by the code changes made by the developers.

**Answer to RQ2:** Developers' test suites allow the detection of energy regressions between consecutive versions of the software system. They reveal both clear regressions reflected in performance metrics and hidden energy-only regressions, confirming that direct energy measurements provide complementary information beyond performance counters.

# 5 Threats to Validity & Lesson Learned

## 5.1 Threats to Validity

**5.1.1 Internal Threats to Validity.** Internal threats to validity involve factors within the study's design that could impact the accuracy and reliability of our findings. In this study, the energy

measurements rely on direct readings of the RAPL package. These measurements are cumulative, so we read the counter before and after execution and compute the difference to obtain the energy related to our tests. This approach can be noisy, as the reported value reflects the entire system rather than just the process under evaluation, which can affect measurement reliability. Occasionally, the RAPL counter reaches its maximum range and wraps around, producing negative values, which we removed during data cleaning. These factors represent a limitation that must be considered when interpreting the energy results. To reduce the impact of measurement noise, each test execution was repeated multiple times. Another factor that can affect our results is that the approach was executed on a machine intended to isolate the environment; however, this isolation was still subject to uncontrollable factors, such as background system activity and OS-level scheduling. These effects can influence both performance counters and execution time, introducing variability unrelated to the code changes themselves, which may affect the analysis of the software system's energy behaviour.

**5.1.2 External Threats to Validity.** External threats to validity consider factors that could limit the generalizability of our findings. Our results are based on a single large Java/Maven system running within Berger-Levrault's development infrastructure. Other programming languages, build tools, or CI configurations may exhibit different energy behaviors, making it uncertain whether the same level of measurement stability or regression detectability would hold in other technical ecosystems. Another factor that can affect the results is that older dependencies were removed from the industrial repositories, which caused several earlier commits to fail to build due to missing or incompatible components. As a result, only 92 commits were executable for analysis. This restriction limits the representativeness of the studied evolution window and may reduce the generalizability of our findings to the full project lifecycle or to other systems with different dependency structures. RAPL availability and accuracy depend on the CPU architecture; therefore, our observations may not fully generalize to environments with different processors, power-management mechanisms, or virtualization layers, making replications on diverse platforms necessary to reinforce external validity.

## 5.2 Lessons Learned

In an industrial setup, this replication yielded several lessons. Adapting academic methodologies to industrial constraints requires flexibility, as several tools used in the original work were incompatible with the target environment. This limitation led us to implement custom measurement scripts combining perf and RAPL. The study also highlighted practical challenges in conducting energy measurements in a production-like environment. Despite efforts to isolate the machine, background processes, system-level scheduling, and infrastructure constraints introduced noise into the measurements, reinforcing the importance of repeated executions and statistical aggregation.

During commit extraction, several earlier versions could no longer be built because their dependencies had been removed or had become incompatible with the current repository structure. This limitation reduced the number of candidate commits in real industrial
environments. Unlike controlled academic projects, industrial systems evolve under constraints such as dependency deprecation, repository cleanup, and architectural restructuring, which challenge long-term reproducibility. This experience shows that preserving reproducible builds is essential for conducting reliable energy comparisons across software versions.

Some of the detected regressions were due to dependency changes, such as parent project updates, library upgrades, or configuration adjustments. These modifications led to energy or performance regressions that are not immediately visible through performance or energy metrics alone. As a result, identifying the root causes of such regressions required manual analysis of the commits. This highlights that detected regressions must be complemented by an analysis of the associated changes to correctly interpret energy-impacting modifications in industrial systems.

## 6 Related Work

The literature includes several works on the energy measurement of software systems, addressing the topic from different perspectives and at varying levels of granularity.

Danglot et al. [4] presented *Energy Regression Testing* (ERT), an approach designed to detect energy regressions in *Continuous Integration* (CI) environments by leveraging developers’ tests. ERT analyzes whether changes introduced by developers increase energy consumption across software versions. Their work shows that hardware performance counters, such as CPU cycles and instructions, are significantly more stable than direct energy measurements obtained via RAPL. Moreover, they report a strong correlation between performance metrics—particularly CPU cycles—and energy consumption, enabling cycles to serve as a reliable proxy for detecting energy regressions. While ERT provides valuable insights into identifying energy-impacting code changes during CI workflows, its evaluation was conducted on small- to medium-sized open-source projects. This limitation raises the question of whether the same findings hold in large-scale industrial environments, where CI workflows, codebases, and development activities are substantially more complex.

Pereira et al. [16] proposed *SPpectrum-based Energy Leak Localization* (SPELL), an approach that aims to identify energy-inefficient code regions by combining spectrum-based fault localization with energy measurements. SPELL combines energy measurements collected during test execution and program tracing with statistical techniques from *Spectrum-based Fault Localization* (SFL). This technique then analyzes the results to assign responsibility scores to individual source-code components, effectively ranking them according to their contribution to overall energy consumption. This process leads to a classification that allows developers to investigate energy-inefficient code fragments. The technique provides valuable insights to help developers understand their code's behavior and guide them to parts of the system that should be optimized. However, this technique focuses mainly on a single software version; understanding behavior across versions would help detect regressions caused by code changes.

PowerJoular and JoularJX, two tools proposed by Noureddine [15], enable real-time energy monitoring. PowerJoular allows developers to monitor the energy behavior of their applications during execution by tracking hardware power consumption at the process level via specific PIDs, using Intel RAPL or empirical regression models. JoularJX extends this capability to the source-code level by attributing energy consumption to specific methods within Java applications, providing fine-grained insights into which parts of the code contribute most to overall energy usage. However, PowerJoular is currently limited to supported hardware architectures and components—primarily CPUs and GPUs via Intel RAPL or regression models—while JoularJX remains focused on Java applications, limiting its applicability across diverse systems and programming environments.

Maquoi et al. [13] introduced Energy Codesumption, an approach that analyzes how the source code of Java projects impacts their energy consumption. This approach measures energy usage by executing tests without modifying the source code. It uses JoularJX to collect energy consumption data during test execution, with measurements linked to call traces. The collected data is then filtered to remove outliers and subjected to statistical tests to evaluate energy behavior and categorize lines of code. This approach reveals significant relationships between object construction complexity and energy consumption. However, it has notable limitations: it relies on manual analysis, lacks integration with version control systems, and does not account for how energy consumption evolves across software versions, making it unsuitable for detecting energy regressions introduced by code changes.

Misu et al. [14] investigated the relationship between test smells and the energy consumption of software. By mining 12 open-source Apache projects and measuring test execution energy using Intel PowerLog, they showed that the “smelly” parts of test cases consume more energy than their clean counterparts, especially for smells such as Assertion Roulette, Lazy Test, and Eager Test. The study also demonstrated that refactoring to remove test smells significantly reduces energy usage, although it remains limited by potential influences from production code, limited developer coverage, reliance on @Test-based detection, hardware-specific energy measurements, and dependence on tools such as JNose and Intel PowerLog. Moreover, the work focuses solely on energy behavior within individual software versions and does not address how energy consumption evolves across versions, making it unsuitable for detecting energy regressions introduced by code changes.

De Medeiros et al. [5] conducted a large-scale analysis of 1,167 CI workflows from popular Java projects on GitHub. This analysis enabled them to measure the energy consumption of 183,355 tasks executed automatically using the Apache Maven and Gradle build systems. The measurement process relied on SmartWatts, after which the tasks were manually classified to compare the energy behavior of different CI activities. Their results show that build-automation tasks account for nearly a quarter of the total energy consumed by CI workflows, with testing tasks being by far the most energy-intensive—responsible for 47% of Maven’s energy usage and 36% of Gradle’s. The study is limited to popular Java projects using GitHub Actions with Maven and Gradle, which limits the generalizability of the findings to other programming languages or build tools. While this work provides valuable insights
into the energy profile of CI activities, it focuses on categorizing task-level energy consumption rather than analyzing how energy behavior evolves across software versions.

Mancebo et al. [12] conducted an empirical study investigating the relationship between software maintainability and energy consumption. Their work evaluates maintainability metrics such as total lines of code, cyclical complexity, code duplication, and commenting density, and relates them to energy consumption measured using SonarQube and a dedicated Energy Efficiency Tester (EET) device. The results indicate that software evolution and maintainability changes can influence energy behavior, with total lines of code showing a clear correlation with energy consumption. The study is limited to a small number of software versions executed in a controlled experimental environment, and significant correlations are observed primarily for code size, while other maintainability metrics do not exhibit clear relationships with energy consumption. While this work provides early empirical evidence of a link between maintainability and energy usage, it focuses on coarse-grained, version-level analysis and does not address measurement stability or energy variations at finer granularities.

Ding et al. [6] study the use of tests in the release pipeline as performance tests for the Hadoop and Cassandra projects. Their results show that, for 102 of 127 identified performance issues, at least one existing test exposes a performance improvement after the corresponding fix. However, the study also reports that only a small subset of available tests is effective at revealing such improvements, and the analysis is limited to performance metrics rather than direct energy measurements, leaving open questions about measurement stability and energy-related regressions.

# 7 Conclusion & Future Work

In this study, we proposed a four-step process to overcome the compatibility challenges encountered in the industrial replication of the original study. By adapting the methodology to Berger-Levrault's environment, we were able to measure energy consumption and performance metrics across multiple versions of a large-scale project, presenting a realistic and rich commit history.

The results show that, in an industrial setup, it is possible to measure energy and performance metrics stably, and further analysis revealed that performance metrics—mainly cycles—are correlated with energy measurements, making them a suitable proxy for estimating the energy consumption of the software system. In addition, our approach enabled us to classify changes as breaking when they introduce an energy regression or as passing when they do not, and these regressions were validated through manual analysis of the corresponding commits.

As a continuation of this replication, a promising direction is to extend the analysis to detect and characterize energy drift across software evolution. While this study focused on identifying regressions between consecutive commits, drift analysis aims to capture long-term deviations in the energy behaviour of code changes. Such drifts can result from evolving dependencies, configuration updates, runtime optimizations, or environmental factors. Incorporating drift analysis into future work would enable a deeper understanding of how energy consumption evolves over longer periods.

# References

[1] Hervé Abdi. 2010. Coefficient of variation. *Encyclopedia of research design* 1, 5 (2010), 169–171.

[2] Mohammed Chaouki ABOUNAIMA, Fatima Zahra EL MAZOURI, Loubna LAM-RINI, Najib NFISSI, Noureddine EL MAKHFI, and Mohamed OUZARF. 2020. The Pearson Correlation Coefficient Applied to Compare Multi-Criteria Methods: Case the Ranking Problematic. In *2020 1st International Conference on Innovative Research in Applied Science, Engineering and Technology (IRASET)*. 1–6. doi:10.1109/IRASET48871.2020.9092242

[3] Davidson Consulting. 2024. *diff-jjoules Documentation*. https://github.com/davidson-consulting/diff-jjoules/blob/main/doc/documentation.md.

[4] Benjamin Danglot, Jean-Rémy Falleri, and Romain Rouvoy. 2024. Can we spot energy regressions using developers tests? *Empirical Software Engineering* 29, 5 (2024), 121.

[5] Sérgio Queiroz de Medeiros, Romain Lefeuvre, Benoit Combemale, and Quentin Perez. 2025. Evaluating the Energy Profile of Tasks Managed by Build Automation Tools in Continuous Integration Workflows: The Case of Apache Maven and Gradle. In *2025 11th International Conference on ICT for Sustainability (ICT4S)*. 12–22. doi:10.1109/ICT4S68164.2025.00011

[6] Zishuo Ding, Jinfu Chen, and Weiyi Shang. 2020. Towards the Use of the Readily Available Tests from the Release Pipeline as Performance Tests. Are We There Yet?. In *2020 IEEE/ACM 42nd International Conference on Software Engineering (ICSE)*. 1435–1446.

[7] Olukorede Fakorede, Modeste Atsague, and Jin Tian. 2024. Standard-Deviation-Inspired Regularization for Improving Adversarial Robustness. *arXiv:2412.19947 [cs.LG]* https://arxiv.org/abs/2412.19947

[8] Hugues Ferreboeuf, Maxime Efooui-Hess, Marlene De Bank, Ilana Toledano, and Alexandre Theve. 2024. *Energy and climate: What virtual worlds for a sustainable real world? Positioning our technological choices towards digital sobriety*. Final Report. The Shift Project.

[9] Hugues Ferreboeuf, Maxime Efooui-Hess, and Xavier Verne. 2021. *Environmental impact of digital: 5-year trends and 5G governance*. Technical Report. The Shift Project.

[10] Louay Khrouf, Anas Shatnawi, Boubou Thiam Niang, and Benoit Verhaeghe. 2025. On the Energy Consumption of Web Applications: An Empirical Study of their Design Solutions. In *Proceedings of the 13th International Workshop on Green and Sustainable Software (GREENS). IEEE*.

[11] Louay Khrouf, Anas Shatnawi, Romain Rouvoy, Benoit Verhaeghe, and Boubou Thiam Niang. 2026. Energy Consumption of Web Applications: Measurement Challenges in Practice. *IEEE Software Special Issue on Green Clean Software Sustainability (March 2026)*. doi:10.1109/MS.2025.3641504

[12] Javier Mancebo, Coral Calero, and Félix García. 2021. Does maintainability relate to the energy consumption of software? A case study. *Software Quality Journal* 29, 1 (01 Mar 2021), 101–127. doi:10.1007/s11219-020-09536-9

[13] Jérôme Maquoi, Maxime Cauz, Benoit Vanderose, and Xavier Devroey. 2025. Energy Codesumption, Leveraging Test Execution for Source Code Energy Consumption Analysis. In *Proceedings of the 33rd ACM International Conference on the Foundations of Software Engineering*. 1432–1436.

[14] Md Rakib Hossain Misu, Jiawei Li, Adithya Bhattiprolu, Yang Liu, Eduardo Santana de Almeida, and Iftekhar Ahmed. 2025. Test smell: A parasitic energy consumer in software testing. *Information and Software Technology* 181 (2025), 107671. doi:10.1016/j.infsof.2025.107671

[15] Adel Noureddine. 2022. PowerJoular and JoularJX: Multi-Platform Software Power Monitoring Tools. In *18th International Conference on Intelligent Environments (IE2022)*. Biarritz, France.

[16] Rui Pereira, Tiago Carção, Marco Couto, Jácome Cunha, João Paulo Fernandes, and João Saraiva. 2020. SPELLing out energy leaks: Aiding developers locate energy inefficient code. *Journal of Systems and Software* 161 (2020), 110463. doi:10.1016/j.jss.2019.110463

[17] Linux perf wiki Contributors. [n. d.]. "perf: Linux profiling with performance counters". https://perfwiki.github.io/main/.

[18] Vie Publique. 2021. Loi du 15 novembre 2021 visant à réduire l'empreinte environnementale du numérique en France. Available at: https://www.vie-publique.fr/loi/278056-loi-15-novembre2021-reen-reduire-empreinte-environnementale\-du-numerique

Received 12 December 2025; accepted 18 January 2026