# An analysis for the design of an efficient replica management strategy

Cherif Si Mohammed, Adrien Lebre, Alexandre van Kempen

► To cite this version:

Cherif Si Mohammed, Adrien Lebre, Alexandre van Kempen. An analysis for the design of an efficient replica management strategy. COMPAS 2025 - Conférence francophone d'informatique en Parallélisme, Architecture et Système, Jun 2025, Bordeaux, France. pp.1-7. hal-05079577

HAL Id: hal-05079577

https://hal.science/hal-05079577v1

Submitted on 22 May 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# An analysis for the design of an efficient replica management strategy

Cherif SI MOHAMMED<fnref n="1" />, Adrien LEBRE<fnref n="2" />, Alexandre VAN KEMPEN<fnref n="3" />

## Résumé

The transition from centralized systems to geo-distributed infrastructures offers increased flexibility by enabling the use of geographically distributed resources. These infrastructures are employed in fields requiring significant processing capabilities, such as scientific research. However, data management, which is an important aspect of efficiently managing geo-distributed infrastructure and achieving optimal performance, is difficult to implement due to various constraints (hardware and software constraints). In addition, energy consumption has become a crucial factor to consider in modern infrastructures. In this work, we focus on replication, which is a common approach in data management, but, it is important to understand that naive replication can lead to costly transfers and resource loses. The success of a replication policy relies on a balance between performance, availability, and data transfer optimization. Build on this, we present a preliminary study aimed at minimizing transfers, especially by looking for an efficient number of replicas, while ensuring optimal performance. Through this study, we have shown that by focusing only on important transfers, it is possible to achieve good performance while reducing energy costs.

**Mots-clés** : Geo-distributed, replication, scheduling, energy efficiency.

## 1. Introduction

Since the emergence of cloud computing, IT infrastructures have undergone a radical transformation. By providing access to powerful and scalable computing resources, the cloud has enabled organizations to deploy applications and services on a large scale without having to invest heavily in physical infrastructure. Major industry players such as Amazon Web Services (AWS), Google Cloud, and Microsoft Azure have been at the forefront of this evolution, providing businesses with the necessary tools to expand their computing operations globally.

However, as demands for high-performance computing and large-scale data storage have increased—along with evolving international regulations on data storage the centralized cloud model has started to show its limitations. Modern applications and use cases, which generate massive amounts of data and require significant computational power with near-instant resource availability, have highlighted challenges related to data availability, network latency, and fault tolerance. To address these challenges, centralized cloud infrastructures have gradually transitioned to geo-distributed ones, where computing power and storage are distribu-

<footnote n="1">¹ADEME, STACK team, LS2N, IMT Atlantique, Nantes - France. firstname.lastname@ademe.fr</footnote>

<footnote n="2">²INRIA, STACK team, LS2N, IMT Atlantique, Nantes - France. firstname.lastname@inria.fr</footnote>

<footnote n="3">³Qarnot Computing, Montrouge - France. firstname.lastname@qarnot-computing.fr</footnote>
ted across different geographical locations. This shift brings computational resources closer to
users, enabling faster access and improved fault tolerance.

Nevertheless, as geo-distributed infrastructures expand, the complexity and energy costs associated with resource management continue to grow due to increasing data volumes, higher communication overhead. One of the major challenges in these infrastructures is data management, regardless of the data type, data must be continuously exchanged across geographically dispersed sites to ensure optimal performance, as end-users are distributed. As an example, in distributed computing infrastructures, optimizing data placement can help reduce execution time [2]. Conversely, poor data management can lead to excessive data transfers and inefficient storage usage, which may result in high bandwidth and energy costs [6], increased latency, and network congestion risks. To overcome these limitations, selecting an effective data management strategy in geo-distributed environments is crucial for ensuring optimal performance while maintaining cost efficiency [1], [3], [7], [8].

In this paper, we present a preliminary study aimed at analyzing and identifying various KPIs and evaluating the impact of replication policies in geo-distributed infrastructure environments on the performance of bag-of-tasks job execution, as well as the environmental impact of data transfers. Based on this information, we aim to propose a replication policy that, by considering job-related information within the system, determines the appropriated number of replicas are needed, while taking into account both the performance of each job and the costs incurred by transfers.

The rest of the paper is organized as follows: In Section 2, we discuss the problem and replication methods. Section 3 presents the experimental setup, followed by a discussion of some results in Section 4.

## 2. Problem statement

In this study, we focus on a specific scenario where the infrastructure includes a type of job that consists of multiple tasks and a dataset, commonly referred to as a bag of tasks. The main objective of this study is to identify the important KPIs we can use to optimize management of data replication while considering both performance (such as job execution time) and overhead (such as transfers made). This scenario requires defining a dynamic replication policy that not only maximizes overall performance, but also minimizes the overhead related to data transfers on the infrastructure while ensuring efficient data management across multiple computing nodes.

In this study, we assume that task scheduling is performed in an abstract manner. Furthermore, if data is sent to a node, that node will necessarily be the location where the corresponding tasks are executed.

### 2.1. Replication policies

#### 2.1.1. Full Replication and One replica

We first consider two replication policies to initiate our study : **One replica** and **Full Replication**.

The first approach, **One replica**, consists of sending a **single and unique copy** of the dataset to the node where the scheduler has planned to execute all the job's tasks. As a result, tasks are executed sequentially, one after another. This method reduces costs in terms of data transfers, storage space, and bandwidth usage. However, it significantly increases the job execution time due to the lack of parallelism.
On the other hand, the second approach, **Full Replication**, involves **fully replicating** the dataset on all necessary nodes. When a job is submitted, we start by replicating the dataset on all required nodes. For example, if the job have 5 tasks, we replicate the dataset on 5 nodes and then we start execution. This approach enables **massive parallel execution** and delivers **optimal performance** by ensuring full utilization of the available computing resources. However, it comes at a higher cost in terms of storage and bandwidth usage, as well as increased energy consumption due to multiple data transfers and copies.

These two methods represent the extreme bounds of the search space for replication strategies in geo-distributed infrastructure (Either we replicate everywhere to achieve better performance, or we minimize the number of replicas but get undesirable results). They therefore serve as useful reference points for positioning other approaches between them, aiming to minimize data transfers while maintaining reasonable performance.

### 2.1.2. Pro-active Replication

Through the two replication methods previously introduced, we have laid the foundation for our goal by proposing an initial approach.

First, we begin by deploying a **single replica** of the dataset within the system. The purpose of this initial replica is to **gather sufficient information** on task execution in order to build a relevant execution profile, primarily based on **task execution time** (since we assume that tasks on the same job has approximately the same execution time). The method relies on a job-specific dynamic condition, which evaluates the necessity of adding additional replicas to the system in real time.

The underlying hypothesis is as follows : when a task from a job *J* is executed on the single available replica, if the condition in equation 1 is validated, an event is triggered and sent to the master to initiate the creation of a new replica, allowing parallel execution of additional tasks. This approach aims to optimize resource utilization by only triggering beneficial replica transfers, thereby avoiding unnecessary costs.

$$ transfer\_time < \frac{(execution\_time\_task \times (\#remaining\_tasks - 1))}{\#replicas} \quad (1) $$

For example, if the dataset transfer time is one hour, it would not be cost-effective to create a new replica just to execute very short tasks (e.g., 1-minute tasks), as the transfer cost would far exceed the performance gain. By applying this adaptive strategy, we may achieve a balance between execution efficiency and minimizing data transfer costs.

## 3. Experimental Setup

### 3.1. Infrastructure

For the infrastructure setup, we opted for an architecture composed of fully interconnected nodes. To simplify performance analysis and assess the impact of replication strategies in a controlled environment, we enforced **homogeneous network parameters**. This ensures that **bandwidth** and **latency** between nodes remain identical, providing uniform communication conditions across the entire system.

Additionally, all cluster nodes are homogeneous in terms of computing power, memory, and storage capacity. This uniformity eliminates biases related to hardware differences, allowing us to focus solely on the impact of replication policies on job execution.
## 3.2. Bag of tasks framework

In order to evaluate performance and identify relevant KPIs, we developed a dedicated framework for our interconnected node cluster. This tool enables job injection, execution monitoring, and optimization of resource management.

The framework, fully implemented in Python, follows a centralized architecture. The master node hosts a **job scheduler** and a **replica manager**, while each cluster node runs an **agent** to collect information and execute tasks. Communication between nodes is carried out using **HTTP requests** handled by a **Flask server**.<fnref n="1" />

A **Redis** [4] instance on each node facilitates data transfers and caching. We assume in this study an **unlimited storage space** to focus on optimizing performance and energy costs related to replication and execution.

## 3.3. Environment

Two experimental environments have been developed to evaluate our replication strategies and optimize resource management.

Firstly, we designed a **simulator** based on Python's **SimPy**<fnref n="2" /> library. This simulator allows us to thoroughly explore our search space by manipulating various parameters related to the infrastructure, such as the number of nodes, system load, latency, and bandwidth. It also enables adjustment of job-specific parameters, including the number of jobs injected, the number of tasks per job, task execution time and dataset size.

Secondly, to bring our setup closer to real-world conditions, we emulated the infrastructure using the **Enoslib** [5] library on the **Grid'5000** testbed. The previously presented framework was deployed on this infrastructure with **N client nodes** and a **master node** responsible for orchestrating job execution and replica management. This deployment allows us to assess the performance of replication strategies in a realistic distributed environment, considering network and computational constraints specific to modern computing infrastructures.

## 3.4. Experimental protocol

Regarding the experimental protocol, the job injector, deployed on the master node, generates synthetic job traces and injects them following a Poisson distribution. This probabilistic model simulates a realistic arrival of jobs into the system, with a variable infrastructure load.

# 4. Results and discussion

In this section, we present and discuss key metrics extracted from experiments conducted exclusively using simulator. We simulate an infrastructure with 100 homogeneous nodes and uniform network parameters (bandwidth and latency). Then, we inject 1 000 jobs, each containing between 1 and 50 tasks, forming a total of 25 905 tasks. Additionally, each job is associated with a dataset ranging in size 1MB and 60MB.

## 4.1. Transfers, Average Number of Replicas and Average Execution Time

The two KPIs that best illustrate our upper and lower bounds, as well as the balance we aim to achieve, are the number of data transfers performed during the workload, the average number of replicas per job, and the average job execution time.

<footnote n="1">1. Flask: https://flask.palletsprojects.com/en/stable/</footnote>

<footnote n="2">2. SimPy: https://simpy.readthedocs.io/en/latest/</footnote>
Table 1 below summarizes these three KPIs for the three replication policies. We observe that as more data is replicated, the average job execution time decreases. This can serve as a valuable reference point to position ourselves between these two extremes. By adding a condition that limits replication only to cases where it has a real impact on execution time, we can further optimize our approach.

<table><caption>TABLE 1 – Table of Transfers, Average Number of Replicas, and Average Execution Time</caption><thead><tr><th>Replication policy</th><th>One replica</th><th>Pro-Active replication</th><th>Full replication</th></tr></thead><tbody><tr><td>Nb of transfers</td><td>1000</td><td>9648</td><td>25905</td></tr><tr><td>Average replicas number per job</td><td>1</td><td>9.6</td><td>25.9</td></tr><tr><td>Average job's execution time (s)</td><td>816.61</td><td>278.64</td><td>46.11</td></tr></tbody></table>

With this additional condition, there is a performance trade-off compared to the aggressive replication policy (Full replication). However, this approach results in a significant reduction in transfers and average execution time. Specifically, we observe a 62% reduction in the average number of replicas compared to the aggressive policy. With an acceptable increase in average execution time.

## 4.2. Waiting Time and Flow Time

A deeper analysis of the three replication policies shows that waiting time, defined as the delay between job submission and execution start, varies with the strategy. The policy with a single replica per job minimizes waiting time by freeing resources for other jobs.

Conversely, the most aggressive replication method increases waiting time because it involves a high degree of parallelism, which causes the job to occupy a lot of resources, blocking the next job and requiring it to wait in order to minimize its execution time.

Another important KPI is the flow time, defined as the time elapsed from job submission to job completion, accounting for both execution time and waiting time. This metric allows us to assess the trade-offs between different replication policies.

In these experiments, we observe that, in terms of flow time, the proactive replication strategy performs better compared to the other two methods. As shown in the CDF plot 1, the distribution of flow time for proactive replication is more concentrated towards the left, indicating lower flow times. In contrast, One replication method and full replication exhibit higher flow times, with full replication showing the most significant delays due to increased waiting time. Additionally, the mean and standard deviation markers confirm that proactive replication not only achieves lower flow times on average but also maintains more consistent performance.

## 4.3. Evaluating the Utility of a Replica

The number of replicas per job are insufficient to evaluate replication's impact. Thus, we introduced a metric assessing transfer usefulness (2) by computing the ratio between the total execution time of tasks that used the replica on the target node and the replica transfer time.

$$ \text{Transfer usefulness} = \frac{\text{Total execution time of tasks using the replica on the target node}}{\text{Replica transfer time}} \quad (2) $$
<figure><img src="image_2.png" /><figcaption>FIGURE 1 – CDF of the flow time</figcaption></figure>

The interpretation is straightforward: the higher the ratio, the more cost-effective the replica transfer is. A ratio above 1 indicates that the execution time saved is lower than the transfer time, making replication less beneficial. This metric can be a good indicator for determining which transfers are costly and whether the data is being used efficiently.

<table><caption>TABLE 2 – Inverted values for the comparison of replication strategies based on replica utility</caption><thead><tr><th></th><th>Full Replication</th><th>Pro-Active Replication</th><th>One Replica</th></tr></thead><tbody><tr><th>Mean</th><td>1.96</td><td>23.88</td><td>1719.77</td></tr><tr><th>Std</th><td>3.22</td><td>126.49</td><td>3534.84</td></tr><tr><th>Max</th><td>40.06</td><td>3824.95</td><td>35732.24</td></tr><tr><th>Min</th><td>0.019</td><td>0.163</td><td>0.166</td></tr></tbody></table>

As shown in Table 2, it is evident that with this metric, the **One Replica** method provides significantly better performance compared to the other replication strategies. The high mean value suggests that the replica transfers in this approach are highly efficient. Moreover, our approach achieves results that are very close to those of the One Replica method, demonstrating its effectiveness in optimizing replica utilization.

## 5. Conclusion and perspectives

Through this paper, we presented a preliminary study on selecting a replication method that optimizes data transfer costs while delivering promising performance levels. We demonstrated that by introducing a condition to limit replication to dataset with a significant impact on performance, transfers can be reduced by over 60% while still ensuring a reasonable flow time. However, we also observed that, in terms of execution time, our approach remains somewhat distant from the optimal case, which is full replication (an increase of more than 70% in execution time for certain jobs). These insights are crucial for the continuation of this work, as they help us better understand and identify the key performance indicators (KPIs) needed to define an effective replication policy. Additionally, the key performance indicators (KPIs), such as flow, waiting time, and the utility of a replica, discussed in this paper, provide useful metrics for evaluating the impact of these policies.
## Acknowledgments

Our work is done in the context of the INRIA-Qarnot-ADEME Pulse project
https://www.inria.fr/en/pulse.

Some of the experiments in this work were carried out using the Grid'5000 testbed, supported by a scientific interest group hosted by Inria and including CNRS, RENATER and several Universities as well as other organizations (see https://www.grid5000.fr).

## 6.

## Bibliographie

1. Abad (C. L.), Lu (Y.) et Campbell (R. H.). – DARE : Adaptive Data Replication for Efficient Cluster Scheduling. – In *2011 IEEE International Conference on Cluster Computing*, pp. 159–168, Austin, TX, USA, septembre 2011. IEEE.

2. Breitbach (M.), Schafer (D.), Edinger (J.) et Becker (C.). – Context-Aware Data and Task Placement in Edge Computing Environments. – In *2019 IEEE International Conference on Pervasive Computing and Communications (PerCom)*, pp. 1–10, Kyoto, Japan, mars 2019. IEEE.

3. Bsoul (M.), Otoom (A. F.), Abdallah (A. E.) et Hamadneh (N.). – An enhanced bandwidth hierarchy based replication strategy for dynamic replication in data grid. *MGS*, vol. 13, n2, juillet 2017, pp. 163–175.

4. Carlson (J.). – *Redis in action*. – Simon and Schuster, 2013.

5. Cherrueau (R.-A.), Delavergne (M.), Van Kempen (A.), Lebre (A.), Pertin (D.), Balderrama (J. R.), Simonet (A.) et Simonin (M.). – EnosLib : A Library for Experiment-Driven Research in Distributed Computing. *IEEE Trans. Parallel Distrib. Syst.*, vol. 33, n6, juin 2022, pp. 1464–1477.

6. Loygue (P.), Al Agha (K.) et Pujolle (G.). – Carbon footprint of cloud, edge, and Internet of Edges. *Ann. Telecommun.*, vol. 80, n1-2, février 2025, pp. 153–169.

7. Park (S.-M.), Kim (J.-H.), Ko (Y.-B.) et Yoon (W.-S.). – Dynamic Data Grid Replication Strategy Based on Internet Hierarchy. In : *Grid and Cooperative Computing*, éd. par Kanade (T.), Kittler (J.), Kleinberg (J. M.), Mattern (F.), Mitchell (J. C.), Nierstrasz (O.), Pandu Rangan (C.), Steffen (B.), Sudan (M.), Terzopoulos (D.), Tygar (D.), Vardi (M. Y.), Weikum (G.), Li (M.), Sun (X.-H.), Deng (Q.) et Ni (J.), pp. 838–846. – Berlin, Heidelberg, Springer Berlin Heidelberg, 2004. Series Title : Lecture Notes in Computer Science.

8. Zhao (W.), Xu (X.), Xiong (N.) et Wang (Z.). – A Weight-Based Dynamic Replica Replacement Strategy in Data Grids. – In *2008 IEEE Asia-Pacific Services Computing Conference*, pp. 1544–1549, Yilan, Taiwan, décembre 2008. IEEE.