<figure><img src="image_1.png" /></figure>

HAL
open science

# Data Stream Processing Effectiveness in Heterogeneous Computing Environments

Khaled Arsalane, Guillaume Pierre

<figure><img src="image_2.png" /></figure>

## To cite this version:

Khaled Arsalane, Guillaume Pierre. Data Stream Processing Effectiveness in Heterogeneous Computing Environments. SAC 2026 - 41st ACM/SIGAPP Symposium On Applied Computing, ACM, Mar 2026, Tessaloniki, Greece. hal-05390665

HAL Id: hal-05390665

https://hal.science/hal-05390665v1

Submitted on 1 Dec 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.
# Data Stream Processing Effectiveness in Heterogeneous Computing Environments

Khaled Arsalane

Univ Rennes, Inria, CNRS, IRISA

Rennes, France

khaled.arsalane@irisa.fr

Guillaume Pierre

Univ Rennes, Inria, CNRS, IRISA

Rennes, France

guillaume.pierre@irisa.fr

## Abstract

Distributed Stream Processing (DSP) systems face significant challenges when deployed in heterogeneous environments such as fog computing infrastructures. While these systems were designed for homogeneous cloud environments, their deployment across devices with varying capabilities introduces performance considerations that are not well understood. We present a systematic evaluation of Apache Flink's performance when scaling stateful operations across heterogeneous hardware. Through extensive experiments on a geo-distributed testbed comprising bare-metal servers, virtual machines, and Raspberry Pi devices, we examine how different TaskManager configurations affect throughput and resource utilization. Our results reveal that Flink's default configuration mechanisms can lead to significant performance limitations in heterogeneous environments, particularly due to its resource-agnostic task assignment and hash-based partitioning strategy. We identify optimal resource configurations for different node types and quantify the performance impact of various deployment strategies. These findings highlight important considerations for DSP system design in increasingly heterogeneous computing environments.

## CCS Concepts

* Computer systems organization → Cloud computing; * Network works → Cloud computing; Network resources allocation.

## Keywords

Data Stream Processing, geo-distributed environment, stateful operators, cloud continuum.

## 1 Introduction

The computing resources which constitute the edge-to-cloud continuum are inherently heterogeneous [30]. Besides the obvious resource disparity between cloud servers and edge devices [1, 28], computational requirements and energy constraints within the edge itself also often commend the usage of edge resources with heterogeneous computational capabilities [17, 27, 34] and network connectivity properties [9, 19].

Efficiently exploiting heterogeneous resources is difficult as applications may span nodes in multiple locations. To exploit the available resources to the best of their ability, applications must balance their workload such that each node is given tasks to execute that have proportional complexity to its capabilities [13].

A popular family of edge-cloud applications which needs to tackle the heterogeneity challenge is Data Stream Processing (DSP) [26]. DSP frameworks enable their users to define and execute continuous computation over unbounded data streams such

as those produced by IoT devices. These systems allow developers to specify processing pipelines that transform, aggregate, and analyze data as they are ingested, enabling real-time insights and actions. Deploying DSP systems in the edge-cloud continuum is very popular as it allows one to process data in close proximity to their sources, thereby reducing processing latency and the pressure on long-distance networks [8, 31].

However, modern DSP systems such as Apache Flink were designed for exploiting high-performance clusters with homogeneous computing resources [6]. Therefore, using them in the context of heterogeneous edge-cloud nodes places them out of their comfort zone and raises fundamental questions regarding the way to execute them efficiently using heterogeneous resources spanning at least one order of magnitude in their processing capabilities [16].

In this paper we address the following questions: (1) is it possible for Apache Flink to effectively exploit the capabilities of individual servers with very heterogeneous performance? ; (2) how should different types of machines be configured to deliver the best possible DSP throughput?; and (3) how do DSP applications perform when their operators are deployed on a heterogeneous mix of machines?

We conduct extensive experimental evaluations based on four very different types of machines: a homogeneous set of high-end bare-metal servers, two different types of virtual machines, and a set of Raspberry Pi 4B single-board computers. To preserve the generality of our findings we measure the throughput of a stateful "join" operator using Apache Flink under synthetic workloads and a wide variety of system configurations.

Our work identifies three main findings:

(1) Simply declaring the available number of CPU cores and memory capacity in the default Apache Flink configuration is not enough to obtain the maximum throughput.

(2) In many cases, deploying multiple single-slot TaskManagers consistently outperforms a single large multi-slot TaskManager by up to 350%.

(3) When deploying an operator on a heterogeneous set of machines, the throughput of each operator instance aligns with that of the slowest instance. This means for example that the performance of a system configuration with two fast machines will significantly *drop* if we add a third machine with lower level of performance.

These findings provide both immediately actionable insights for practitioners deploying Flink in diverse environments, and a longer-term research roadmap for the design of future DSP system that can effectively exploit arbitrary sets of heterogeneous resources.

The remainder of the paper is organized as follows: Section 2 presents the technical background and our system design. Section 3
<figure><img src="image_3.png" /><figcaption>Figure 1: A stream processing pipeline with multiple task slots.</figcaption></figure>

details the experimental system and procedure. Then Section 4 evaluates how to obtain the best performance out of a single machine, and Section 5 presents the challenges of scaling out using heterogeneous types of nodes. Finally, Section 6 discusses the related work and Section 7 concludes the paper.

## 2 Background

Apache Flink is a distributed stream processing framework designed for high-throughput, low-latency analysis of unbounded data streams [24]. Flink provides exactly-once processing semantics through its fault tolerance mechanisms, making it suitable for critical data processing applications [12]. Figure 1 presents the deployment of a Flink application and the underlying system.

*Operators* A Flink application is structured as a Directed Acyclic Graph (DAG) of operators where data flows from one or more source node(s) to one or more sink node(s). Each intermediate node is an operator which applies a transformation on data or triggers a user-defined function. Operators can be stateless (applying transformations without maintaining state, such as filters) or stateful (maintaining information across events, such as windows, aggregations, or joins).

*Task slots* The tasks of an operator execute in *Task Slots* (TS), which implement a logical division of available computational resources. Task slots are the smallest computational unit in Flink.

To allow operators to scale beyond the capabilities of a single TS, operators are designed to be distributed over a group of task slots. For instance, in Figure 1 the second operator of the pipeline has been deployed on two separate task slots. This requires a variety of routing strategies for distributing messages originating from upstream operators to the right slot(s) [4]. The most common routing strategy is KeyedStreams which uses consistent hashing to distribute records with the same key to the same operator replica [3]. This way, stateful operators can maintain per-key state without synchronization overhead between the operator instances.

*TaskManagers* Task slots execute within *Task Managers* (TM), which are Flink's worker processes [2]. Each TM may execute one or more TS. Task Managers can run in containers and thus, they can be distributed across several machines.

TMs can be configured with different resource allocations based on the available hardware. Key configuration parameters include memory allocation and number of CPU cores per TM. These resources can then be managed by SlotSharingGroups to further specify resources per task slot [5].

The relationship between these parameters directly impacts performance. For instance, configuring multiple slots on a TM with limited number of cores may lead to resource contention as multiple tasks end up competing for CPU time.

## 3 Experimental Evaluation Procedure

### 3.1 System Design

Figure 2 shows the system architecture used for our evaluations. Our heterogeneous testbed spans two geographically distributed sites connected via a mesh network overlay. The first site consists of seven high-performance “Paradoxoe” servers from the Grid’5000 experimental HPC testbed [10, 20]. The second site hosts ten Raspberry Pi 4B (RPi) single-board computers that form a custom edge cluster [29]. To establish connectivity between the HPC testbed and the Raspberry Pi cluster, we deploy Tailscale to provide a mesh network overlay across both sites [33]. The latency between the two sites averages 20 ms, with a maximum of 25 ms.

Within this distributed infrastructure, we assign specific roles to different node types in our Kubernetes cluster. Producer nodes host data-generating applications including Kafka brokers and load generators. Consumer nodes host data-processing applications, specifically Flink TaskManagers and distributed storage components. From the Grid’5000 servers, we configure one server as a control node hosting the Kubernetes control plane, and three servers as producer nodes. Of the remaining three Grid’5000 servers, one operates as a bare-metal consumer node (labeled “BM”), while the other two host virtual machines that serve as consumer nodes. The first VM host runs four large virtual machines (“VM-L”: 8 cores, 32 GB RAM each), and the second hosts ten small virtual machines (“VM-S”: 2 cores, 8 GB RAM each). All ten Raspberry Pi 4B nodes operate as consumer nodes (labeled “RPi”). Table 1 summarizes the resource specifications for each of the four consumer node types where Flink TaskManagers are deployed.

We orchestrate this platform using the lightweight k3s (v1.32.2+k3s1) distribution [25]. The control node hosts the Flink
<figure><img src="image_4.png" /><figcaption>Figure 2: Kubernetes cluster deployment across geographically distributed sites.</figcaption></figure>

<table><caption>Table 1: Heterogeneous Data Consumer Node Configurations.</caption><thead><tr><td>Name</td><td>Node Type</td><td># nodes</td><td>CPU</td><td># cores/node</td><td>RAM</td><td>Network</td></tr></thead><tbody><tr><td>BM</td><td>Bare metal server</td><td>1</td><td>Intel Xeon Gold 5320 2.2 GHz</td><td>52</td><td>384 GB</td><td>25 Gbps</td></tr><tr><td>VM-L</td><td>Virtual machine</td><td>4</td><td>Intel Xeon Gold 5320 2.2 GHz</td><td>8</td><td>32 GB</td><td>25 Gbps</td></tr><tr><td>VM-S</td><td>Virtual machine</td><td>10</td><td>Intel Xeon Gold 5320 2.2 GHz</td><td>2</td><td>8 GB</td><td>25 Gbps</td></tr><tr><td>RPi</td><td>Raspberry Pi 4 Model B</td><td>10</td><td>Broadcom BCM2711 1.8 GHz</td><td>4</td><td>4 GB</td><td>1 Gbps</td></tr></tbody></table>

<figure><img src="image_5.png" /><figcaption>Figure 3: Data workflow of the benchmark application.</figcaption></figure>

(v1.20) JobManager, monitoring stack, and MinIO controller. Producer nodes run Kafka brokers and load generators. All consumer nodes (bare-metal, VM-L, VM-S, and RPi) host Flink TaskManagers. Bare-metal and VM-L nodes additionally host MinIO object storage tenants where Flink can save its periodic checkpoints. We configure Flink to generate checkpoints at 4-second periodicity. Savepoints used during Flink reconfigurations are stored there as well.

## 3.2 Benchmark Application

We exercise Flink using the stream processing application depicted in Figure 3. This application consists of two source operators which generate separate streams of events, a stateful join operator which processes and matches events from both sources, and a sink operator which simply discards the results. Each source operator reads events from a dedicated Kafka topic. Events are then partitioned by id using the KeyedStream routing strategy: this ensures that events with the same id are processed by the same parallel instance of the join operator. The join operator maintains a tumbling window of all events received during the previous 1 second, and emits a record when it can match two input records with the same ID.

## 3.3 Workload Injection

We generate a synthetic workload using two instances of the Theodolite load generator [23]. Each instance emulates a set of 175,000 IoT sensors that generate one record per second each and write these records in dedicated Kafka topics. Each record is a tuple in the form of < sensor_id, timestamp, value >.

## 3.4 Experimental Procedure

Each experiment generates 2 × 175,000 = 350,000 records/second, and measures the actual ingestion rate of the evaluated Flink operator. This workload is designed to saturate most Flink setups, allowing us to measure the Maximum Sustainable Throughput (MST) of various configurations.

We execute each evaluation during a 300-second period before issuing a Flink reconfiguration to test a different setup. We repeat the full evaluation process five times, and report the average MST values across these five runs. The figures present error bars indicating variability across runs, though the maximum observed standard deviation remains below 2,000 records per second at any parallelism level—representing less than 0.5% variation relative to the measurement scale of up to 400,000 records per second.

# 4 RQ1: Resource Configuration

The first research question addressed in this paper relates to the optimal Flink configuration on each of the four types of machines. Arguably, obtaining the best possible throughput out of a high-end datacenter server, a virtual machine and a single-board computer may not require the same resource allocation parameters.
<figure><img src="image_6.png" /><figcaption>Figure 4: Impact of CPU and memory allocation on the throughput of a single slot on different node types.</figcaption></figure>

## 4.1 CPU and Memory Resources Per Task Manager

The first question we address in this section is to identify the relation between the CPU and memory resources being assigned to a single slot and the throughput of the operator which executes in this slot.

For each machine type we deploy a single Flink slot executing the join operator. We then systematically explore multiple combinations of CPU and memory resources ranging from 1 to 52 CPU cores and 1 GB to 32 GB of memory, adjusting the upper limits based on each node type's available resources. Figure 4 presents the throughput achieved by single-slot Task Managers under different resource allocations.

We first observe that the amount of allocated memory has limited impact on performance. As soon as the TM has access to sufficient memory to store a full data window, adding more memory provides little performance benefits. Note that this observation may be dependent on the nature of the observed operator; other operator functions may have different behavior [32].

Second, an increased number of CPU cores delivers improved throughput, but with strongly diminishing returns. Task Managers with 1 core and 4 GB RAM consistently deliver the highest throughput per core for BM, VM-L and VM-S.

Interestingly, RPi nodes demonstrate significantly lower throughput than other node types, despite having more CPU cores than VM-S nodes. This performance gap is directly attributable to differences in CPU architecture and memory subsystem characteristics, highlighting that raw core count alone is insufficient to predict performance for data stream processing workloads. This type of node achieves the best throughput per core when configured with 2 cores and at least 1 GB RAM.

## 4.2 Throughput Scaling Across Node Types

Since the most efficient TS configurations require small numbers of cores and amount of memory, making the best possible use of a full machine requires multiple TS per node. However, simply
<figure><img src="image_7.png" /><figcaption>Figure 5: Filling different node types with increasing number of TMs and one TS per TM.</figcaption></figure>

<figure><img src="image_8.png" /><figcaption>Figure 6: Two opposite TaskManager and Task Slot configuration strategies on underlying machine cores.</figcaption></figure>

increasing the number of TS per node does not necessarily deliver linear throughput gains.

Figure 5 shows how the aggregate throughput scales when filling a single machine with increasing number of TMs, using one TS per TM. For each type of node, we deploy up to the maximum number of single-slot TM configured with 1 core and 4 GB of RAM each: the BM node can host up to 52 single-slot TMs whereas VM-L nodes can host up to 8, and VM-S up to 4. The RPi nodes can accommodate up to 2 TM with a dual-core TS each, considering that dual-core TS delivered more throughput per core on this type of machine.

The results show linear throughput scaling on RPi and both types of virtual machines, with throughput increasing proportionally to the number of TMs. However, the BM node exhibits a notable deviation from this pattern. As the number of TMs increases beyond a certain threshold, throughput reaches a plateau and eventually fails to increase despite the fact that more and more hardware resources are being utilized.

## 4.3 Single-Slot vs. Multi-Slot TaskManagers

We identified limitations on the BM server which motivate further exploration into alternative ways to efficiently exploit its full hardware capabilities. Specifically, we want to examine whether dividing its resources differently between TMs and TS could provide better resource utilization and throughput performance.

Figure 6 illustrates two opposite approaches for organizing TaskManagers and Task Slots on a multi-core machine: Figure 6(a) shows a single TaskManager with multiple slots that associate all

task slots to the underlying CPU cores, whereas Figure 6(b) shows multiple single-slot TaskManagers that use the same CPU with each slot associated with one core. The results from these scenarios, along with intermediate configurations where TaskManagers are deployed with more than one slot and have more than one core per slot, are presented in Figure 7. The results are presented in the form of boxplots which illustrate the minimum, average and maximum throughput achieved across five runs.

**Single-core task slots:** Figure 7(a) compares three options based on single-core Task Slots: a single TM with 52 task slots, a configuration with 26 TMs and 2 task slots each, and a configuration with 52 TMs with 1 task slot each. The results reveal significant differences. The single-TM option achieves only approximately 100k records/s, suggesting a performance bottleneck when too many slots compete for resources within a single JVM process. Deeper investigations reveal strong contention at the JVM level for shared resources, including the overhead of garbage collection and saturation of memory bandwidth across the 52 concurrent task slots.

In contrast, the 26-TM configuration reaches 330k records/s, representing a 3.3x improvement. Configurations based on multiple TaskManagers can exploit significantly greater number of threads compared to single-TaskManager setups when aggregating threads across all JVMs, as each TaskManager runs its own JVM instance. This dramatic difference highlights the importance of distributing workloads across multiple JVM processes to improve resource isolation and reduce intra-process contention. The superior performance of the 26-TM configuration can be attributed to better workload distribution, requiring approximately 3x less CPU time per thread compared to single TaskManager setups, as the computational load is distributed across multiple JVM instances. Conversely, single TaskManager configurations exhibit lower overall CPU load percentage despite delivering significantly lower throughput, indicating inefficient resource utilization due to internal bottlenecks rather than hardware saturation.

Finally, the 52-TM configuration performs at 220k records/s, significantly worse than the 26-TM configuration, indicating that excessive process creation overhead and inter-process communication costs begin to outweigh the benefits of resource isolation.

**Dual-core task slots:** Figure 7(b) presents three configurations based on dual-core Task Slots: a single TM with 26 task slots, a configuration with 13 TMs and two TS each, and a configuration with 26 TMs and 1 TS each. The single-TM configuration exhibits similar performance limitations to its counterpart in Figure 7(a), confirming the presence of bottlenecks related to memory bandwidth and garbage collection when concentrating too many slots within a single JVM process. The 13-TM and 26-TM configurations achieve 342k and 350k records/s respectively, demonstrating that allocating two cores per slot can enhance performance when properly distributed across TMs. The marginal difference between these configurations suggests that the dual-core allocation reaches an optimal balance between computational resources per task and memory channel utilization, with the 26-TM configuration providing slightly better performance due to improved memory access patterns and reduced garbage collection frequency per JVM instance.
<figure><img src="image_9.png" /><figcaption>Figure 7: Throughput of different TM/TS configurations to fill a single BM server (52 cores / 386 GB).</figcaption></figure>

**Quad-core task slots:** Finally, Figure 7(c) examines two options based on quad-core Task Slots: a single TM with 13 TS, and 13 TMs with 1 TS each. We see that the single-TM option delivers poor performance whereas the 13-TM option achieves approximately 280k records/s. However, this level of performance still does not match that of dual-core TS options, indicating that beyond two cores per slot, the stateful join operator cannot effectively utilize additional CPU resources due to limitations in memory bandwidth per core and the inherently sequential nature of certain join operations. This plateau suggests that the memory subsystem becomes the limiting factor rather than CPU processing capacity, as evidenced by the declining performance despite increased core allocation.

An interesting observation is that single TaskManager setups consistently exhibit checkpoint durations and alignment times that are approximately 3x longer compared to multiple TaskManager configurations. This extended checkpoint overhead directly correlates with the observed 3× longer garbage collection times in single TaskManager setups, creating cascading effects on both performance and fault-tolerance properties.

These results challenge the intuitive assumption that it is best to create a small number of large TMs, to declare the available resources to Flink, and to let Flink manage the resources within the TM. The observed performance patterns rather suggest that optimal configurations balance JVM-level resource management with distributed processing benefits, as evidenced by the superior thread distribution, reduced CPU time per thread, and improved checkpoint performance achieved through multiple TaskManager deployments. These results lead to two key findings on how to use a powerful machine such as our BM node:

**Finding 1: Optimal memory allocation per TaskManager:** Our evaluation of CPU and memory configurations reveals that throughput-per-core is maximized with a 4 GB memory allocation for each TM. While allocating additional memory beyond 4GB shows minimal improvement, providing less than 4 GB severely impacts performance due to increased garbage collection overhead.

**Finding 2: TaskManager and Task Slot configuration:** Deploying multiple single-slot TaskManagers consistently outperforms single large multi-slot TaskManagers by up to 3.5× on bare-metal servers. This improvement results from multiple complementary factors: better thread distribution across JVM instances

(achieving approximately 3× less CPU time per thread), reduced garbage collection overhead (3× shorter GC times), and improved checkpoint performance (3× faster checkpoint duration and alignment times). The optimal configuration uses 26 TaskManagers with dual-core task slots, achieving 350k records/s compared to 100k records/s with a single 52-slot TaskManager.

Based on these findings, we retain the following configurations for using each of our four node types at full capacity:

* RPi nodes use 2 TMs with 1 TS and 2 cores each;

* VM-S nodes use 2 TMs with 1 TS and 2 cores each;

* VM-L nodes use 8 TMs with 1 TS and 1 core each;

* BM nodes use 26 TMs with 1 TS and 2 cores each.

## 5 RQ2: Homogeneous vs. Heterogeneous Computing

We now examine Flink's behavior when extending processing across multiple machines. We compare performance scaling of the stateful operator when using multiple machines of the same type vs. heterogeneous environments based on different machine types.

Figure 8 presents throughput measurements for both scenarios. In Figure 8(a), we observe the scaling behavior when adding homogeneous nodes. The bare-metal nodes start at 350k records/second and achieve a maximum throughput of approximately 380k records per second. The surplus of 30,000 records can be attributed to the system processing buffered records that were placed on hold during the reconfiguration phase. All other machines (VM-L, VM-S, and RPi) show linear throughput scaling when increasing the number of machines, albeit at different absolute performance levels.

However, the BM+RPi experiment, a single bare-metal node achieves 342k records per second. However, when we add a single Raspberry Pi, the system throughput dramatically drops to just 87k records per second—a 74.6% reduction. Adding a second Pi slightly improves performance to 99k records per second, but adding a third Pi causes throughput to drop again to 84k records per second. Similar patterns of severe performance degradation appear in other heterogeneous combinations.

This counter-intuitive behavior is attributable to Flink's utilization of a hash-based data routing strategy, which distributes workload uniformly across all TMs without taking into account their respective processing capabilities. In heterogeneous environments,
<figure><img src="image_10.png" /><figcaption>Figure 8: Scaling performance across several machines running at capacity.</figcaption></figure>

this means that slower nodes receive the same workload as faster ones, creating bottlenecks that severely degrade the overall system performance. Although it is understandable that adding a slow node cannot add much extra throughput to a fast one, it turns out that the presence of a single slow machine actively undermines the processing capability of the entire system.

**Finding 3: Hash-based routing creates performance bottlenecks in heterogeneous deployments.** Flink's default hash-based routing creates severe performance limitations in heterogeneous environments. By distributing data uniformly without considering node capabilities, slower nodes become bottlenecks that dramatically limit overall system throughput. As demonstrated in Figure 8(b), adding Raspberry Pi nodes to a more powerful machine does not just fail to improve performance—it causes throughput reduction of up to 75%. This indicates that Flink deployments in heterogeneous environments require different routing strategies that account for the individual processing capabilities of each node.

# 6 Related Work

Understanding and optimizing Data Stream Processing systems in heterogeneous environments is an active research area [14]. Efforts span multiple levels, from resource allocation for stateful operators to fundamental redesigns of Stream Processing Engines (SPEs) for edge computing.

Edge computing is characterized by the fact that edge nodes are often connected to each other, and to cloud servers, using networks with heterogeneous performance. Several research articles have focused on modeling [7] and/or optimizing [8, 11] the scalability of SPE systems in this type of environment.

A first research direction focuses on designing SPE systems that can operate on resource-constraint single-board computers such as Raspberry Pis. EdgEWisE is designed for the edge including engine-level, congestion-aware operation scheduling and a fixed-size worker pool to improve performance over systems like Apache Storm [18]. More recently, AgileDART builds upon concepts from EdgEWisE and reportedly achieves superior latency and scalability [15]. AgileDART utilizes a dynamic dataflow abstraction via DHT-based peer-to-peer overlays for autonomous operator management and a bandit-based path planning model to adapt data routing to heterogeneous and unreliable edge networks. Notably,

the evaluations of both EdgEWisE and AgileDART are realized using resource-constrained devices representative of the edge such as Raspberry Pi nodes. This underscores the importance of understanding performance on such hardware. Our systematic benchmarking of Flink, particularly its stateful Join operator, across a spectrum of hardware including bare-metal servers, VMs, and Raspberry Pi 4B devices, quantifies the performance characteristics and optimal TaskManager configurations for these distinct node classes.

Although most related work has studied the performance and scalability of stateless SPE operators, some authors specifically investigate stateful operators [21]. For instance, Justin proposes an elastic auto-scaler designed for stateful operators in Apache Flink [32]. Justin dynamically monitors CPU usage and the performance of storage operations, subsequently reconfiguring tasks by adjusting their parallelism and/or their specific memory configuration. Our work complements this work by systematically exploring a wide range of CPU and memory configurations per task slot across different parallelism levels, with a particular emphasis on how these configurations perform on heterogeneous types of machines. Although the authors of Justin did not directly explore the impact of hardware heterogeneity in their evaluation, they acknowledge limitations such as those imposed by Flink's hash-based partitioning and note the potential need for heterogeneous memory allocation between tasks—a direction for which our findings on configuring diverse machine types provide foundational insights.

This paper has highlighted the importance of revisiting Flink's data routing strategies to distribute computing tasks according to the individual capacity of each node. Recent work has proposed interesting directions to realize this using a dynamic data partitioning strategy [22]. The proposed method employs a scoring system that evaluates nodes based on both static hardware characteristics (e.g., CPU, memory capacity) and dynamic resource utilization, aiming to adjust data distribution ratios in real-time to mitigate load imbalances. While this helps route data more effectively according to node capacities, the paper does not detail how the relative weights should be chosen, in particular when dealing with stateful operators. Our research, in contrast, places a direct emphasis on benchmarking the impact of CPU and memory configurations within task slots dedicated to stateful operators. We check how these settings affect
performance and the ability to scale up, especially when running
on a cluster with different kinds of machines.

7 Conclusion

This paper presents a comprehensive evaluation of Apache Flink's
performance when deployed on heterogeneous computing nodes.
Our experiments revealed that Flink's default configuration leads to
significant performance limitations in heterogeneous environments,
primarily due to its resource-agnostic task assignment and data
partitioning strategies.

We found that the best performance in heterogeneous deploy-
ments is achieved with single-slot TaskManagers configured with at
least 2 cores and 4 GB of memory. However, even with this optimal
configuration, smaller nodes can become bottlenecks that limit the
performance of the entire system. The default hash-based routing
strategy in Flink creates performance limitations in heterogeneous
environments, as it does not consider the processing capacity of
the destination nodes.

Future work should focus on developing resource-aware stream
processing systems that can automatically adapt to heterogeneous
environments. This includes implementing adaptive data partition-
ing strategies and resource-aware task assignment.

Acknowledgments

Experiments presented in this paper were carried out using the
Grid'5000 testbed, supported by a scientific interest group hosted
by Inria and including CNRS, RENATER and several Universities
as well as other organizations (see https://www.grid5000.fr).

References

[1] Gayashan Amarasinghe, Marcos D. De Assuncao, Aaron Harwood, and Shanika Karunasekera. 2018. A data stream processing optimisation framework for edge computing applications. In Proc. *IEEE ISORC*.

[2] Apache Software Foundation. 2025. Apache Flink® – Flink Architecture. (2025). https://nightlies.apache.org/flink/flink-docs-release-1.20/docs/concepts/flink-architecture/.

[3] Apache Software Foundation. 2025. Apache Flink® – Flink Keyed Stream. (2025). https://nightlies.apache.org/flink/flink-docs-release-1.20/docs/dev/datastream/fault-tolerance/state/#keyed-datastream.

[4] Apache Software Foundation. 2025. Apache Flink® – Flink Task Lifecycle. (2025). https://nightlies.apache.org/flink/flink-docs-release-1.20/docs/interals task_lifecycle/.

[5] Apache Software Foundation. 2025. Apache Flink® – Slot Sharing Groups. (2025). https://nightlies.apache.org/flink/flink-docs-release-1.20/docs/ deployment/finegrained_resource/.

[6] Apache Software Foundation. 2025. Apache Flink® – Stateful Computations over Data Streams. (2025). https://flink.apache.org/.

[7] HamidReza Arkian, Guillaume Pierre, Johan Tordsson, and Erik Elmroth. 2020. An Experiment-Driven Performance Model of Stream Processing Operators in Fog Computing Environments. In Proc. *ACM SAC*.

[8] HamidReza Arkian, Guillaume Pierre, Johan Tordsson, and Erik Elmroth. 2021. Model-based Stream Processing Auto-scaling in Geo-Distributed Environments. In Proc. *IEEE ICCCN*. https://doi.org/10.1109/ICCCN52240.2021.9522236

[9] Khaled Arsalane, Guillaume Pierre, and Shadi Ibrahim. 2024. Toward Stream Processing Elasticity in Realistic Geo-Distributed Environments. In Proc. *IEEE IC2E*.

[10] Daniel Balouek, Alexandra Carpen Amarie, Ghislain Charrier, Frédéric Desprez, Emmanuel Jeannot, Emmanuel Jeanvoine, Adrien Lèbre, David Margery, Nicolas Niclausse, Lucas Nussbaum, Olivier Richard, Christian Pérez, Flavien Quesnel, Cyril Rohr, and Luc Sarzyniec. 2013. Adding Virtualization Capabilities to the Grid'5000 Testbed. In *Cloud Computing and Services Science*. Communications in Computer and Information Science, Vol. 367. Springer.

[11] Davaadorj Battulga, Daniele Miorandi, and Cédric Tedeschi. 2021. SpecK: Composition of Stream Processing Applications over Fog Environments. In Proc. *DAIS*.

[12] Paris Carbone, Stephan Ewen, Gyula Fóra, Seif Haridi, Stefan Richter, and Kostas Tzoumas. 2017. State Management in Apache Flink®: Consistent Stateful Distributed Stream Processing. Proc. *VLDB Endow*. 10, 12 (Aug. 2017).

[13] Valeria Cardellini, Patrizio Dazzi, Gabriele Mencagli, Matteo Nardelli, and Massimo Torquati. 2025. Scalable compute continuum. *Future Generation Computer Systems* 166 (May 2025).

[14] Valeria Cardellini, Francesco Lo Presti, Matteo Nardelli, and Gabriele Russo Russo. 2022. Runtime Adaptation of Data Stream Processing Systems: The State of the Art. *ACM Computing Surveys* 54, 115 (Sept. 2022).

[15] Cheng-Wei Ching, Xin Chen, Chaeeun Kim, Tongze Wang, Dong Chen, Dilma Da Silva, and Liting Hu. 2025. AgileDART: An Agile and Scalable Edge Stream Processing Engine. *IEEE Transactions on Mobile Computing* 24, 5 (2025).

[16] Marios Fragkoulis, Paris Carbone, Vasiliki Kalavri, and Asterios Katsifodimos. 2024. A survey on the evolution of stream processing systems. *The VLDB Journal* 33, 2 (2024).

[17] Kaihua Fu, Wei Zhang, Quan Chen, Deze Zeng, Xin Peng, Wenli Zheng, and Minyi Guo. 2021. QoS-aware and resource efficient microservice deployment in cloud-edge continuum. In Proc. *IEEE IPDPS*.

[18] Xinwei Fu, Talha Ghaffar, James C Davis, and Dongyoon Lee. 2019. EdgeWise: A better stream processing engine for the edge. In Proc. *USENIX ATC*.

[19] Morgan K. Geldenhuys, Jonathan Will, Benjamin JJ Pfister, Martin Haug, Alexander Scharmann, and Lauritz Thamsen. 2021. Dependable IoT data stream processing for monitoring and control of urban infrastructures. In Proc. *IEEE IC2E*.

[20] Grid'5000. 2025. Grid'5000 Paradoxe cluster. (2025). https://www.grid5000.fr/w/Rennes:Hardware#paradoxe.

[21] Vincenzo Gulisano, Alessandro V. Papadopoulos, Yiannis Nikolakopoulos, Maria Papatriantafilou, and Philippas Tsigas. 2017. Performance Modeling of Stream Joins. In Proc. *DEBS*.

[22] Chenglong He, Yan Huang, Chaoyong Wang, and Nian Wang. 2022. Dynamic data partitioning strategy based on heterogeneous Flink cluster. In Proc. *IEEE ICAIBD*.

[23] Sören Henning and Wilhelm Hasselbring. 2021. Theodolite: Scalability Benchmarking of Distributed Stream Processing Engines in Microservice Architectures. *Big Data Research* 25 (2021).

[24] Fabian Hueske and Vasiliki Kalavri. 2019. *Stream processing with Apache Flink: fundamentals, implementation, and operation of streaming applications*. O'Reilly Media.

[25] K3s Project Authors. 2025. K3s. (2025). https://k3s.io/.

[26] Vasiliki Kalavri. 2020. Why stream processing systems are now more relevant than ever. SIGOPS blog. (June 2020). https://www sigops org/2020/streams/.

[27] Ammar Kazem, Guillaume Pierre, and Laurent Longuevergne. 2025. Enhancing Environmental Observatories with Fog Computing. *Frontiers in Environmental Science* 13 (March 2025). https://doi.org/10.3389/fenvs.2025.1568016

[28] Adrián Orive, Aitor Agirre, Hong-Linh Truong, Isabel Sarachaga, and Marga Marcos. 2022. Quality of service aware orchestration for cloud-edge continuum applications. *Sensors* 22, 5 (2022).

[29] Raspberry Pi Foundation. 2025. Raspberry Pi 4B. (2025). https://www.raspberrypi.com/products/raspberry-pi-4-model-b/.

[30] Daniel Rosendo, Alexandru Costan, Patrick Valduriez, and Gabriel Antoniu. 2022. Distributed intelligence on the edge-to-cloud continuum: A systematic literature review. *J. Parallel and Distrib. Comput.* 166 (2022).

[31] Gabriele Russo Russo, Valeria Cardellini, and Francesco Lo Presti. 2023. Hierarchical Auto-scaling Policies for Data Stream Processing on Heterogeneous Resources. *ACM Transactions on Autonomous and Adaptive Systems* 18, 4 (Oct. 2023). https://doi.org/10.1145/3597435

[32] Donatien Schmitz, Guillaume Rosinosky, and Etienne Rivière. 2025. Justin: Hybrid CPU/Memory Elastic Scaling for Distributed Stream Processing. In Proc. *DAIS*.

[33] Tailscale. 2020. Tailscale. (2020). https://tailscale.com/blog/how-tailscale-works.

[34] Joel Wolfrath and Abhishek Chandra. 2022. Efficient transmission and reconstruction of dependent data streams via edge sampling. In Proc. *IEEE IC2E*.