# Multi-Level Elasticity for Wide-Area Data Streaming Systems: A Reinforcement Learning Approach
## 8 Evaluation
### Simulation Setup  
As a reference application, we considered a simple _WordCount_ topology, composed of one (or more) data sources and four processing operators: splitter, filter, counter, and consumer. The _data source_ emits sentences, which are split into words by the first operator; for each tuple coming into the _splitter_, five new tuples on average are emitted. The next operator is a _filter_, which drops about 60% of the incoming word stream. The final two operators, i.e., _counter_ and _consumer_, respectively, keep a count of the frequency of each word, and publish the most frequent ones. To simulate the behavior of a real application [8], we modeled each operator as a M/D/1 queue, with service rate \(\mu=330\) tuple/s. Each operator can be executed using up to 20 parallel replicas to keep the total application processing latency within \(R_{max}=60\) ms. The data source emission rate ranges from 0 to 600 tuples per second, following the daily pattern shown in Figure 3. We simulated the behavior of the system over \(100,000\) min (about two months).  
For our evaluation, we considered a wide-area infrastructure that comprises three geographically distributed micro-data centers: _dc1_, _dc2_, and _dc3_. In our framework, each data center is modeled as a region, and thus will be controlled by a RM. We assumed that we can run up to 50 computing nodes  
Figure 3: Workload used in our simulations, which is characterized by a daily pattern.  
for the DSP system within each region. Each node is a virtual machine with two CPU cores, thus can host at most two operator replicas. We simulated the following network latencies between the different regions: \(d_{(d\in L,d\in\Omega)}=2\) ms, \(d_{(d\in L,d\in\Omega)}=2\) ms, and and \(d_{(d\in\Omega,d\in\Omega)}=1\) ms.  
In E2 DF, the planning components are activated periodically, every minute of simulated time, and we considered that all the adaptation actions enacted by the framework are completed within the next minute. We compared the behavior of the proposed RL policy for the ICS against a baseline strategy, where computing resources are statically provisioned, and against the simple provisioning policy, described in Section 6.1. We used the following values in the RL-based policy: \(\gamma=0.99\), \(\alpha=0.1\), \(\bar{u}=0.1\). In particular, the discount factor \(\gamma\), being close to 1, allows the RL agent to act with foresight, while the learning rate \(\alpha\) enables the RL-based policies to continuously yet slowly adapt to (possibly) varying system dynamics. The \(\bar{u}\) parameter aims to find a good trade-off between model approximation and number of system states; indeed, both these features depend on the utilization quantum \(\bar{u}\). We considered different settings for the cost function weights, exploring distinct trade-offs among application requests satisfaction and resource usage. As regards the ACS, we considered two adaptation policies, namely the threshold-based policy (Section 7.1) and the RL-based one (Section 7.2). The threshold-based policy aims to keep the utilization of each operator replica between 50% and 75%, therefore, it uses as parameters \(U_{s-out}=0.75\) and \(c=0.75\). The RL-based policy used by the OMs uses a balanced configuration of weights for the cost function, i.e., \(w_{rcf}=w_{perf}=w_{res}=\frac{1}{3}\), meaning that, for the application, it is equally important to not exceed the response time bound and to avoid resource wastage.