# Multi-Level Elasticity for Wide-Area Data Streaming Systems: A Reinforcement Learning Approach
## 8 Evaluation
### Results with Different Objectives  
A classic advantage of RL-based solutions is their ability to determine optimal policies given a specification of _what_ the user wants to achieve, instead of _how_ to achieve it. In this set of experiment, we investigated the flexibility of our RL-based policy in optimizing different trade-offs among resource usage cost and application requirements satisfaction. Specifically, we ran experiments with different configurations of weights for the cost function expressed in (1). For the application-level elasticity, we used both the threshold-based and the RL-based policies.  
The results we obtain are reported in Table 2. We first considered an extreme configuration in which we almost only optimize the resource usage cost (i.e., \(w_{d}=0.1,w_{c}=0.9\)). In this setting, E2 DF only uses 3.1 computing nodes on average, but the application performance is really poor: with the threshold-based scaling policy, the target response time is exceeded almost 80% of the time; with the RL-based policy for the ACS, the performance is better but still bad, with the response time constraint violated 40% of the time.  
Then, we tried less unbalanced settings, with \(w_{d}=0.3\) and \(w_{c}=0.7\) as well as \(w_{d}=0.7\) and \(w_{c}=0.3\), and the perfectly balanced setting with \(w_{d}=w_{c}=0.5\). As expected, as the satisfaction of the  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**ICS Policy** & **ACS Policy** & R\({}_{max}\)** Violations** & **Reconfigurations** & **Replicas** & **Active Nodes** \\ \hline (Static) & Threshold & 0.01\% & 3.21\% & 17.84 & 36.00 \\ (Static) & RL & 0.15\% & 1.02\% & 18.32 & 36.00 \\ Simple & Threshold & 0.01\% & 3.26\% & 17.83 & 16.57 \\ Simple & RL & 0.15\% & 1.04\% & 18.32 & 16.82 \\ RL & Threshold & 0.17\% & 3.20\% & 17.63 & 9.66 \\ RL & RL & 0.17\% & 1.10\% & 18.25 & 9.76 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Results with different combinations of policies for the ICS and the ACS. We report the percentage of time in which the target response time \(R_{max}\) is violated, the percentage of time in which the ACS decides to reconfigure the application, the average number of operator replicas, and the average number of used computing nodes.  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**ICS Policy** & **ACS Policy** & R\({}_{max}\)** Violations** & **Reconfigurations** & **Replicas** & **Active Nodes** \\ \hline \(w_{d}=0.1,w_{c}=0.9\) & Threshold & 79.20\% & 0.91\% & 6.13 & 3.14 \\ \(w_{d}=0.3,w_{c}=0.7\) & Threshold & 8.45\% & 2.88\% & 16.60 & 8.57 \\ \(w_{d}=0.5,w_{c}=0.5\) & Threshold & 1.37\% & 3.09\% & 17.35 & 9.10 \\ \(w_{d}=0.7,w_{c}=0.3\) & Threshold & 0.17\% & 3.21\% & 17.63 & 9.66 \\ \(w_{d}=0.9,w_{c}=0.1\) & Threshold & 0.01\% & 3.25\% & 17.75 & 10.23 \\ \(w_{d}=0.1,w_{c}=0.9\) & RL & 41.52\% & 0.69\% & 15.81 & 8.15 \\ \(w_{d}=0.3,w_{c}=0.7\) & RL & 0.72\% & 1.09\% & 18.91 & 9.87 \\ \(w_{d}=0.5,w_{c}=0.5\) & RL & 0.20\% & 1.18\% & 18.64 & 9.74 \\ \(w_{d}=0.7,w_{c}=0.3\) & RL & 0.17\% & 1.10\% & 18.25 & 9.76 \\ \(w_{d}=0.9,w_{c}=0.1\) & RL & 0.16\% & 1.07\% & 18.32 & 10.33 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Results with different objectives for our RL-based ICS policy. We report the percentage of time in which the target response time \(R_{max}\) is violated, the percentage of time in which the ACS decides to reconfigure the application, the average number of operator replicas, and the average number of used computing nodes.  
application demand for resources is weighted more, the number of response time violations quickly decreases, whereas the amount of allocated nodes increases, up to nine. As a final scenario, we used the extreme setting with \(w_{d}=0.9\) and \(w_{c}=0.1\), almost ignoring the resource usage cost. In this case, the amount of allocated nodes is slightly more than 10 on average, but the beneficial effect on the application performance is negligible, as illustrated in Figure 5.