# Multi-Level Elasticity for Wide-Area Data Streaming Systems: A Reinforcement Learning Approach
## 8 Evaluation
### Results with Different Combinations of Policies  
We first evaluated E2 DF considering different combinations of policies for the ICS and ACS. In these experiments, we used the following weights for the cost function: \(w_{d}=0.7\) and \(w_{c}=0.3\). Table 1 reports several metrics of interest corresponding to the different evaluated scenarios, whereas Figure 4 shows the application response time and the amount of used resources. To show the full potential of the policies, we discuss the E2 DF behavior when the RL agents have completed their learning process. We postpone to Section 8.4 considerations on their learning and convergence rate.  
**Static provisioning.** As a baseline scenario, we consider the case in which the number of active nodes in each region is statically defined as 12, and the application elasticity is driven by a simple threshold-based policy. (By preliminary experiments, we verified that this configuration guarantees enough resources to the application.) In this scenario, the application runs 17.8 operator replicas on average, violating the target response time only for 0.01% of the time, but reconfiguring (thus causing downtime) for 3% of the experiment. The RL-based policy for application-level elasticity is aware of the adaptation cost. Therefore, it allows reducing the reconfigurations to about 1%, by increasing the average number of running replicas to 18.3 and accepting slightly more response time violations (however, they are less than 1%). In these experiments, in which the infrastructures is statically provisioned, the total number of active computing resources is equal to 36. Figure 3(a),b show the different behavior of the two application-level elasticity policies, with the RL-based one avoiding too frequent oscillations.  
**Simple ICS provisioning policy**. We now consider the simple provisioning policy for the ICS presented in Section 6.1, which aims at avoiding resource wastage. In Figure 3(c),d, the overall application behavior is not significantly affected by the ICS policy. In Table 1, we can confirm this insight: with both threshold-based and RL-based policies for the ACS, the application performance is almost identical to the baseline scenario. However, the ICS policy is effective in dramatically reducing the amount of used computing resources; on average, it uses 16 nodes, instead of 36 nodes as in the baseline case.  
**Reinforcement Learning based ICS policy**. We compare the behavior of our RL-based ICS policy to that of the simple provisioning policy and to the baseline static scenario. In Figure 3(e),f it seems that the application behavior is not significantly affected. In Table 1, we note that, when using the threshold-based ACS policy, the percentage of time in which the response time requirement is violated is larger. This value is identical in the case of the RL-based ACS policy, which, however, reduces the number of adaptation actions (hence the downtime) by a third. Thus, while the application performance is overall only slightly affected by the RL-based ICS policy, we observe that the number of allocated computing resources is significantly reduced, to about nine. In other words, equipped with this policy, E2 DF is able to achieve a 50% reduction in terms of used computing resources without compromising the application performance.  
Figure 4: Application response time, number of operator replicas and active computing nodes during five simulated days at the end of the experiments, with different combinations of policies for ICS and ACS.  
Interestingly, our model-based RL solution at the infrastructure level does not suffer from instability during the learning phase, even when it is coupled with another RL agent at the application level. Moreover, the coupling of the two RL-based policies represents the most promising configuration among the evaluated ones; indeed, the RL-based ACS policy can be further tuned to achieved a different trade-off among the considered metrics to better suit the user's requirements (see [8] for a more detailed discussion).