# Multi-Level Elasticity for Wide-Area Data Streaming Systems: A Reinforcement Learning Approach
## 2 Related Work
### System Architectures  
Approaches that enable elasticity are often, either explicitly or implicitly, architected as self-adaptive software systems based on the MAPE loop, which is a well known pattern to design self-adaptive systems [16]. Following this model, in response to (observed or predicted) changes, the application deployment is adapted by scaling the number of operator replicas or computing resources or by relocating some operators (or eventually their replicas) onto the computing resources. In the latter case, the set of computing nodes can be also elastically scaled-in/out as needed. Most of the existing system architectures consider a centralized management solution, where a single coordination entity exploits its global knowledge about the entire system state to plan the proper adaptation actions (e.g., [6; 7; 17; 18; 19; 20; 21]). Although this approach can potentially achieve a global optimum adaptation strategy, it may not be suitable for a wide-area distributed environment, because of the tight coupling among the system components and the fact that a central manager represents a bottleneck in a large-scale system due to monitoring and planning overheads. Conversely, other solutions rely on decentralized adaptation planners that exploit a limited local view of the system, thus overcoming the scalability issues due to the management of global information. In this case, the vast majority of the proposed approaches rely on a fully decentralized architecture (e.g., [9; 10; 22; 23]). Although these decentralized solutions appear to be appealing for wide-area distributed environments, the lack of coordination among the decentralized planning agents may cause frequent reconfiguration decisions, which can cause instability that compromises the DSP application performance, as shown in [12]. Differently from the above approaches, we consider a hierarchical distributed architecture, which can take the best of centralized and fully decentralized architectures, thus improving performance and scalability without compromising stability. A hierarchical control scheme has been previously adopted by [PERSON] et al. [24] to address the placement and relocation at runtime of the DSP operators. In their solution, there are multiple coordinators organized in a hierarchical tree, and each coordinator is in charge of a subgraph of the DSP application and of the underlying computing network. However, their control knob is based on redistributing the DSP operators, without scaling the number of operator replicas or computing resources. The system architecture we propose in this paper is built on our previous works [8; 13], where we presented a hierarchical decentralized architecture to control elasticity first at the DSP application level [8] and then also at the infrastructure level [13]. Here, we propose and evaluate new control policies for the infrastructure elasticity. Moreover, we investigate the benefits of a loosely coupled system architecture for controlling elasticity both at the application and infrastructure level. In particular, we show the the intertwined effects of autonomous policies operating at different levels of the system architecture. Few solutions have so far explicitly considered the run-time adaptation of DSP applications in fog and cloud distributed environments, where the ever increasing amount of resources at the network periphery can be exploited to improve the application latency. Distributed Storm [12] extends the Apache Storm architecture with decentralized monitoring and planning components to deal with a wide-area distributed operating environment. An extended framework based on Apache Storm and with a centralized planning component that determines the application redeployment to reduce communication latencies has been later presented by [PERSON] et al. [25]. DRS [19] is a dynamic resource scaling framework for cloud-based DSP applications, which has been integrated with Apache Storm.