# Simulating and Evaluating a Real-World ElasticSearch System Using the RECAP DES Simulator
## 4 Modelling and Simulation of Opening.io
### Application and Workload Propagation Models  
Application behaviour for the simulation experiment is realised by implementing a modelling concept that captures data flow through multiple interconnected, distributed components (nodes) in the Opening.io ES search engine.  
Opening.io's architecture contains a set of VMs which can host one or two nodes (data nodes only or data nodes with ES nodes) as shown in Figure 1. Each data node contains data files that are indexed by the ES node. The ES node takes advantage of the indexing to speed up the redirection of each query to the data nodes that contain the targeted data files. Requests considered by the Opening.io ES search engine are characterised by the fact that their entire result resides in only one data node. Therefore, the ES node forwards the query and gets the result from a unique data node.  
The ES search engine is characterised by a distributed architecture with parallel request processing and coordination behaviour. The ES node forwards the query to the targeted data node, collects the result and returns it as the result of the query. The ES is also co-hosted in the same VM as one of the data nodes (data node 1) to which it sends a query/gets results. While this has the advantage of avoiding the latency of inter-VM network connections, it might lead to an overload of the CPU and RAM usage of the ES and the data node (see Figure 1).  
Events and task in the RECAP DES simulator are defined by cloudlets. A cloudlet represents a submitted job. As such, to simulate the Opening.io workload, each query is modelled as a series of cloudlets moving through the system's nodes (as shown in Figure 2).  
Provided that a node is capable of running a cloudlet, the cloudlet's execution time is determined by (i) the cloudlet's overall computational cost, (ii) the total amount of available CPU per VM, (iii) the number of cores the cloudlet can use in parallel and (iv) the amount of CPU and RAM instructions the cloudlet can use at any given time.  
\begin{table}
\begin{tabular}{c l l l l} \hline \hline
**VM-ID** & **Data Node** & **CPU (\#Cores)** & **RAM (GB)** & **Storage (TB)** \\ \hline VM0 & ES node \& Data node 1 & 4 & 64 & 1 \\ \hline VM1 & Data node 2 & 4 & 64 & 1 \\ \hline VM2 & Data node 3 & 6 & 64 & 1 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Opening.io VM features.  
In ES, the workload is distributed among the nodes depending on a variety of criteria; e.g., the data can be distributed based on the frequency of access to a particular type of data that resides in a particular node in the system. In the case of Opening.io, the workload (requests) is distributed over the data nodes depending on the type of query they address (three types based on their service time). Figure 2 shows the application and workload propagation models. As shown, a collection of cloudlets is created and executed sequentially when a query is launched. The first cloudlet is executed at the ES node. Then, another cloudlet is sent to the targeted data node with the appropriate file (query results). Finally, another cloudlet is sent from the data node to the ES as the result of the query. Therefore, our simulation produces a total number of cloudlets equal to \(Cloudlet\_no=2+n\) to model a query load, where \(n\) is the number of data nodes queried by ES. In the case of Opening.io architecture, \(n=3\).