## 3 Research Method
### Cluster Architecture and Setup  
This aspect represents common architectural patterns and features of CO frameworks that a cluster administrator must understand in order to be able to set up a running container cluster on top of a particular operating system and/or cloud provider infrastructure. \begin{table}
\begin{tabular}{|l|l|c|c|c|c|c|c|c|c|} \hline
**Aspects** & \multicolumn{10}{c|}{**Container Orchestration Frameworks**} \\ \hline
**Subspaces** & **Features** & \multicolumn{1}{c|}{**Container Orchestration Frameworks**} \\ \hline  
**Configuration management approach**. All container orchestration (CO) frameworks follow a declarative configuration management approach instead of an imperative configuration management approach [68]. _Declarative configuration management_ implies that an application manager describes or generates a declarative specification of the desired state of the distributed application. The CO framework then continuously adapts the deployment and configuration of containers until the actual state of the distributed application matches the described desired state. The configuration language that is used for describing the desired state varies among CO frameworks. Docker Swarm stand-alone [69], Docker Swarm integrated mode [70] and Kubernetes [71] support the YAML mark-up language. Kubernetes also support the JSON mark-up language but recommends YAML. Aurora [72] uses the Python programming. Marathon [73] and DC/OS [74] use the JSON mark-up language. **Architectural patterns. **  
_Master-work architecture._ The core architectural pattern underlying a container cluster is very similar between all frameworks: it is based on the Master-Workers architecture where a Master node controls that running applications are always in their desired state by scheduling containers to the Worker nodes and by monitoring the actual run-time state of nodes and containers. Masters use a distributed data store (e.g., etcd, Consul, or Zookeeper) for storing the actual configuration state about all deployed containers and services. In opposition to Kubernetes and Docker Swarm, Mesos supports a two-level scheduler architecture [75]:  
1. To deal with the differences between frameworks (e.g., some frameworks execute applications in containers, while other frameworks do not), Mesos uses the generic concept of Task for launching both containerized and non-containerized processes. 2. Mesos consists of a two-level scheduler architecture, i.e., the Mesos master and multiple framework schedulers. The Mesos master offers free resources to a particular framework based on the Dominant Resource Fairness [76] algorithm. The selected framework can then accept or reject the offer, for example based on data locality constraints [41]. In order to accept an offer, the framework must explicitly reserve the offered resources [75]. Once a subset of resources is reserved by a framework, the scheduler of that framework can schedule tasks using these resources by sending the tasks to the Mesos master [77]. The Mesos master continues to offer the reserved resources to the framework that has performed the reservation. This is because the framework can respond by unreserving [78] the resources. 3. Since the task life cycle management is distributed across the Mesos master and the framework scheduler, task state needs to be kept synchronized. Mesos supports at-most-once [79], unreliable message delivery between the Mesos master and the frameworks. Therefore, when a framework's scheduler has requested the master to start a task, but doesn't receive an update from the Mesos master, the framework scheduler needs to perform task reconciliation [80]. _Highly-Available (HA) master design._ To ensure high-availability of the cluster, Masters can be replicated in all CO frameworks (see Table 2). _Generic and automated setup of HA masters._ A fully automated and portable framework for setting up replicated Masters in different execution infrastructures is supported in Docker Swarm integrated mode [81], Aurora [82], Marathon [79] and DC/OS [83].