## 3 Research Method
### Application and Cluster Management
Reusable and configurable framework for checking the health of containersAll CO frameworks also offer a framework for developing custom health checks per container. Different health check methods are possible including HTTP checks and checking via a shell command. Relevant configuration parameters include the timeout period, the interval between two checks and the minimum number of consecutive failed checks for the health check to be considered failed [31]. Central monitoring of distributed eventsDocker Swarm integrated mode, Kubernetes, Mesos, Aurora and Marathon also support an API for monitoring of events about new requests for creating services, container, container state changes and errors [31]. Logging and debugging of CO framework and containersL  
Logging of containers is supported via the CLI and/or dashboard of Docker Swarm [304], Kubernetes [305], Mesos [306] and DC/OS' Marathon [307]. Internal logging of CO framework components is supported by all CO frameworks. Which specific logging tool is used, depends on the used deployment method: when the CO framework is deployed as a set of containers, container logging can be used; when the CO framework is installed as Linux package, the journald [308] service is used [31]. Integration with external log aggregation frameworks is documented in Docker Swarm [309], Kubernetes [310] and DC/OS [311]. Cluster maintenanceL  
Cluster state backup and recovery is a built-in feature of Docker Swarm integrated mode [312], Mesos [313], Aurora [314] and Marathon [315]. For Kubernetes an external project for cluster state management operations [316] such as backup and restore exists. Note that Mesos uses state machine replication (SMR) for storing the state of the entire cluster, including the state of the running frameworks. Aurora uses Mesos' SMR while Marathon does not. Documentation about how to upgrade a running cluster to a next release is provided by Kubernetes [317], DC/OS distribution of Mesos [318], Aurora [319] and Marathon [320]. Upgrades do not affect active containersDocker, the kubeadm deployment tool of Kubernetes and all Mesos-based frameworks support that when the Docker daemon is shut down for upgrade on a node, the containers on that node can continue running [31]. A CLI command for draining all containers from a node for maintenance is supported by Docker Swarm integrated mode [321], Kubernetes [322], Mesos [323], Marathon [324] and DC/OS [325]. Garbage collection of containers and/or images is differently supported by different CO frameworks. Docker [326] supports manual garbage collection of images only at the level of the local registry; Kubernetes' kubelet agent [327] supports automated garbage collection of container images as well as containers. Mesos v1.5 [328] supports automated garbage collection of only Docker images for the Unified Container Runtime. Finally, DC/OS extends Mesos with support for garbage collection of container images for both the Unified Container Runtime as well as the Docker containerizer. Moreover, the architecture of DC/OS [329] also includes support for garbage collection of Docker containers. Multi-cloud supportOne cluster across multiple availability zones or regionsDocker Swarm stand-alone [330] as well as integrated mode [331] allow deploying multiple master nodes across multiple availability zones. Kubernetes [332] provides limited support for multi-zone deployments as generic support for automated HA master setups is not provided. However, Kubernetes-as-a-Service platforms such as Google Kubernetes Engine (GKE) and Amazon Elastic Container Service for Kubernetes (Amazon EKS) [28] offer scalable and highly-available Kubernetes clusters where multiple masters can be deployed across different availability zones.