## 3 Research Method
### Application Configuration and Deployment  
This aspect covers features of CO frameworks that an application manager must understand in order to configure, compose, deploy, scale and upgrade containerized software services and applications. **Supported workload types. ** All CO framework offer support for running different types of workloads: (i) user-facing latency-sensitive, elastically scalable, stateless services; (ii) throughput-sensitive job processing; and (iii) stateful applications. In this sub-aspect we zoom into the former two types of workloads while the next sub-aspect focusses on the support for stateful applications. _Smallest unit of deployment._ Docker Swarm integrated mode [150] and all Mesos-based frameworks [77] propose the concept of Task, which is the smallest atomic unit of deployment. In Docker Swarm, a task encapsulates always a single container. In Mesos-based frameworks, a task encapsulates at most one container, but a task can also run non-containerized processes. In opposition, in Kubernetes, the smallest unit of deployment is a Pod [151], which is a set of co-located containers that logically belong together and therefore are always deployed together on the same node. _Pods._ The abovementioned Kubernetes concept of Pod has also be adopted in Mesos [152], Marathon [153] and DC/OS [154]. Here multiple containers can be launched atomically as part of a task group and these containers are all started inside an underlying Executor container. Such nested containers are only supported in the Mesos containerizer runtime [155]. _Container-based jobs._ Job-oriented workloads where jobs can be configured to execute either (i) _sequentially_ or (ii) _in parallel_ are supported by Kubernetes [156] and Aurora [157]. (iii) _Cron jobs_, which run at predetermined time intervals, are also supported by Kubernetes [158] and Aurora [159]. _Container-based services._ As already stated in Section 4.3, all CO frameworks, except Docker Swarm stand-alone, offer a Service concept for exposing a replicated set of containers to customers as a stable service endpoint that is served by the distributed load balancer or the internal DNS service. Such stable service endpoint can be represented by one or more forms: a virtual IP address, a global service port, a fully qualified DNS name, or just a unique service name. To deploy the container image that implements the service, the application manager declares, besides properties for bootstrapping containers from that image, also information related to the type of load balancing that must be used (internal to the cluster, external, DNS round-robin). A detailed account of the specific approach for each CO framework is presented in the technical report [31]. Kubernetes additionally introduces the concept of ReplicaSet [160] for managing Pod replicas of services and for restarting Pods after a node failure. A ReplicaSet attaches labels [161] to Pods. A service configuration file then only contains a so-called label selector [162] that selects Pods based on their attached labels. _Elastic scaling of services._ In all CO frameworks, the containers behind a service can be horizontally scaled in a dynamic fashion. The number of container replicas can be increased or decreased and for those CO frameworks with a built-in load balancer, the load balancer will automatically reconfigure itself to take into account newly added or removed container replicas. _Auto-scaling of services._ Kubernetes [163] also supports auto-scaler functionality that automatically adapts the number of Pod replicas depending on one or more threshold values with respect to a performance or resource consumption metric. In order for the auto-scaling to work, resource monitoring features (see Section 4.9) must be enabled. The DC/OS [164] distribution of Marathon also supports auto-scaling of Marathon applications by running a Python implementation inside a separate Docker container. It also includes third-party documentation other approaches for auto-scaling services.