## 3 Research Method
### Application Configuration and Deployment
Kubernetes v1.9+ [177] and Mesos v1.5+ [178] support the CSI specification [176] that allows run-time installation of external volume plugins. Dynamic provisioning of persistent volumes is supported by most CO frameworks. This feature entails that volumes must not be manually created by the application manager in advance, but instead, volumes are automatically created when a new container is started. The technical report [31] presents a more in-depth comparison of the CO frameworks with respect to the above features. Reusable container configuration.There are a number of commonly supported features related to supporting generic yet configurable container images. Passing environment variables to a container.First, all CO frameworks allow passing of environment variables to a container, which is a common way for configuring the software that is running inside the containers. Self-inspection API.Kubernetes [181] and Marathon [182] enable a container to retrieve information about itself via a so-called downward API. Therefore, this information must not be specified as part of the container configuration or container image. Storing non-sensitive information (such as configuration files) outside a container's image.Docker Swarm integrated mode [183] and Kubernetes [184] additionally support separating configuration data from images in order to keep containerized applications portable. Configuring a custom ENTRYPOINT and CMD.All CO frameworks allow customizing the default ENTRYPOINT and CMD entries of a Docker image at run-time. ENTRYPOINT specifies the command that must be run when starting the container (e.g., /bin/sh--c opens a shell), while CMD specifies the arguments for the entrypoint's command (e.g., cassandra -f starts the Cassandra program of the official Cassandra container image) [31]. Service upgrades.Rolling upgrades of services are supported by all CO frameworks by means of restarting the containers of the service with a new image. In this way, the old version of the service gets gradually replaced with a new version. Note Aurora supports both rolling upgrades of jobs and services [185]. Monitoring the progress of a rolling upgrade.The status of the rolling upgrade can be monitored. Default health and readiness checks (see Section 4.9) are used in order to monitor the health of container replicas and the readiness of new container replicas to start processing requests. In case of failures, the upgrade can be paused, resumed or rolled back. Configuration of custom readiness checks.It is also possible to configure custom readiness checks in Kubernetes [186] and Marathon [187]. Customizing the enactment of the rolling upgrade.Docker Swarm integrated mode [188], Kubernetes [189], Marathon and DC/OS [190] offer various options to customize how the rolling upgrade process is executed/enacted. A common enactment customization is controlling how many instances of the old and new version of the service should always be running during the upgrade. Rollback.Docker Swarm integrated mode [191], Kubernetes [192], Aurora [193] and the DC/OS [194] distribution of Marathon support rolling back an upgrade. Aurora does not offer a command for rolling back an upgrade but can be configured to automatically rollback in case of a failure. Note that recovering from a failed upgrade is a more complicated problem than what a rollback can resolve. In most case, it is better to roll forward by upgrading to a resolved application state [195]. _Canary deployments._ A variant of rolling upgrades, named blue-green deployments or canary deployments [196], intents the same effect as a rolling upgrade but allows for more manual control over the upgrade. The application manager will deploy a completely new service next to the existing service and the application manager can manually control when to redirect users from the old to the new service. Typically, this redirection is only performed after testing the health and readiness of the new service.