## 3 Research Method
### Securing Clusters  
This aspect covers features that a cluster administrator must understand in order to setup a secure cluster. Note that this aspect only focuses on the security provisions at the level of the container orchestration framework as it does not entail features related to the security of the applications running inside containers.  
User identity and access managementSecure access to the Master API by means of authentication and authorization of usersDifferent CO frameworks differ in the range of supported authentication and access control models, as well as the plug-ability of the solutions) (see technical report [31] for a detailed comparison).  
Tenant-aware access controlAll CO frameworks, except Docker Swarm stand-alone, support _tenant-aware access-control_ that grants users, teams or organizations specific access permissions to a particular user group (see user groups and resource quota management in Section 4.5).  
Cluster network securityAuthentication of worker nodes with the master API is supported by Docker Swarm stand-alone [238], Docker Swarm integrated mode [239], Kubernetes [240], Mesos [241] and Aurora [242].  
Automated bootstrap of authentication tokens for worker nodes is also supported by Docker Swarm integrated mode [243] and Kubernetes [244]. An authentication token is a symmetric key that enables worker nodes to more easily register with the master node to join the cluster.  
Authorization of CO agents on worker nodes towards the master API is additionally supported by Kubernetes [245], Mesos [246], and Aurora [242]. In Kubernetes, this feature allows one to grant API access at the Master node to the Kubelet agent of any node based on the containers that are currently running on that node. Mesos can be configured with an ACL to allow or deny worker nodes to (re-)register with the master. Aurora relies on Zookeeper's ACL mechanisms [247] for controlling Aurora-specific actions of the worker nodes.  
Encryption of control messages between masters and workers is supported by Docker Swarm integrated mode [248] and Kubernetes Container-as-a-Service offering Google Container Engine [249]. Moreover, DC/OS [250] can be configured to startup in a strict or permissive security mode that respectively enables or enforces TLS encryption of communications between masters and agents.  
Encryption of application messages is supported by Docker Swarm integrated mode [251] and DC/OS [250] as an optional feature. Finally, the Weave NET plugin of Kubernetes [252] also supports encryption of application messages.  
Restricting external access to service portsAs stated in Section 4.3, containers of which the services are exposed via a service port can be accessed from outside the cluster if there exists a cluster node with a load balancer which has an IP address that is routable from outside the cluster. However, a security risk ensues that any container with a service port is susceptible to outside malicious attacks, especially if the load balancer is deployed distributed on every node. In order to manage this security risk, one needs a way to segregate the nodes of a cluster into those attached to a private network only and those attached to a private and public network.  
With this end in view, Docker Swarm integrated mode [253] allows exposing master and worker endpoints at a specific IP address or network interface so that service ports on that node are only accessible from the subnet to which the endpoints are attached.  
Kubernetes v1.10+ [254] added a similar feature but also allows specifying a range of IP addresses instead of a single IP address for a master or worker node.  
Finally, DC/OS [255] distinguishes directly between private and public node types. Public nodes support inbound connections from outside the cluster and are thus primarily meant for externally facing load balancers such as marathon-lb or edge-lb. Private nodes cannot be directly accessed from outside the cluster.