## 10 Lessons Learned
### Genericity  
* The ratio of common features over unique features is relatively large and most common features are supported by at least 50% of the CO frameworks. Such a high ratio of common features allows for direct comparison of the CO frameworks with respect to non-functional requirements such as scalability and performance of feature implementation strategies.
* Features in the sub-aspects \"improved security isolation\" and \"allocation of other resources\" are only supported by two or three CO frameworks  
* Kubernetes offers the highest number of common features and the highest number of unique features. When adding up both common and unique features, Kubernetes even offers the highest number of features for all 9 aspects and it offers the highest number of features for 15 sub-aspects.
* Significant differences in genericity with Docker EE and DC/OS have however not been found. After all, when taking into account only common features, Kubernetes offers the absolute highest number of common features for 7 sub-aspects, whereas Docker Swarm integrated mode offers the highest number of common features for the sub-aspects \"services networking\", \"host port conflict management\" and \"cluster network security\". Mesos offers the most common features for the sub-aspect \"persistent volumes\" and DC/OS offers the most common features of the sub-aspects \"cluster maintenance\" and \"multi-cloud deployments\".
* In the sub-aspects \"services networking\" and \"host port conflict management\", Docker Swarm integrated mode and DC/OS offer support for the features _host mode services networking_, _stable DNS name for services_ and _dynamic allocation of host ports_. We have found that the other approaches to services networking such as routing meshes and virtual IP networks introduce a substantial performance overhead in comparison to running Docker containers in host mode. As such, a host mode service networking approach with appropriate host port conflict management is a viable alternative for high-performance applications.
* For some of these sub-aspects, there remain differences between whether a particular set of features is offered by the open-source distribution or commercial version of these frameworks:Architectural patterns. The open-source distributions of Docker Swarm and DC/OS all support _automated setup of highly available clusters_, where Kubernetes only provides support for this feature in particular commercial versions.
* User identity and access management. Kubernetes offers the most extensive support for authentication and authorization of cluster administrators and application managers because the open-source distributions of these frameworks offer support for _tenant-aware access control lists_. In opposition, only the commercial versions of Docker Swarm and DC/OS offer support for this feature.
* Creation, management and inspection of cluster and applications. The open-source distributions of Kubernetes and DC/OS offer the most extensive _command-line interfaces_ and _web-based user interfaces_ with support for common features such as _labels for organizing API objects_ and _visual inspection of resource usage graphs_. The commercial version of Docker Swarm also includes a web-based UI with the same set of features, though.
* Logging and debugging of containers and CO frameworks. The open-source distribution of Kubernetes and DC/OS offer support for _integrating existing log aggregation systems_. In opposition, only the commercial version of Docker Swarm supports this feature.
* Multi-cloud support. Docker Swarm and all Mesos-based systems have invested most of their effort in building extensive support for running a single container cluster in high availability mode where multiple masters are spread across different cloud availability zones. Support for such an automated HA cluster across multiple availability zones is not supported by the open-source contribution of Kubernetes; it is only supported by the commercial Kubernetes-as-a-Service offerings on top of AWS and Google cloud.