# A Comprehensive Survey on Knowledge-Defined Networking
## 5 Control Plane
### Control Models
Even though scalability issues in a single controller architecture are reduced by a flat hierarchy of controllers, when the number of controllers is limited and fixed under very large networks, scalability issues can still exist, as a given controller can handle a fixed number of user equipment without having a performance bottleneck. SMaRtLight is a fault-tolerant controller platform with a logically centralized flat hierarchy master-slave mode of operation [319]. In SMaRtLight, a master controller is in charge of the network, backed up by slave controller replicas synchronized with the master to take over master control in case of master controller failure. Replicated state machines and a shared data store are used to achieve consistency between the master controller and slave controller replicas. In HyperFlow, every controller in the physically distributed control plane has the global network view, where a change in a network domain controlled by a given controller is published to other controllers whenever a change is detected, such as controller failures, such that affected data plane elements will be handed over to a nearby controller [298]. ONOS has a master-slave working principle, where it will protect the network from controller failures by connecting a given data plane element with multiple controllers, where one will be the master controller and others will be the backup controllers. In the event of a master controller failure, one of the backup controllers will take over [280]. Onix uses a network information base data structure to store global network state, which is synchronized between other Onix controllers. Onix also has each controller, which is responsible for a subset of the network, having a global network view, where scalability is ensured using network partitioning and aggregation [300]. Ravana is a logically centralized and physically distributed fault-tolerant controller platform that uses replicated state machines that are extended to switch-side mechanisms to process control messages in order and exactly once, even during failures. In the Ravana framework, the network programmers need to write applications for the main controller, where the Ravana master-slave protocol will ensure replication of control logic to backup controllers such that, when the master crashes, a slave controller will be elected to take over the affected switches [320]. Distributed control systems should also have consistency, which means that data updates on distinct nodes should be updated on the controller nodes. Some distributed control systems have weak consistency, which means that updates in the data plane will eventually be updated in the control plane. Weak consistency can cause different controllers to read different values of the same property over a period of time. HyperFlow has weak consistency compared to other distributed control operating systems. In a logically centralized and physically distributed hierarchical control architecture, the network control plane is both horizontally and vertically partitioned. Note that vertical separation results in a hierarchy of controllers, where some controllers are controlled by one or more other controllers. We can find local controllers at the bottom of the hierarchy and global controllers at the top, as depicted in Figure 7c. The local controllers deal with local applications' requirements with frequent events, whereas the global controllers deal with non-local applications requiring a global network view with rare events. Due to the global/root control having a global view of the network, the control is logically centralized. This is different compared to flat-hierarchy distributed control, as, in that architecture all controllers can have a global view of the network, as opposed to local controllers, which have only a local view of the network in the hierarchically distributed architecture.