# A Comprehensive Survey on Knowledge-Defined Networking
## 3 Knowledge Plane
### Knowledge Generation Plane
#### 3.3.2 Generating Knowledge Using Machine Learning Methods
Multi-objective optimization and deep Q-learning are utilized in a multicast routing framework where multicast tree state, link bandwidth, link delay, and link packet loss rate are considered as the state-space and links are considered as the action space, where the knowledge is generated in the form of a multicast tree to be used by the controller for installing flow rules [189]. A traffic engineering framework uses a reinforcement learning-based scheme to learn a policy to select critical flows for a given traffic matrix in order to reroute those critical flows using linear programming [190]. Using traffic distribution information collected by the controller, a deep reinforcement learning algorithm is used to dynamically adjust a set of links' weights for critical links as a knowledge generation step, where a weighted shortest path algorithm uses such knowledge to generate forwarding paths [191]. A general framework for security management using reinforcement learning, where a deep neural network-based Q-learning agent has been used to mitigate advanced persistent threats [192]. Multi-agent deep deterministic policy gradient-based deep reinforcement leaning has been used in KDNs for joint routing optimization and DDoS attack detection [193]. An anomaly detection and mitigation system uses reinforcement learning to learn policies for dealing with anomalies based on rewards for each action for each profile grouped based on collected network metrics [194]. A reinforcement learning framework has been used in a multimedia-based KDN environment to select the routing algorithm for QoS based traffic flows for QoS provisioning [195]. Deep reinforcement learning has been utilized to learn and build knowledge for 5G and beyond network slicing optimization where generated knowledge has been utilized for making optimal decisions [196]. For 5G cellular Radio Access Network (RAN), closed-loop power adjustment of radio transmitters has been realized in an automated and self-configuring manner by employing a deep reinforcement learning agent to develop intelligence with the objective of increasing equipment throughput [197]. Joint optimization of bandwidth allocation and position overlap of ultra-reliable low latency communication users in 5G networks has been achieved with the aid of deep deterministic policy gradient based deep reinforcement learning to generate knowledge by observing channel variations and traffic arrivals [198]. Now, let us analyze the task-wise classification of knowledge generation techniques discussed above. Table 2 summarizes the details of machine learning or heuristic model-based methods for different knowledge generation scenarios in KDNs. ML models that are capable of generating knowledge are usually trained by vendors, who may keep the models proprietary. However, keeping them proprietary can cause redundant computations and an inefficient use of resources [199]. ML or heuristic models can be used to detect features from input information/data to create knowledge. However, this generated knowledge alone may not be sufficient to arrive at decisions regarding network functions. Thus, there is a requirement for a knowledge composition plane to further analyze and combine the different pieces of generated knowledge.