# Manufacturing Scheduling Using Colored Petri Nets and Reinforcement Learning
## 2 Literature Review
### PNs and CPNs Combined with AI Techniques for Manufacturing Scheduling
In heuristic search algorithm methods, PNs model the system and a search algorithm based on a heuristic function is used to expand a portion of the reachability graph, containing the most promising nodes that leads to an optimal scheduling solution [3; 22; 23; 24; 25; 26; 36]. The heuristic function must be admissible in order to obtain the optimal solution. In [PERSON] and [PERSON] [3] a heuristic search algorithm based on the A\({}^{*}\) algorithm, a branch and bound type algorithm, was introduced to limit the search in the PN reachability graph in order to find the optimal solution for FMS scheduling problems. Modified and improved versions of the A\({}^{*}\) based heuristic search method have been presented in the literature [25; 26; 27; 28; 29; 30; 31; 32; 33; 34; 35; 36]. In order to reduce the search time, non-admissible heuristic functions were proposed in [31; 32]. A heuristic search method for FMS scheduling based on combination of admissible and non-admissible heuristic functions in the A\({}^{*}\) algorithm was presented in [33]. In the area of agent-based systems and HMS both PNs and CPNs have found applications. A predicate/transition net model for a robotic MAS planning has been presented in [PERSON] et al. [37]. A formal methodology for the modeling and validation of the ADACOR-holonic architecture has been proposed in [PERSON] et al. [38], aiming to improve the performance of control systems in industrial scenarios. A framework to model and control HMS has been suggested in [PERSON] [39] based on fusion of CPNs and multi-agent system theory. The problem of managing and controlling automatic guided vehicles (AGVs) in manufacturing shop floor systems using an agent-based PN model has been addressed in [PERSON] and [PERSON] [40]. PN modeling of FMS with random recipes and control with an agent-based architecture has been proposed in [PERSON] [41]. The model was implemented in the agent platform JADE. A CPN model to represent relative dynamic factors in agent-based scheduling and planning systems that can analyze future states of a system has been proposed in [PERSON] et al. [42]. A PN model to represent the interaction protocols between order and resource agents in a holonic manufacturing execution system, and a game theoretic approach to analyze the possible outcomes of the negotiation process have been presented in [43]. A CTPN-based method was presented in [PERSON] and [PERSON] [44] to model an agent-based warehouse control system in order to evaluate system properties and performance. The method was applied to a dynamic resource allocation of an order-picking process. A new class of PNs, object-oriented knowledge-based PNs, incorporating knowledge-based expert systems and fuzzy logic into ordinary PNs, was introduced and used for integration of design and assembly planning processes in manufacturing in [PERSON] [45]. A fuzzy PN model to represent the fuzzy production rule of a rule-based system has been proposed in [PERSON] et al. [46]. Applications of PNs in AI have been explored in [PERSON] [47] and an application was shown to a rule-based production system. A knowledge PN-based AI protocol that integrated knowledge based expert system and fuzzy logic with ordinary PNs was defined and used for intelligent integration of product design [48]. In [PERSON] et al. [49] a fuzzy expert PN method was presented that integrated knowledge-based systems, fuzzy logic, and artificial neural networks. A generic expert PN model of a single neuron was presented using the method. Knowledge Attribute PNs were introduced in [PERSON] [50] enabling mobile knowledge bases attached to tokens. By using the combination of PNs and AI, the method aimed to enhance the effectiveness of modeling and simulation in applications including FMS.