# A Multi-Agent Based Intelligent Training System for Unmanned Surface Vehicles
## 3 Multi-Agent-Based Training System Design
### Multi-Agent-Based USV Training Algorithm Design  
The intelligent control model, as depicted in Figure 4, is responsible for performing learning and reasoning. It controls a team of agents in the simulation environment by sending agent actions to the environment every time step. Thus, at time step \(t\), it sends the joint actions \(u_{t}\) to the environment for execution where the action for agent \(i,i\leq n\), is \(u_{i,t}\in u_{t}\). The intelligent control model consists of a learner and a reasoner that employs fuzzy logic and genetic algorithms (GA) known as genetic fuzzy systems (GFS) [27]. In this technique, GA is used to learn and tune the rule base and membership function of an FIS, respectively. To do this, an initial population of solutions, or strings, is created to encode the rule base and membership functions. Classical approaches such as Michigan [28], Pittsburgh [29], and iterative rule learning [30] are mostly used to derive GFS fuzzy rules. We give an instance to illustrate how the rule base is encoded in this study using the Pittsburgh approach. Suppose \(X_{1}\) and \(X_{2}\) are input variables each with linguistic terms \(\{term_{1},term_{2}\}\) and output variable \(Y\) with terms \(\{a_{1},a_{2}\}\) with an arbitrary rule base of an FIS as follows:  
1. IF X1 IS term1 AND X2 IS term1 THEN Y IS a1
2. IF X1 IS term1 AND X2 IS term2 THEN Y IS a2
3. IF X1 IS term2 AND X2 IS term1 THEN Y IS a2
4. IF X1 IS term2 AND X2 IS term2 THEN Y IS a1  
Figure 3: The architecture of the agent data collection platform based on Future Airborne Capability Environment (FACE) components. Assume we assign codes zero and one to \(a_{1}\) and \(a_{2}\) respectively. The chromosome 0110 is obtained. This approach can represent an if-then rule in a single digit. This implies that each chromosome encodes possible outputs of a set of rules. Similarly to tuning membership functions, each digit in the string corresponds to some endpoint of a membership function. GA can then be used to tune the parameters as part of the evolution process; such as found in [31], where initial parameters of a triangular MF \(\mathbb{T}\left(\alpha,\beta,\gamma\right)\) are tuned using:  
\[\alpha_{i+1}\leftarrow\left(\alpha_{i}+\delta_{i}\right)-\eta_{i} \tag{4}\]  
\[\beta_{i+1}\leftarrow\left(\beta_{i}+\delta_{i}\right) \tag{5}\]  
\[\gamma_{i+1}\leftarrow\left(\gamma_{i}+\delta_{i}\right)-\eta_{i} \tag{6}\]  
where \(\delta\) and \(\eta\) are the tuning coefficients, whereas \(\alpha,\beta\) and \(\gamma\) parameterize \(\mathbb{T}\). \(\delta\) shifts the MF to the right or left, whereas \(\eta\) shrinks or expands the MF. Therefore, every MF has two tuning parameters that can be optimized using GA. Although a GFS can be used to find KBs that optimize the fitness function, it is inefficient to use a single GFS for a complex control problem. In such a case, there is an increase in GA search space complexity, the tendency to have redundant rules, and KB size. Therefore, using genetic fuzzy trees (GFT) in [2] helps to mitigate this problem. A GFT is, essentially, an ensemble of GFSs arranged in a tree structure according to a logical and conditional sequence of execution. A GFT enables the decomposition of a complex fuzzy control system (FCS) or GFS into logical sections with each node in the tree focusing on an aspect of the control problem. Each GFS in a GFT defines its own KB. Hence, the KB of a GFT consisting of \(m\in\mathbb{R}\) GFSs \(G=\{gfs_{1},gfs_{2},...,gfs_{m}\}\) may have the genetic structure (represented as a concatenated string) \(gfs_{1}^{rb}gfs_{2}^{rb}  
Algorithm 1 shows how beginning with an initial population, the KB of a GFT can be learned/optimized using the GA process. In Line 2, the initial population of the GFT KB is generated as the current population. This can be based on a predefined KB set or randomly generated. In Lines 3-4, the current population is passed on to the the FIS to be used for the control task.