## 3 Machine Learning for the Design Optimization of Electromagnetic Devices  
From the review and discussions in Section 2, it can be seen that there are two major challenges in the design optimization of electromagnetic devices. First, accurate multi-physics analysis is required for many applications, but it normally requires huge computation cost of FEA, for example, for the design of high-speed PM motors. Second, highly-accurate surrogate models are essential for the optimization process. Naturally, the surrogate models of electromagnetic devices are highly nonlinear. In this case, non-parametric models may be superior to the parametric and semi-parametric models for the performance prediction as there is no specific relationship (like linear) between the inputs and outputs. For example, the relationship between efficiency of a PM motor and its dimension may not be able to predict accurately by using polynomials (or RSMs). Fortunately, machine learning presents an opportunity to address these two challenges.  
Machine learning is a method of data analysis (including prediction and optimization) that automates analytical model building. It is seen as a subset of artificial intelligence. As a type of non-parametric modeling technique, machine learning is good at developing complex nonlinear relationships between a number of inputs and outputs by using different neural networks. Thus, it can be used to build surrogate models for models (1)-(6). Many machine learning algorithms have been used to the design optimization of electromagnetic devices, such as artificial neural networks (ANN), support vector machines (SVM), extreme learning machines (ELM), random forest (RF), and deep learning (DL) [3]. DL is a kind of deep neural network (DNN), and is one subset of machine learning algorithms. There are many more layers of neurons in the architectures of DL, compared with ANN, which can be employed to achieve specialized functionalities. To apply them to design electromagnetic devices, there are two major contributions in the common practice. First, these algorithms have been used to predict/estimate the device's field distribution or performance. Second, they can be used to develop surrogate models for optimization [129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153].  
Table 1 lists a comparison of several surrogate models for performance prediction and optimization of electromagnetic devices. There are three types regarding their parametrization. The first category is about parametric models. It includes RSM and RBF. The second one gives semi-parametric models, e.g., Kriging based approaches. The last group involves non-parametric models. It includes three popular machine learning models with explicit mathematical expressions, i.e., ANN, SVM, and ELM. Please note that RF and DL models are not included in this table as they are hard to be expressed by explicit mathematical equations.  
These models have been employed to design and optimize different types of electromagnetic devices recently. Please note that different networks may be applied to these machine learning models. For example, there are two popular networks of ANN, back-propagation (BP) and radial basis function networks. Additionally, there are many types of DL, such as the convolutional neural network (CNN), recurrent neural network (RNN), and generative adversarial networks (GAN). Table 2 lists some selective bibliography focusing on electromagnetic device design optimization based on different machine learning methods. More details are discussed in the following subsections.