# Transmission Condition Monitoring of 3D Printers Based on the Echo State Network
## 2 Methodologies
### The Concept of \(Esn\)  
Both the ESN and liquid state machine (LSM) are machine learning methods for time-series analysis, which are collectively referred to as reservoir computing methods [29]. Just as its name implies, the ESN is composed of an input layer, hidden layer (i.e., reservoir), and an output layer. The hidden layer is designed as a sparse network with many neurons. By adjusting the weights of the network, the function of memory data can be achieved. The dynamic reservoir of the network, which contains a large number of sparsely connected neurons, has the advantage of a short training time. The ESN training is only needed to train the connection weights from the hidden layer to the output layer. In this work, the reason for choosing the ESN was because of its more efficient construction and training method. As shown in Figure 1, there are three distinct sets of neurons including input, the reservoir, and the output layer to construct the ESN. The input, the reservoir, and the output layer contain \(k\), \(m\) and \(L\) nodes respectively and \(\mathbf{w}_{out}\) and \(\mathbf{w}_{back}\) represent output and feedback weight matrices respectively. The weight matrices \(\mathbf{w}_{in}\) and \(\mathbf{w}\) are initialized randomly when training an ESN model [30]. The state update of the network is governed by two state-space equations, as written in Equations (12) and (13). Firstly, updating the reservoir states at each time step is defined by Equation (12). \[x(\mathbf{t})=f(\mathbf{w}_{in}u(t)+\mathbf{w}_{back}x(t-1)), \tag{12}\]  
where \(\mathbf{w}_{in}\) and \(\mathbf{w}\) are randomly initialized for a standard normal distribution to ensure that the ESN has properties of an echo state and is fixed at the time when the network was first built. \(x(t)\) represents  
Figure 1: Schematic of the ESN (Echo State Network). the reservoir states, \(u(t)\) denotes network inputs, and \(f(. )\) is a predefined activation function. Then the reservoir output is governed by Equation (13). \[\mathrm{z}(t)=g(\mathbf{w}_{out}[u(t);x(\mathbf{t})]), \tag{13}\]  
where \(z(t)\) is the reservoir output at a given time step, \(g(. )\) represents the output layer neuron activation function, \([u(t);x(t)]\) denotes the extended state vector, and \(\mathbf{w}_{out}\) is optimized via minimizing the mean square error of \(z(t)\). It is the core fundamental of an ESN that the reservoir is a randomly generated, large-scale, sparse connection, which usually maintains one to five percent of the recursive connections structure. The parameters' optimization, including spectrum radius (SR), reservoir scale (\(N\)), input unit scale (IS), and the sparse degree (SD), is performed to improve the performance of the ESN. The definition of SR is described in Equation (14). Spectrum radius should be satisfied with Equation (15) to ensure reservoir stability. \[\lambda_{\mathrm{max}}=\mathrm{max}\{\sqrt{\lambda(\mathbf{w})}\}, \tag{14}\]  
\[\lambda_{\mathrm{max}}<1, \tag{15}\]  
where \(\lambda_{\mathrm{max}}\) represents the parameter of SR, \(\lambda(\mathbf{w})\) is the eigenvalues of connection weight matrices. The parameter \(N\) is the number of neurons in the reservoir, which has a great influence on the ESN's performance. The larger the reservoir is, the more accurate the ESN describes the given dynamic system. However, it may also lead to over-fitting problems. The parameter IS is related to the degree of non-linearity of samples. The parameter SD represents the connection between neurons in the reservoir. The definition of SD is described in Equation (16). \[\mathrm{SD}\ =\frac{\mathrm{m}}{\mathrm{M}^{\prime}}, \tag{16}\]  
where \(m\) represents the number of interconnected neurons, \(M\) denotes the total number of neurons. The ESN can improve the performance of non-linear processing when the parameter SD increases. The training process of the ESN is divided into two stages: samples generation and weight calculation, i.e., the connection weight matrix \(\mathbf{w}_{out}\) is determined according to the given training samples.