## 2 Methodology of Time-Frequency Separation Neural Network
### Enhanced Defect Detection with a Time-Frequency Separation Neural Network
#### 2.3.1 TF Feature Extractor  
The TF feature extractor is actually shared among the weight estimators in different channels. The first four layers of the feature extractor are composed of two stacked one-dimensional convolutional neural network (1D-CNN) layers and two max pooling layers, which are used to extract the dominant spectrum characteristics of a single time-frame in the input STFD spectrogram.  
A one-dimensional convolutional neural network is a well-known local connectivity architecture characterized by its filter kernels and activation functions. The computing process of an \(l\)th-layer 1D-CNN is described as follows:  
\[\mathbf{a}_{k}^{l}=\delta_{l}\left(b_{k}^{l}+\sum_{i=1,l\in\mathbf{M}_{l}}^{N_ {l-1}}\mathbf{w}_{ik}^{l}\mathbf{*}\mathbf{a}_{i}^{l-1}\right) \tag{7}\]  
Where \(\mathbf{M}_{l}\) is the set of filter kernels in the \(l\)th-layer, and \(\mathbf{w}_{k}^{l},b_{k}^{l}\) are the weights and bias for the \(k\)th kernel, respectively. \(\mathbf{a}^{l-1}\) is the \(l\)-1 th-layer output and \(l\)th-layer input. \"\(\mathbf{*}\)\" denotes the convolution operator. \(\delta_{l}\) is the activation function to perform a nonlinear transformation, adopted as tangent or ReLU (rectified linear unit) in this paper. A max pooling layer is added subsequently for dimension reduction of the features. After these successive operations, the spectrum features in time frame \(t_{0}\), \(STFD(t_{0},f)\), can be significantly compressed, and the dominant features of the AE are automatically selected by the extractor in an end-to-end manner. They are used in the sequential modeling for the next step.  
To capture the inherent dependency of the sequential spectrogram, a gated recurrent unit (GRU) layer is employed after 1D-CNN layers. A GRU is an improved and expedited version of a recurrent neural network (RNN), which overcomes the time gradient explosion problem in conventional RNNs through designed gates and converges faster than the long short-term memory (LSTM) network through positive structural simplification [27]. The general mechanism of a GRU layer is controlled by two gates, update gate \(\mathbf{z}_{l}\) and reset gate \(\mathbf{r}_{l}\), whose formula is shown below.  
\[\left\{\begin{array}{l}\widetilde{\mathbf{h}}_{t}=\delta(\mathbf{W}_{h}\cdot [\mathbf{r}_{t}*\mathbf{h}_{l-1},\mathbf{a}_{t}]+\mathbf{b}_{h})\\ \mathbf{h}_{t}=(1-\mathbf{z}_{t})\odot\mathbf{h}_{t-1}+\mathbf{z}_{t}\odot \widetilde{\mathbf{h}}_{t}\\ \mathbf{z}_{t}=\sigma(\mathbf{W}_{z}\cdot[\mathbf{h}_{t-1},\mathbf{a}_{t}]+ \mathbf{b}_{z})\\ \mathbf{r}_{t}=\sigma(\mathbf{W}_{r}\cdot[\mathbf{h}_{t-1},\mathbf{a}_{t}]+ \mathbf{b}_{r})\\ \end{array}\right. \tag{8}\]  
where \(\mathbf{a}_{t},\widetilde{\mathbf{h}}_{t},\mathbf{h}_{t}\) denote the input, memory state, and output in the time frame t. \(\mathbf{W}_{\mu}\) and \(\mathbf{b}_{\mu}\) imply the weights and biases in the gates; \(\mu=h,r,z\). \(\sigma\) is the sigmoid function, and \"\(\odot\)\" is the element-wise multiplying operator. The memory state, \(\widetilde{\mathbf{h}}_{t}\), is updated in accordance  
Figure 2: Architecture of the proposed time–frequency separation neural network.  
with \(\mathbf{z}_{t}\), and \(\mathbf{r}_{t}\) decides if the historical state should be forgotten. In this way, the correlation between frequency characteristics in different time frames can be accurately modelled and gives a better understanding of the above-mentioned spatial time-frequency distributions.