## 2 The Proposed Architecture of Convolutional GAN
### Design of Generator and Discriminator  
It is well-known that GAN is difficult to train, even with the use of WD and gradient penalty methods. One of the prime challenges is the diversity of the generated data, which is needed to cover the data distribution sufficiently. Otherwise, the Generator may become 'lazy', therefore producing the homogeneous data fooling the Discriminator. However, a good point in early fault detection, i.e., in early anomaly detection in the streaming of diagnostic data, is that the primary focus should be the performance of the Discriminator. This is the difference between our task and many other tasks, such as image generation or imbalanced data problems in fault diagnostics. In our work, it is not necessary that the Generator's input is a random signal. Hence, the Autoencoder-based Generator architecture is adopted in this work, i.e., both the input and output of the Generator are real data. The Autoencoder is known as a powerful and versatile non-linear dimensionality reduction technique employing neural networks for which the target output is the same as the input. The proposed architectures of the Generator and Discriminator are shown in Figure 2. The Generator comprises an encoder block and a decoder block with a bottleneck block between them. The encoder is built of two convolutional layers to learn hierarchical representations of the real input, and each convolutional layer is followed by a down-sampling layer to reduce the feature size. Moreover, the down-sampling layer helps to boost the model's robustness to noise and variations in input data. Then, the real input is compressed into a compact representation called the bottleneck. In this paper, the LSTM cell is used in the bottleneck layer to capture the time-series correlation in data. By stacking two layers of LSTM cells, the network can learn more complex patterns in the input data and have better long-term memory retention. Next, the compressed representation is fed into the decoder, which consists of two convolutional layers, one up-sampling layer and one fully connected layer that finally reconstructs the input. In the competition between the Generator and the Discriminator, if the Discriminator is too powerful, it will quickly converge before the Generator can learn useful information from the input. Therefore, a more concise structure is used in the Discriminator, which is composed of three convolutional layers, three down-sampling layers, two fully connected layers and a one-dimensional output layer. The most important mathematical details related to the above structure are presented below. 1. **Convolutional layer. ** The convolution process refers to a specialized linear operation where a small window called a kernel or filter overlays and slides through the entire input with a preset stride, which is mathematically expressed as \[\mathbf{H}_{k}=\mathit{Activ}\left(\sum_{i=1}^{inMap}\mathit{Conv}(\mathbf{W}_{k},\ \mathbf{X}_{i})+b_{k}\right)\] (7) where \(\mathit{Conv}(\cdot)\) denotes the convolution window, \(\mathbf{W}_{k}\) is the kernel that slides through the data \(\mathbf{X}\) and \(b_{k}\) is the corresponding bias. \(\mathit{Activ}(\cdot)\) represents the activation function,  
Figure 2: Architectures of the proposed Generator and Discriminator. stands for the feature vector extracted by the \(k\)-th kernel and the subscript \(k\) defines the number of kernels. 2. **Average down-sampling layer**. The down-sampling layer is also referred to as the pooling layer, which is commonly applied after a convolution layer to reduce the dimension of the feature maps. It refers to a special type of convolution whereby the kernel slides through the entire input map, and, generally, instead of creating the element-wise product, the average value of the overlaid input region is extracted; this quantity is the so-called 'average pooling'. 3. **Up-sampling layer**. In contrast to down-sampling, up-sampling is generally used after the encoder to restore the resolution of the original data.