# Noise Reduction in CWRU Data Using DAE and Classification with ViT
## 1 Introduction
### Background
Additionally, singular value decomposition (SVD) is highly effective for removing noise from vibrational signals, as it does not introduce additional components during the noise removal process. However, SVD struggles to maintain effectiveness with low signal-to-noise ratio signals [16]. Deep learning efficiently learns complex structures, large volumes of samples, and high-dimensional data, attracting the attention of many researchers [8]. Recently, vari  
Figure 1: The importance of data preprocessing. ous deep neural networks, including convolutional neural networks (CNNs), generative adversarial networks (GANs), and recurrent neural networks (RNNs), have been widely used for noise removal in images and audio signals. In particular, CNNs can effectively filter noise and extract abstract features in vibrational signal noise removal [17]. [PERSON] et al. (2023) combined denoising autoencoders (DAE) with CNNs to create one-dimensional convolutional denoising autoencoder (DCAE-1D) networks, effectively removing noise without preprocessing [1]. However, high-dimensional vibration signals containing fault characteristics can complicate the generation of machine learning models. The vanishing gradient problem becomes more pronounced as the number of layers in predictive models increases. [PERSON] et al. (2020) combined measured signals with noise components to extract features in vibrational signals. However, acquiring labeled data presented limitations. Additionally, training on high-dimensional vibration signals required longer training times and more iterations. Recently, deep learning has gained attention for addressing machinery failure diagnosis issues in environments with high noise levels. [PERSON] et al. (2018) introduced the maximum second-order cyclostationary blind convolution algorithm, capable of extracting source signals with bearing fault impact components and featuring a multi-impulse deconvolution function [18]. [PERSON] et al. (2023) transformed vibrational signals with noise into two-dimensional images using recursion graphs, enabling precise diagnosis of failure patterns in the presence of noise through the combination of CNNs. [PERSON] et al. (2023) introduced a failure diagnosis model and a Markov transfer field that enhances a network's feature learning performance by maintaining the temporal dependencies and frequency structures of vibrational signals and converting these signals into two-dimensional images using the Markov transfer field. The vision transformer (ViT) used herein has been developed for image recognition tasks and is also leveraged for fault diagnosis due to its high performance in image processing [19; 20]. ViT decomposes an image into multiple small patches and converts each patch into a vector of fixed dimensions. The transformer encoder processes these vectors, learning the relationships between each patch. The transformer encoder effectively understands the global context of an image using the multi-head self-attention mechanism. The patch representations generated by the encoder are integrated for classification. [PERSON] et al. (2023) introduced an NN-based ViT to enhance early image fusion accuracy. This model merges the feature extraction component through an NN before adding it to the ViT to achieve more optimal results. The merged reconstruction components limit spatial and detail loss to restore image information [19]. [PERSON] et al. (2024) proposed a facial feature recognition network model that combines a ViT-based CNN with a transformer [20]. Traditional deep learning-based failure diagnosis algorithms typically perform classification without learning the temporal characteristics of most time series data. However, while introducing modifications in the image input stage, ViT preserves the encoder structure and attention mechanisms of bidirectional encoder representations from transformers (BERT), commonly used in natural language processing.