# TSViT: A Time Series Vision Transformer for Fault Diagnosis of Rotating Machinery
## 1 Introduction  
Rotating machinery is an important component of modern industrial equipment. With the increasing enhancement of product complexity and the integration of functional modules, it poses greater challenges to the security, stability, and overall equipment's robustness [1]. Consequently, the development of effective fault diagnosis methods for rotating machinery has become imperative. Industry 4.0, which aims to realize intelligent upgrading of the manufacturing industry through digitalization and automation technology, has promoted the rapid development of the Industrial Internet of Things (IIoT) [2]. The IIoT connects industrial equipment and sensors to a network to enable real-time data collection and analysis [3]. As the IIoT and big data analytics advance, research on fault diagnosis based on deep learning has emerged [4]. Deep learning models exhibit robust learning capabilities, allowing for the automatic extraction of fault features from data without manual intervention. This significantly reduces the reliance on expert experience and domain knowledge. Many deep learning models have been successfully proposed for fault diagnosis in recent years such as Convolutional Neural Networks (CNNs) [5, 6]. Recurrent Neural Networks (RNNs) [7, 8], Deep AutoEncoders (DAEs) [9, 10], and Deep Belief Networks (DBNs) [11, 12]. However, the sequential nature of RNN models poses challenges in terms oftraining parallelization and gradient explosion or vanishing and is susceptible to significant long-term memory loss. Convolutional filters in CNNs are constrained to process a small local regions and cannot capture global information adequately [13]. Unsupervised DAEs and DBNs exhibit unsatisfactory performance in the case of large-scale and complex datasets due to their structural characteristics [14]. The transformer model has achieved significant improvement in Natural Language Processing (NLP) since it was first proposed based on the self-attention mechanism [15]. However, its ability to capture local features is insufficient. Inspired by transformer models in NLP, researchers recently started to apply transformers to Computer Vision (CV) tasks, known as Vision Transformers, achieving notable results [16]. A Vision Transformer divides an input image into a sequence of patches. Each patch is then converted into a vector and reduced to a smaller dimension. These vector embeddings are subsequently processed by a transformer. [PERSON] and [PERSON] et al. [17] from Google Brain first applied a transformer to CV in 2018 and proposed the Image Transformer model. [PERSON] and [PERSON] et al. [18] from Facebook AI proposed a method called DERT for object detection that uses a transformer architecture to directly predict a set of bounding boxes for each object in an image. [PERSON] and [PERSON] et al. [19] from Google Brain proposed a ViT model that applies a transformer architecture to sequences of image patches. The success of Image Transformer, DERT, and ViT have significantly propelled the rapid development of Vision Transformers and swept the entire CV field. In the latest survey on Vision Transformers, [PERSON] and [PERSON] et al. [20] acknowledged that currently the transformer model is a potential alternative to CNN. The successful application of transformers implies a trend where the transformer architecture is becoming a unified framework for developing models in both CV and NLP. The adoption of a transformer facilitates the seamless integration of vision and language modeling. This emerging trend eases the joint modeling for vision and language processing, fostering a shared learning experience that accelerates advancement in both domains. More and more studies applying Vision Transformers to fault diagnosis have been reported recently [21]. [PERSON] et al. [22] proposed a one-dimensional Vision Transformer with multi-scale convolution fusion (MCF-1 DViT), which combines a CNN and Vision Transformer to diagnose rolling bearing faults based on vibration signals.