# Research on State Diagnosis Methods of UHF Partial Discharge Sensors Based on Improved ViT
## 1 Introduction
Transformers, known for their ability to model long-range dependencies within sequential data, have demonstrated remarkable success in NLP [19], prompting researchers to explore their application in computer vision [20; 21]. This led to the development of the Vision Transformer (ViT), which adapts the Transformer model for image-based tasks. The ViT model has shown superior performance compared to CNNs in several domains, quickly becoming a mainstream model in the field of computer vision [22]. Nevertheless, the ViT model also has limitations when applied to sensor fault diagnosis. One challenge arises from the varying resolutions of image data collected by different sensors. ViT typically compresses images to a fixed resolution, which can negatively affect classification accuracy, especially when working with images of differing resolutions. Another limitation lies in the global self-attention mechanism used by ViT [23], whereby the relationships between all patches (tokens) are computed globally. This results in quadratic computational complexity with respect to the number of patches, making it computationally expensive and less suitable for vision tasks involving high-resolution images or those requiring fine-grained predictions [24; 25]. Existing sensor fault diagnosis methods can be roughly divided into model-based, knowledge-based, and data-driven methods, each with its own advantages and limitations [26]. Model-based methods provide high accuracy and strong interpretability by utilizing physical system characteristics but have difficulty handling complexity and noise robustness in real-world scenarios. Knowledge-based methods benefit from expert knowledge and can achieve rapid diagnosis, but their dependence on predefined rules limits their adaptability to new types of faults [27]. Data-driven methods, especially those that use machine learning, have shown excellent efficiency and accuracy, often surpassing traditional techniques, such as CNN and FFN models, which have different architecture [28]. In theory, FNN has a simple structure, strong universality, and is suitable for processing structured or small-scale data, among which ViT performs well in processing image data. CNN has strong feature extraction ability through convolution operations, which can preserve the spatial structure of data with fewer parameters. However, challenges still exist. For example, for CNN, its network structure is relatively complex, and the training time of the network is long [29]. It heavily relies on large-scale data and computing resources, and its performance is limited on small datasets. Meanwhile, although ViT can capture global features, it has issues such as low computational efficiency and sensitivity to changes in input resolution [30; 31]. To address these challenges, this paper proposes an improvement to the ViT model, designed specifically for sensor state diagnosis. The improved model introduces a sliding window mechanism to construct hierarchical representations. It begins by processing small-sized blocks of image data and gradually merges neighboring blocks as the Transformer layers deepen. This hierarchical structure allows the model to maintain a balance between local and global feature extraction, improving its performance on tasks that require intensive prediction or the representation of high-resolution images. Compared to existing methods, the enhanced model not only improves classification accuracy but also reduces the computational burden associated with global self-attention. In the latter sections of this paper, a detailed comparison is presented between the improved ViT model and two typical CNN models, namely the baseline ResNet50 and ResNet50V2 model, using industrial data to demonstrate the performance of different models. The comparison highlights how the improvements made to the ViT architecture provide superior performance in sensor state diagnosis, particularly in terms of accuracy and computational efficiency.