## 1 Introduction  
Visual anomaly detection (VAD) is vital to product quality, defect detection, and other industrial applications [1; 2; 3; 4; 5]. The automation of the AD process is essential for handling vast quantities of objects or extended operational durations. Typically, anomalies are rare events, and the number of positive samples (defects or anomalies) is substantially lower than that of negative samples (\"normal\" instances). Moreover, fine-annotating and collecting sufficient samples is labor-intensive and costly for domain experts. Supervised anomaly detection models encounter significant challenges due to the diversity in fault sample types and the pronounced imbalance in inter-class distributions. During the training phase, these models tend to overfit the majority classes, often referred to as \"head\" classes, making it challenging for them to learn the distinctive features of the minority or \"tail\" classes. To tackle these challenges, a transformative approach has emerged, wherein anomaly detection is reimagued as an out-of-distribution (OOD) problem [6; 7; 8]. Unsupervised anomaly detection models are trained using datasets devoid of anomalies, referred to as anomaly-free datasets [9; 10; 11]. These models become adept at learning the underlying distribution of normal or non-anomalous data patterns. When a test image is presented, the model quantifies the dissimilarity between the test sample and the learned distribution. If this dissimilarity surpasses a certain threshold, the sample is classified as an anomaly. These OOD models can operate at various granularity levels, ranging from basic image-level anomaly detection, classifying entire images as normal or anomalous, to the more complex pixel-level segmentation, which is crucial for intricate fault detection tasks. Pixel-level segmentation requires the model to precisely identify the regions or pixels within an image that deviate from the norm, offering a fine-grained prediction of anomalies. This paradigm shift in reinterpreting anomaly detection as an OOD problem provides a practical and resource-efficient solution to address the challenges posed by long-tail anomaly class distributions and the costs associated with expert annotation [12; 13]. This approach has revitalized the field by offering promising solutions for visual anomaly detection in industrial applications. However, the widespread adoption of unsupervised anomaly detection algorithms in industrial fault detection tasks is hindered by frequent false alarms. These false alarms, triggered by detectors in response to benign variations or minor fluctuations in industrial processes, introduce substantial noise and uncertainty into the system. Consequently, the high rate of false positives undermines the credibility of the anomaly detection system and exacerbates costs for massive re-identifications. Operators and engineers, overwhelmed by false alarms, may become desensitized to alerts, potentially missing genuine anomalies when they occur. Therefore, it is imperative to address the issue of frequent false alarms to unlock the full potential of unsupervised anomaly detection in industrial fault detection tasks. In this article, we primarily focus on detection models that generate prediction scores synthetically at both image and pixel levels, as illustrated in Figure 1. The image-level prediction score indicates the presence of defects in the corresponding image (abnormal/normal). Simultaneously, pixel-level prediction scores form the anomaly map, which becomes pixel-wise segmentation masks through thresholding at a default threshold, i.e., 0.5. Upon investigating false positives in testing images, we discover that these patches share distribution characteristics with specific patterns of high-score regions in the anomaly maps of anomaly-free training images. Defining defects in industrial applications is more intricate than merely detecting large-area obvious appearance differences, leading to diverse intra-class normal (negative) features in the training dataset.