Original: A Unified Model for Multi-class Anomaly Detection
Revised: A Unified Model for Multi-class Anomaly Detection

Original: Abstract
Revised: Abstract

Original: Despite the rapid advance of unsupervised anomaly detection, existing methods require to train separate models for different objects.
Revised: Despite the rapid advance of unsupervised anomaly detection, existing methods require to train separate models for different objects.

Original: In this work, we present
Revised: In this work, we present

Original: UniAD that accomplishes anomaly detection for multiple classes with a unified framework.
Revised: UniAD that accomplishes anomaly detection for multiple classes with a unified framework.

Original: Under such a challenging setting, popular reconstruction networks may fall into an “identical shortcut”, where both normal and anomalous samples can be well recovered, and hence fail to spot outliers.
Revised: Under such a challenging setting, popular reconstruction networks may fall into an “identical shortcut”, where both normal and anomalous samples can be well recovered, and hence fail to spot outliers.

Original: To tackle this obstacle, we make three improvements.
Revised: To tackle this obstacle, we make three improvements.

Original: First, we revisit the formulations of fully-connected layer, convolutional layer, as well as attention layer, and confirm the important role of query embedding ( i.e. , within attention layer) in preventing the network from learning the shortcut.
Revised: First, we revisit the formulations of fully-connected layer, convolutional layer, as well as attention layer, and confirm the important role of query embedding ( i.e. , within attention layer) in preventing the network from learning the shortcut.

Original: We therefore come up with a layer-wise query decoder to help model the multi-class distribution.
Revised: We therefore come up with a layer-wise query decoder to help model the multi-class distribution.

Original: Second, we employ a neighbor masked attention module to further avoid the information leak from the input feature to the reconstructed output feature.
Revised: Second, we employ a neighbor masked attention module to further avoid the information leak from the input feature to the reconstructed output feature.

Original: Third, we propose a feature jittering strategy that urges the model to recover the correct message even with noisy inputs.
Revised: Third, we propose a feature jittering strategy that urges the model to recover the correct message even with noisy inputs.

Original: We evaluate our algorithm on MVTec-AD and CIFAR-10 datasets, where we surpass the state-of-the-art alternatives by a sufficiently large margin.
Revised: We evaluate our algorithm on MVTec-AD and CIFAR-10 datasets, where we surpass the state-of-the-art alternatives by a sufficiently large margin.

Original: For example, when learning a unified model for 15 categories in MVTec-AD, we surpass the second competitor on the tasks of both anomaly detection (from 88.1% to 96.5%) and anomaly localization (from 89.5% to 96.8%).
Revised: For example, when learning a unified model for 15 categories in MVTec-AD, we surpass the second competitor on the tasks of both anomaly detection (from 88.1% to 96.5%) and anomaly localization (from 89.5% to 96.8%).

Original: Code is available at https:// github.com/zhiyuanyou/UniAD .
Revised: Code is available at https:// github.com/zhiyuanyou/UniAD .

Original:  Introduction
Revised:  Introduction

Original: Anomaly detection has found an increasingly wide utilization in manufacturing defect detection [4], medical image analysis [17], and video surveillance [47].
Revised: Anomaly detection has found an increasingly wide utilization in manufacturing defect detection [4], medical image analysis [17], and video surveillance [47].

Original: Considering the highly diverse anomaly types, a common solution is to model the distribution of normal samples and then identify anomalous ones via finding outliers.
Revised: Considering the highly diverse anomaly types, a common solution is to model the distribution of normal samples and then identify anomalous ones via finding outliers.

Original: It is therefore crucial to learn a compact boundary for normal data, as shown in Fig.
Revised: It is therefore crucial to learn a compact boundary for normal data, as shown in Fig.

Original: 1a.
Revised: 1a.

Original: For this purpose, existing methods [6, 11, 26, 28, 49, 50, 53] propose to train separate models for different classes of objects, like in Fig. 1c.
Revised: For this purpose, existing methods [6, 11, 26, 28, 49, 50, 53] propose to train separate models for different classes of objects, like in Fig. 1c.

Original: However, such a one-class-one-model scheme could be memory-consuming especially along with the number of classes increasing, and also uncongenial to the scenarios where the normal samples manifest themselves in a large intra-class diversity ( i.e. , one object consists of various types).
Revised: However, such a one-class-one-model scheme could be memory-consuming especially along with the number of classes increasing, and also uncongenial to the scenarios where the normal samples manifest themselves in a large intra-class diversity ( i.e. , one object consists of various types).

Original: In this work, we target a more practical task, which is to detect anomalies from different object classes with a unified framework.
Revised: In this work, we target a more practical task, which is to detect anomalies from different object classes with a unified framework.

Original: The task setting is illustrated in Fig.
Revised: The task setting is illustrated in Fig.

Original: 1d, where the training data covers normal samples from a range of categories, and the learned model is asked to accomplish anomaly detection for all these categories without any fine-tuning.
Revised: 1d, where the training data covers normal samples from a range of categories, and the learned model is asked to accomplish anomaly detection for all these categories without any fine-tuning.

Original: It is noteworthy that the categorical information ( i.e. , class label) is inaccessible at both the training and the inference stages, considerably ∗ Contribute Equally.
Revised: It is noteworthy that the categorical information ( i.e. , class label) is inaccessible at both the training and the inference stages, considerably ∗ Contribute Equally.

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: Boundary
Revised: Boundary

Original: [Figure]
Revised: [Figure]

Original: easing the difficulty of data preparation.
Revised: easing the difficulty of data preparation.

Original: Nonetheless, solving such a task is fairly challenging.
Revised: Nonetheless, solving such a task is fairly challenging.

Original: Recall that the rationale behind unsupervised anomaly detection is to model the distribution of normal data and find a compact decision boundary as in Fig.
Revised: Recall that the rationale behind unsupervised anomaly detection is to model the distribution of normal data and find a compact decision boundary as in Fig.

Original: 1a.
Revised: 1a.

Original: When it comes to the multi-class case, we expect the model to capture the distribution of all classes simultaneously such that they can share the same boundary as in Fig.
Revised: When it comes to the multi-class case, we expect the model to capture the distribution of all classes simultaneously such that they can share the same boundary as in Fig.

Original: 1b.
Revised: 1b.

Original: But if we focus on a particular category, say the green one in Fig.
Revised: But if we focus on a particular category, say the green one in Fig.

Original: 1b, all the samples from other categories should be considered as anomalies no matter whether they are normal ( i.e. , blue circles) or anomalous ( i.e. , blue triangles) themselves.
Revised: 1b, all the samples from other categories should be considered as anomalies no matter whether they are normal ( i.e. , blue circles) or anomalous ( i.e. , blue triangles) themselves.

Original: From this perspective, how to accurately model the multi-class distribution becomes vital.
Revised: From this perspective, how to accurately model the multi-class distribution becomes vital.

Original: A widely used approach to learning the normal data distribution draws support from image (or feature) reconstruction [2, 5, 27, 40, 52], which assumes that a well-trained model always produces normal samples regardless of the defects within the inputs.
Revised: A widely used approach to learning the normal data distribution draws support from image (or feature) reconstruction [2, 5, 27, 40, 52], which assumes that a well-trained model always produces normal samples regardless of the defects within the inputs.

Original: In this way, there will be large reconstruction errors for anomalous samples, making them distinguishable from the normal ones.
Revised: In this way, there will be large reconstruction errors for anomalous samples, making them distinguishable from the normal ones.

Original: However, we find that popular reconstruction networks suggest unsatisfying performance on the challenging task studied in this work.
Revised: However, we find that popular reconstruction networks suggest unsatisfying performance on the challenging task studied in this work.

Original: They typically fall into an “identity shortcut”, which appears as returning a direct copy of the input disregarding its content.
Revised: They typically fall into an “identity shortcut”, which appears as returning a direct copy of the input disregarding its content.

Original: 1 As a result, even anomalous samples can be well recovered with the learned model and hence become hard to detect.
Revised: 1 As a result, even anomalous samples can be well recovered with the learned model and hence become hard to detect.

Original: Moreover, under the unified case, where the distribution of normal data is more complex, the “identical shortcut” problem is magnified.
Revised: Moreover, under the unified case, where the distribution of normal data is more complex, the “identical shortcut” problem is magnified.

Original: Intuitively, to learn a unified model that can reconstruct all kinds of objects, it requires the model to work extremely hard to learn the joint distribution.
Revised: Intuitively, to learn a unified model that can reconstruct all kinds of objects, it requires the model to work extremely hard to learn the joint distribution.

Original: From this perspective, learning an “identical shortcut” appears as a far easier solution.
Revised: From this perspective, learning an “identical shortcut” appears as a far easier solution.

Original: To address this issue, we carefully tailor a feature reconstruction framework that prevents the model from learning the shortcut.
Revised: To address this issue, we carefully tailor a feature reconstruction framework that prevents the model from learning the shortcut.

Original: First, we revisit the formulations of fully-connected layer, convolutional layer, as well as attention layer used in neural networks, and observe that both fully-connected layer and convolutional layer face the risk of learning a trivial solution.
Revised: First, we revisit the formulations of fully-connected layer, convolutional layer, as well as attention layer used in neural networks, and observe that both fully-connected layer and convolutional layer face the risk of learning a trivial solution.

Original: This drawback is further amplified under the multi-class setting in that the normal data distribution becomes far more complex.
Revised: This drawback is further amplified under the multi-class setting in that the normal data distribution becomes far more complex.

Original: Instead, the attention layer is sheltered from such a risk, benefiting from a learnable query embedding (see Sec. 3.1).
Revised: Instead, the attention layer is sheltered from such a risk, benefiting from a learnable query embedding (see Sec. 3.1).

Original: Accordingly, we propose a layer-wise query decoder to intensify the use of query embedding.
Revised: Accordingly, we propose a layer-wise query decoder to intensify the use of query embedding.

Original: Second, we argue that the full attention ( i.e. , every feature point relates to each other) also contributes to the shortcut issue, because it offers the chance of directly copying the input to the output.
Revised: Second, we argue that the full attention ( i.e. , every feature point relates to each other) also contributes to the shortcut issue, because it offers the chance of directly copying the input to the output.

Original: To avoid the information leak, we employ a neighbor masked attention module, where a feature point relates to neither itself nor its neighbors.
Revised: To avoid the information leak, we employ a neighbor masked attention module, where a feature point relates to neither itself nor its neighbors.

Original: Third, inspired by Bengio et al.
Revised: Third, inspired by Bengio et al.

Original: [3], we propose a feature jittering strategy, which requires the model to recover the source message even with noisy inputs.
Revised: [3], we propose a feature jittering strategy, which requires the model to recover the source message even with noisy inputs.

Original: All these designs help the model escape from the “identity shortcut”, as shown in Fig.
Revised: All these designs help the model escape from the “identity shortcut”, as shown in Fig.

Original: 2b.
Revised: 2b.

Original: Extensive experiments on MVTec-AD [4] and CIFAR-10 [24] demonstrate the sufficient superiority of our approach, which we call UniAD , over existing alternatives under the unified task setting.
Revised: Extensive experiments on MVTec-AD [4] and CIFAR-10 [24] demonstrate the sufficient superiority of our approach, which we call UniAD , over existing alternatives under the unified task setting.

Original: For instance, when learning a single model for 15 categories in MVTec-AD, we achieve state-of-the-art performance on the tasks of both anomaly detection and anomaly localization, boosting the AUROC from 88.1% to 96.5% and from 89.5% to 96.8%, respectively.
Revised: For instance, when learning a single model for 15 categories in MVTec-AD, we achieve state-of-the-art performance on the tasks of both anomaly detection and anomaly localization, boosting the AUROC from 88.1% to 96.5% and from 89.5% to 96.8%, respectively.

Original:  Related work
Revised:  Related work

Original: Anomaly detection.
Revised: Anomaly detection.

Original: 1) Classical approaches extend classical machine learning methods for one-class classification, such as one-class support vector machine (OC-SVM) [39] and support vector data description (SVDD)
Revised: 1) Classical approaches extend classical machine learning methods for one-class classification, such as one-class support vector machine (OC-SVM) [39] and support vector data description (SVDD)

Original: Patch-level embedding [49], geometric transformation [18], and elastic weight consolidation [34] are incorporated for improvement.
Revised: Patch-level embedding [49], geometric transformation [18], and elastic weight consolidation [34] are incorporated for improvement.

Original: 2) Pseudo-anomaly converts anomaly detection to supervised learning, including classification [26, 33, 46], image denoising [53], and hypersphere segmentation [28].
Revised: 2) Pseudo-anomaly converts anomaly detection to supervised learning, including classification [26, 33, 46], image denoising [53], and hypersphere segmentation [28].

Original: However, these methods partly rely on how well proxy anomalies match real anomalies that are not known [13].
Revised: However, these methods partly rely on how well proxy anomalies match real anomalies that are not known [13].

Original: Modeling then comparison assumes that the pre-trained network is capable of extracting discriminative features for anomaly detection [11, 35].
Revised: Modeling then comparison assumes that the pre-trained network is capable of extracting discriminative features for anomaly detection [11, 35].

Original: PaDiM [11] and MDND [35] extract pre-trained features to model normal distribution, then utilize a distance metric to measure the anomalies.
Revised: PaDiM [11] and MDND [35] extract pre-trained features to model normal distribution, then utilize a distance metric to measure the anomalies.

Original: Nevertheless, these methods need to memorize and model all normal features, thus are computationally expensive.
Revised: Nevertheless, these methods need to memorize and model all normal features, thus are computationally expensive.

Original: Knowledge distillation proposes that the student distilled by a teacher on normal samples could only extract normal features [6, 13, 38, 45, 46].
Revised: Knowledge distillation proposes that the student distilled by a teacher on normal samples could only extract normal features [6, 13, 38, 45, 46].

Original: Recent works mainly focus on model ensemble [6], feature pyramid [38, 45], and reverse distillation [13].
Revised: Recent works mainly focus on model ensemble [6], feature pyramid [38, 45], and reverse distillation [13].

Original: Reconstruction-based anomaly detection.
Revised: Reconstruction-based anomaly detection.

Original: These methods rely on the hypothesis that reconstruction models trained on normal samples only succeed in normal regions, but fail in anomalous regions [5, 8, 27, 37, 50].
Revised: These methods rely on the hypothesis that reconstruction models trained on normal samples only succeed in normal regions, but fail in anomalous regions [5, 8, 27, 37, 50].

Original: Early attempts include Auto-Encoder (AE) [5, 9], Variational Auto-Encoder (VAE) [23, 27], and Generative Adversarial Net (GAN) [2, 31, 37, 52].
Revised: Early attempts include Auto-Encoder (AE) [5, 9], Variational Auto-Encoder (VAE) [23, 27], and Generative Adversarial Net (GAN) [2, 31, 37, 52].

Original: However, these methods face the problem that the model could learn tricks that the anomalies are also restored well.
Revised: However, these methods face the problem that the model could learn tricks that the anomalies are also restored well.

Original: Accordingly, researchers adopt different strategies to tackle this issue, such as adding instructional information ( i.e. , structural [54] or semantic [40, 47]), memory mechanism [19, 21, 30], iteration mechanism [12], image masking strategy [48], and pseudo-anomaly
Revised: Accordingly, researchers adopt different strategies to tackle this issue, such as adding instructional information ( i.e. , structural [54] or semantic [40, 47]), memory mechanism [19, 21, 30], iteration mechanism [12], image masking strategy [48], and pseudo-anomaly

Original: Recently, DRAEM [53] first recovers the pseudo-anomaly disturbed normal images for representation, then utilizes a discriminative net to distinguish the anomalies, achieving excellent performance.
Revised: Recently, DRAEM [53] first recovers the pseudo-anomaly disturbed normal images for representation, then utilizes a discriminative net to distinguish the anomalies, achieving excellent performance.

Original: However, DRAEM [53] ceases to be effective under the unified case.
Revised: However, DRAEM [53] ceases to be effective under the unified case.

Original: Moreover, there is still an important aspect that has not been well studied, i.e. , what architecture is the best reconstruction model?
Revised: Moreover, there is still an important aspect that has not been well studied, i.e. , what architecture is the best reconstruction model?

Original: In this paper, we first compare and analyze three popular architectures including MLP, CNN, and transformer.
Revised: In this paper, we first compare and analyze three popular architectures including MLP, CNN, and transformer.

Original: Then, accordingly, we base on the transformer and further design three improvements, which compose our UniAD.
Revised: Then, accordingly, we base on the transformer and further design three improvements, which compose our UniAD.

Original: Transformer in anomaly detection.
Revised: Transformer in anomaly detection.

Original: Transformer [43] with attention mechanism, first proposed in natural language processing, has been successfully used in computer vision [7, 16].
Revised: Transformer [43] with attention mechanism, first proposed in natural language processing, has been successfully used in computer vision [7, 16].

Original: Some attempts try to utilize transformer for anomaly detection.
Revised: Some attempts try to utilize transformer for anomaly detection.

Original: InTra [32] adopts transformer to recover the image by recovering all masked patches one by one.
Revised: InTra [32] adopts transformer to recover the image by recovering all masked patches one by one.

Original: VT-ADL [29] and AnoVit [51] both apply transformer encoder to reconstruct images.
Revised: VT-ADL [29] and AnoVit [51] both apply transformer encoder to reconstruct images.

Original: However, these methods directly utilize vanilla transformer, and do not figure out why transformer brings improvement.
Revised: However, these methods directly utilize vanilla transformer, and do not figure out why transformer brings improvement.

Original: In contrast, we confirm the efficacy of the query embedding to prevent the shortcut, and accordingly design a layer-wise query decoder.
Revised: In contrast, we confirm the efficacy of the query embedding to prevent the shortcut, and accordingly design a layer-wise query decoder.

Original: Also, to avoid the information leak of the full attention, we employ a neighbor masked attention module.
Revised: Also, to avoid the information leak of the full attention, we employ a neighbor masked attention module.

Original:  Method 3.1 Revisiting feature reconstruction for anomaly detection
Revised:  Method 3.1 Revisiting feature reconstruction for anomaly detection

Original: In Fig. 2, following the feature reconstruction paradigm [40, 50], we build an MLP, a CNN, and a transformer (with query embedding) to reconstruct the features extracted by a pre-trained backbone.
Revised: In Fig. 2, following the feature reconstruction paradigm [40, 50], we build an MLP, a CNN, and a transformer (with query embedding) to reconstruct the features extracted by a pre-trained backbone.

Original: The reconstruction errors represent the anomaly possibility.
Revised: The reconstruction errors represent the anomaly possibility.

Original: The architectures of the three networks are given in Appendix .
Revised: The architectures of the three networks are given in Appendix .

Original: The metric is evaluated every 10 epochs.
Revised: The metric is evaluated every 10 epochs.

Original: Note that the periodic evaluation is impractical since anomalies are not available during training.
Revised: Note that the periodic evaluation is impractical since anomalies are not available during training.

Original: As shown in Fig.
Revised: As shown in Fig.

Original: 2a, after a period of training, the performances of the three networks decrease severely with the losses going extremely small.
Revised: 2a, after a period of training, the performances of the three networks decrease severely with the losses going extremely small.

Original: We attribute this to the problem of “identical shortcut”, where both normal and anomalous regions can be well recovered, thus failing to spot anomalies.
Revised: We attribute this to the problem of “identical shortcut”, where both normal and anomalous regions can be well recovered, thus failing to spot anomalies.

Original: This speculation is verified by the visualization results in Fig.
Revised: This speculation is verified by the visualization results in Fig.

Original: 2b (more results in Appendix ).
Revised: 2b (more results in Appendix ).

Original: However, compared with MLP and CNN, the transformer suffers from a much smaller performance drop, indicating a slighter shortcut problem.
Revised: However, compared with MLP and CNN, the transformer suffers from a much smaller performance drop, indicating a slighter shortcut problem.

Original: This encourages us to analyze as follows.
Revised: This encourages us to analyze as follows.

Original: We denote the features in a normal image as x + ∈ R K × C , where K is the feature number, C is the channel dimension.
Revised: We denote the features in a normal image as x + ∈ R K × C , where K is the feature number, C is the channel dimension.

Original: The batch dimension is omitted for simplicity.
Revised: The batch dimension is omitted for simplicity.

Original: Similarly, the features in an anomalous image are denoted as x − ∈ R K × C .
Revised: Similarly, the features in an anomalous image are denoted as x − ∈ R K × C .

Original: The reconstruction loss is chosen as the MSE loss.
Revised: The reconstruction loss is chosen as the MSE loss.

Original: [Figure]
Revised: [Figure]

Original: During the training of MLP, CNN, and transformer, the reconstruction error keeps going smaller on normal samples, but the performance on anomalies suffers from a severe drop after reaching the peak.
Revised: During the training of MLP, CNN, and transformer, the reconstruction error keeps going smaller on normal samples, but the performance on anomalies suffers from a severe drop after reaching the peak.

Original: This is caused by the model learning an “identical shortcut”, which tends to directly copy the input as the output regardless of whether it is normal or anomalous.
Revised: This is caused by the model learning an “identical shortcut”, which tends to directly copy the input as the output regardless of whether it is normal or anomalous.

Original: (b) Visual explanation of the shortcut issue, where the anomalous samples can be well recovered and hence become hard to detect from normal ones.
Revised: (b) Visual explanation of the shortcut issue, where the anomalous samples can be well recovered and hence become hard to detect from normal ones.

Original: In contrast, UniAD overcomes such a problem and manages to reconstruct anomalies as normal samples .
Revised: In contrast, UniAD overcomes such a problem and manages to reconstruct anomalies as normal samples .

Original: It is noteworthy that all models are learned for feature reconstruction and a separate decoder is employed to render images from features.
Revised: It is noteworthy that all models are learned for feature reconstruction and a separate decoder is employed to render images from features.

Original: This decoder is only used for visualization.
Revised: This decoder is only used for visualization.

Original: We provide a rough analysis using a simple 1-layer network as the reconstruction net, which is trained with x + and tested to detect anomalous regions in x − .
Revised: We provide a rough analysis using a simple 1-layer network as the reconstruction net, which is trained with x + and tested to detect anomalous regions in x − .

Original: Fully-connected layer in MLP .
Revised: Fully-connected layer in MLP .

Original: Denote the weights and bias in this layer as w ∈ R C × C , b ∈ R C , respectively, this layer can be represented as,
Revised: Denote the weights and bias in this layer as w ∈ R C × C , b ∈ R C , respectively, this layer can be represented as,

Original: [Equation]
Revised: [Equation]

Original: With the MSE loss pushing y to x + , the model may take shortcut to regress w → I (identity matrix), b → 0 .
Revised: With the MSE loss pushing y to x + , the model may take shortcut to regress w → I (identity matrix), b → 0 .

Original: Ultimately, this model could also reconstruct x − well, failing in anomaly detection.
Revised: Ultimately, this model could also reconstruct x − well, failing in anomaly detection.

Original: Convolutional layer in CNN .
Revised: Convolutional layer in CNN .

Original: A convolutional layer with 1 × 1 kernel is equivalent to a fullyconnected layer.
Revised: A convolutional layer with 1 × 1 kernel is equivalent to a fullyconnected layer.

Original: Besides, An n × n ( n > 1 ) kernel has more parameters and larger capacity, and can complete whatever 1 × 1 kernel can.
Revised: Besides, An n × n ( n > 1 ) kernel has more parameters and larger capacity, and can complete whatever 1 × 1 kernel can.

Original: Thus, this layer also has the chance to learn a shortcut.
Revised: Thus, this layer also has the chance to learn a shortcut.

Original: Transformer with query embedding .
Revised: Transformer with query embedding .

Original: In such a model, there is an attention layer with a learnable query embedding, q ∈ R K × C .
Revised: In such a model, there is an attention layer with a learnable query embedding, q ∈ R K × C .

Original: When using this layer as the reconstruction model, it is denoted as,
Revised: When using this layer as the reconstruction model, it is denoted as,

Original: [Equation]
Revised: [Equation]

Original: To push y to x + , the attention map, softmax ( q ( x + ) T / √ C ) , should approximate I (identity matrix), so q must be highly related to x + .
Revised: To push y to x + , the attention map, softmax ( q ( x + ) T / √ C ) , should approximate I (identity matrix), so q must be highly related to x + .

Original: Considering that q in the trained model is relevant to normal samples, the model could not reconstruct x − well.
Revised: Considering that q in the trained model is relevant to normal samples, the model could not reconstruct x − well.

Original: The ablation study in Sec.
Revised: The ablation study in Sec.

Original: 4.6 shows that without the query embedding, the performance of transformer drops dramatically by 13.4% and 18.1% in pixel ROAUC and image ROAUC, respectively.
Revised: 4.6 shows that without the query embedding, the performance of transformer drops dramatically by 13.4% and 18.1% in pixel ROAUC and image ROAUC, respectively.

Original: Thus the query embedding is of vital significance to model the normal distribution.
Revised: Thus the query embedding is of vital significance to model the normal distribution.

Original: However, transformer still suffers from the shortcut problem, which inspires our three improvements.
Revised: However, transformer still suffers from the shortcut problem, which inspires our three improvements.

Original: According to that the query embedding can prevent reconstructing anomalies, we design a Layerwise Query Decoder (LQD) by adding the query embedding in each decoder layer rather than only the first layer in vanilla transformer.
Revised: According to that the query embedding can prevent reconstructing anomalies, we design a Layerwise Query Decoder (LQD) by adding the query embedding in each decoder layer rather than only the first layer in vanilla transformer.

Original: We suspect that the full attention increases the possibility of the shortcut.
Revised: We suspect that the full attention increases the possibility of the shortcut.

Original: Since one token could see itself and its neighbor regions, it is easy to reconstruct by simply copying.
Revised: Since one token could see itself and its neighbor regions, it is easy to reconstruct by simply copying.

Original: Thus we mask the neighbor tokens when calculating the attention map, called
Revised: Thus we mask the neighbor tokens when calculating the attention map, called

Original: Neighbor Masked Attention (NMA).
Revised: Neighbor Masked Attention (NMA).

Original: We employ a Feature Jittering (FJ) strategy to disturb the input features, leading the model to learn normal distribution from denoising.
Revised: We employ a Feature Jittering (FJ) strategy to disturb the input features, leading the model to learn normal distribution from denoising.

Original: Benefiting from these designs, our UniAD achieves satisfying performance, as illustrated in Fig.
Revised: Benefiting from these designs, our UniAD achieves satisfying performance, as illustrated in Fig.

Revised: Relation between the “identical shortcut” problem and the unified case .

Revised: In Fig.

Revised: 2a, we aim to visualize the “identical shortcut” problem, where the loss becomes smaller yet the performance drops.

Original: [Figure]
Revised: [Figure]

Original: Masked Attention NeighborMasked Attention
Revised: Neighbor

Revised: [Figure]

Original: Neighbor Masked Attention
Revised: Neighbor Masked Attention

Original: Query Embedding
Revised: Query Embedding

Original: Feed Forward Net
Revised: Feed Forward Net

Original: N ×
Revised: N ×

Original: MSE Loss
Revised: MSE Loss

Original: Feature Tokens
Revised: Feature Tokens

Original: Reconstructed Tokens
Revised: Reconstructed Tokens

Original: Feature Tokens
Revised: Feature Tokens

Original: Feed Forward Net
Revised: Feed Forward Net

Original: K
Revised: K

Original: N ×
Revised: N ×

Original: V
Revised: V

Original: Q
Revised: Q

Original: K
Revised: K

Original: V
Revised: V

Original: Q
Revised: Q

Original: Outputs of Last LQD Layer
Revised: Outputs of Last LQD Layer

Original: Perturbation
Revised: Perturbation

Original: K
Revised: K

Original: V
Revised: V

Original: Q
Revised: Q

Original: NME
Revised: NME

Original: LQD
Revised: LQD

Original: Feature Jittering
Revised: Feature Jittering

Original: Reconstructed Tokens
Revised: Reconstructed Tokens

Original: Difference
Revised: Difference

Original: Feature Tokens
Revised: Feature Tokens

Original: Reconstructed Tokens
Revised: Reconstructed Tokens

Original: +
Revised: +

Original: Localization Results
Revised: Localization Results

Original: Avg Pool & Max
Revised: Avg Pool & Max

Original: Detection Results
Revised: Detection Results

Original: Train on Normal Samples
Revised: Train on Normal Samples

Original: Test for Anomaly Localization & Detection + + + + + - + Add Followed by Layer Norm
Revised: Test for Anomaly Localization & Detection + + + + + - + Add Followed by Layer Norm

Original: [Figure]
Revised: [Figure]

Revised: the by averaging the unified a smaller worse

Original: [Figure]
Revised: [Figure]

Original: Relation between the “identical shortcut” problem and the unified case .

Original: We compare the training curves of MLP between the unified case and the separate case.
Revised: We conduct the same experiment under the separate case on MLP.

Original: As shown in Fig.
Revised: As shown in Fig.

Original: 4, the accuracy (red for localization and green for detection) keeps growing up along with the loss (blue) getting smaller.
Revised: 4, the accuracy (red for localization and green for detection) keeps growing up along with the loss (blue) getting smaller.

Original: This helps reveal the relation between the “identical shortcut” problem and the unified case, which is that the unified case is more challenging and hence magnifies the “identical shortcut” problem .
Revised: This helps reveal the relation between the “identical shortcut” problem and the unified case, which is that the unified case is more challenging and hence magnifies the “identical shortcut” problem .

Original: Therefore, since our approach is specially designed to solve the “identical shortcut” problem, our method can be effective in the unified case.
Revised: Therefore, since our approach is specially designed to solve the “identical shortcut” problem, our method can be effective in the unified case.

Original: 3.2 Improving feature reconstruction for unified anomaly detection
Revised: 3.2 Improving feature reconstruction for unified anomaly detection

Original: Overview .
Revised: Overview .

Original: As shown in Fig.
Revised: As shown in Fig.

Original: 3, our UniAD is composed of a Neighbor Masked Encoder (NME) and a Layer-wise Query Decoder (LQD).
Revised: 3, our UniAD is composed of a Neighbor Masked Encoder (NME) and a Layer-wise Query Decoder (LQD).

Original: Firstly, the feature tokens extracted by a fixed pre-trained backbone are further integrated by NME to derive the encoder embeddings.
Revised: Firstly, the feature tokens extracted by a fixed pre-trained backbone are further integrated by NME to derive the encoder embeddings.

Original: Then, in each layer of LQD, a learnable query embedding is successively fused with the encoder embeddings and the outputs of the previous layer (self-fusion for the first layer).
Revised: Then, in each layer of LQD, a learnable query embedding is successively fused with the encoder embeddings and the outputs of the previous layer (self-fusion for the first layer).

Original: The feature fusion is completed by the Neighbor Masked Attention (NMA).
Revised: The feature fusion is completed by the Neighbor Masked Attention (NMA).

Original: The final outputs of LQD are viewed as the reconstructed features.
Revised: The final outputs of LQD are viewed as the reconstructed features.

Original: Also, we propose a Feature Jittering (FJ) strategy to add perturbations to the input features, leading the model to learn normal distribution from the denoising task.
Revised: Also, we propose a Feature Jittering (FJ) strategy to add perturbations to the input features, leading the model to learn normal distribution from the denoising task.

Original: Finally, the results of anomaly localization and detection are obtained through the reconstruction differences.
Revised: Finally, the results of anomaly localization and detection are obtained through the reconstruction differences.

Original: Neighbor masked attention .
Revised: Neighbor masked attention .

Original: We suspect that the full attention in vanilla transformer [43] contributes to the “identical shortcut”.
Revised: We suspect that the full attention in vanilla transformer [43] contributes to the “identical shortcut”.

Original: In full attention, one token is permitted to see itself, so it will be easy to reconstruct by simply copying.
Revised: In full attention, one token is permitted to see itself, so it will be easy to reconstruct by simply copying.

Original: Moreover, considering that the feature tokens are extracted by a
Revised: Moreover, considering that the feature tokens are extracted by a

Original: CNN backbone, the neighbor tokens must share lots of similarities.
Revised: CNN backbone, the neighbor tokens must share lots of similarities.

Original: Therefore, we propose to mask the neighbor tokens when calculating the attention map, called Neighbor Masked Attention (NMA).
Revised: Therefore, we propose to mask the neighbor tokens when calculating the attention map, called Neighbor Masked Attention (NMA).

Original: Note that the neighbor region is defined in the 2D space, as shown in Fig.
Revised: Note that the neighbor region is defined in the 2D space, as shown in Fig.

Original: Neighbor masked encoder .
Revised: Neighbor masked encoder .

Original: The encoder follows the standard architecture in vanilla transformer.
Revised: The encoder follows the standard architecture in vanilla transformer.

Original: Each layer consists of an attention module and a Feed-Forward Network (FFN).
Revised: Each layer consists of an attention module and a Feed-Forward Network (FFN).

Original: However, the full attention is replaced by our proposed NMA to prevent the information leak.
Revised: However, the full attention is replaced by our proposed NMA to prevent the information leak.

Original: Layer-wise query decoder .
Revised: Layer-wise query decoder .

Original: It is analyzed in Sec.
Revised: It is analyzed in Sec.

Original: 3.1 that the query embedding could help prevent reconstructing
Revised: 3.1 that the query embedding could help prevent reconstructing

Original: anomalies well.
Revised: anomalies well.

Original: However, there is only one query embedding in the vanilla transformer.
Revised: However, there is only one query embedding in the vanilla transformer.

Original: Therefore, we design a Layer-wise Query Decoder (LQD) to intensify the use of query embedding, as shown in Fig.
Revised: Therefore, we design a Layer-wise Query Decoder (LQD) to intensify the use of query embedding, as shown in Fig.

Original: Specifically, in each layer of LQD, a learnable query embedding is first fused with the encoder embeddings, then integrated with the outputs of the previous layer (self-integration for the first layer).
Revised: Specifically, in each layer of LQD, a learnable query embedding is first fused with the encoder embeddings, then integrated with the outputs of the previous layer (self-integration for the first layer).

Original: The feature fusion is implemented by NMA.
Revised: The feature fusion is implemented by NMA.

Original: Following the vanilla transformer, a 2-layer FFN is applied to handle these fused tokens, and the residual connection is utilized to facilitate the training.
Revised: Following the vanilla transformer, a 2-layer FFN is applied to handle these fused tokens, and the residual connection is utilized to facilitate the training.

Original: The final outputs of LQD serve as the reconstructed features.
Revised: The final outputs of LQD serve as the reconstructed features.

Original: Feature jittering .
Revised: Feature jittering .

Original: Inspired by Denoising Auto-Encoder (DAE) [3, 44], we add perturbations to feature tokens, guiding the model to learn knowledge of normal samples by the denoising task.
Revised: Inspired by Denoising Auto-Encoder (DAE) [3, 44], we add perturbations to feature tokens, guiding the model to learn knowledge of normal samples by the denoising task.

Original: Specifically, for a feature token, f tok ∈ R C , we sample the disturbance D from a Gaussian distribution,
Revised: Specifically, for a feature token, f tok ∈ R C , we sample the disturbance D from a Gaussian distribution,

Original: [Equation]
Revised: [Equation]

Original: where α is the jittering scale to control the noisy degree.
Revised: where α is the jittering scale to control the noisy degree.

Original: Also, the sampled disturbance is added to f tok with a fixed jittering probability, p .
Revised: Also, the sampled disturbance is added to f tok with a fixed jittering probability, p .

Original: 3.3 Implementation details
Revised: 3.3 Implementation details

Original: Feature extraction .
Revised: Feature extraction .

Original: We adopt a fixed EfficientNet-b4 [41] pre-trained on ImageNet [14] as the feature extractor.
Revised: We adopt a fixed EfficientNet-b4 [41] pre-trained on ImageNet [14] as the feature extractor.

Original: The features from stage-1 to stage-4 are selected.
Revised: The features from stage-1 to stage-4 are selected.

Original: Here the stage means the combination of blocks that have the same size of feature maps.
Revised: Here the stage means the combination of blocks that have the same size of feature maps.

Original: Then these features are resized to the same size, and concatenated along channel dimension to form a feature map, f org ∈ R C org × H × W .
Revised: Then these features are resized to the same size, and concatenated along channel dimension to form a feature map, f org ∈ R C org × H × W .

Original: Feature reconstruction .
Revised: Feature reconstruction .

Original: The feature map, f org , is first tokenized to H × W feature tokens, followed by a linear projection to reduce C org to a smaller channel, C .
Revised: The feature map, f org , is first tokenized to H × W feature tokens, followed by a linear projection to reduce C org to a smaller channel, C .

Original: Then these tokens are processed by
Revised: Then these tokens are processed by

Original: NME and LQD.
Revised: NME and LQD.

Original: The learnable position embeddings [15, 16] are added in attention modules to inform the spatial information.
Revised: The learnable position embeddings [15, 16] are added in attention modules to inform the spatial information.

Original: Afterward, another linear projection is used to recover the channel from C to
Revised: Afterward, another linear projection is used to recover the channel from C to

Original: C org .
Revised: C org .

Original: After reshape, the reconstructed feature map, f rec ∈ R C org × H × W , is finally obtained.
Revised: After reshape, the reconstructed feature map, f rec ∈ R C org × H × W , is finally obtained.

Original: Objective function .
Revised: Objective function .

Original: Our model is trained with the MSE loss as,
Revised: Our model is trained with the MSE loss as,

Original: [Equation]
Revised: [Equation]

Original: Inference for anomaly localization .
Revised: Inference for anomaly localization .

Original: The result of anomaly localization is an anomaly score map, which assigns an anomaly score for each pixel.
Revised: The result of anomaly localization is an anomaly score map, which assigns an anomaly score for each pixel.

Original: Specifically, the anomaly score map, s , is calculated as the L2 norm of the reconstruction differences as,
Revised: Specifically, the anomaly score map, s , is calculated as the L2 norm of the reconstruction differences as,

Original: [Equation]
Revised: [Equation]

Original: Then s is up-sampled to the image size with bi-linear interpolation to obtain the localization results.
Revised: Then s is up-sampled to the image size with bi-linear interpolation to obtain the localization results.

Original: Inference for anomaly detection .
Revised: Inference for anomaly detection .

Original: Anomaly detection aims to detect whether an image contains anomalous regions.
Revised: Anomaly detection aims to detect whether an image contains anomalous regions.

Original: We transform the anomaly score map, s , to the anomaly score of the image by taking the maximum value of the averagely pooled s .
Revised: We transform the anomaly score map, s , to the anomaly score of the image by taking the maximum value of the averagely pooled s .

Original:  Experiment 4.1 Datasets and metrics
Revised:  Experiment 4.1 Datasets and metrics

Original: MVTec-AD [4] is a comprehensive, multi-object, multi-defect industrial anomaly detection dataset with 15 classes.
Revised: MVTec-AD [4] is a comprehensive, multi-object, multi-defect industrial anomaly detection dataset with 15 classes.

Original: For each anomalous sample in the test set, the ground-truth includes both image
Revised: For each anomalous sample in the test set, the ground-truth includes both image

Original: [Table]
Revised: [Table]

Original: label and anomaly segmentation.
Revised: label and anomaly segmentation.

Original: In the existing literature, only the separate case is researched.
Revised: In the existing literature, only the separate case is researched.

Original: In this paper, we introduce the unified case, where only one model is used to handle all categories.
Revised: In this paper, we introduce the unified case, where only one model is used to handle all categories.

Original: CIFAR-10 [24] is a classical image classification dataset with 10 categories.
Revised: CIFAR-10 [24] is a classical image classification dataset with 10 categories.

Original: Existing methods [6, 25, 38] evaluate CIFAR-10 mainly in the one-versus-many setting, where one class is viewed as normal samples, and others serve as anomalies.
Revised: Existing methods [6, 25, 38] evaluate CIFAR-10 mainly in the one-versus-many setting, where one class is viewed as normal samples, and others serve as anomalies.

Original: Semantic AD [1, 10] proposes a many-versus-one setting, treating one class as anomalous and the remaining classes as normal.
Revised: Semantic AD [1, 10] proposes a many-versus-one setting, treating one class as anomalous and the remaining classes as normal.

Original: Different from both, we propose a unified case ( many-versus-many setting), which is detailed in Sec. 4.4.
Revised: Different from both, we propose a unified case ( many-versus-many setting), which is detailed in Sec. 4.4.

Original: Metrics .
Revised: Metrics .

Original: Following prior works [4, 6, 53], the Area Under the Receiver Operating Curve (AUROC) is used as the evaluation metric for anomaly detection.
Revised: Following prior works [4, 6, 53], the Area Under the Receiver Operating Curve (AUROC) is used as the evaluation metric for anomaly detection.

Original: 4.2 Anomaly detection on MVTec-AD
Revised: 4.2 Anomaly detection on MVTec-AD

Original: Setup .
Revised: Setup .

Original: Anomaly detection aims to detect whether an image contains anomalous regions.
Revised: Anomaly detection aims to detect whether an image contains anomalous regions.

Original: The anomaly detection performance is evaluated on MVTec-AD [4].
Revised: The anomaly detection performance is evaluated on MVTec-AD [4].

Original: The image size is selected as 224 × 224 , and the size for resizing feature maps is set as 14 × 14 .
Revised: The image size is selected as 224 × 224 , and the size for resizing feature maps is set as 14 × 14 .

Original: The feature maps from stage-1 to stage-4 of
Revised: The feature maps from stage-1 to stage-4 of

Original: EfficientNet-b4 [41] are resized and concatenated together to form a 272-channel feature map.
Revised: EfficientNet-b4 [41] are resized and concatenated together to form a 272-channel feature map.

Original: The reduced channel dimension is set as 256.
Revised: The reduced channel dimension is set as 256.

Original: AdamW optimizer [22] with weight decay 1 × 10 − 4 is used.
Revised: AdamW optimizer [22] with weight decay 1 × 10 − 4 is used.

Original: Our model is trained for 1000 epochs on 8 GPUs (NVIDIA Tesla V100 16GB) with batch size 64.
Revised: Our model is trained for 1000 epochs on 8 GPUs (NVIDIA Tesla V100 16GB) with batch size 64.

Original: The learning rate is 1 × 10 − 4 initially, and dropped by 0.1 after 800 epochs.
Revised: The learning rate is 1 × 10 − 4 initially, and dropped by 0.1 after 800 epochs.

Original: The layer numbers of the encoder and decoder are both 4.
Revised: The layer numbers of the encoder and decoder are both 4.

Original: The neighbor size, jittering scale, and jittering probability are set as 7 × 7, 20, and 1, respectively.
Revised: The neighbor size, jittering scale, and jittering probability are set as 7 × 7, 20, and 1, respectively.

Original: The evaluation is run with 5 random seeds.
Revised: The evaluation is run with 5 random seeds.

Original: In both the separate case and the unified case, the reconstruction models are trained from the scratch.
Revised: In both the separate case and the unified case, the reconstruction models are trained from the scratch.

Original: Baselines .
Revised: Baselines .

Original: Our approach is compared with baselines including: US [6], PSVDD [49], PaDiM [11],
Revised: Our approach is compared with baselines including: US [6], PSVDD [49], PaDiM [11],

Original: CutPaste [26], MKD [38], and DRAEM [53].
Revised: CutPaste [26], MKD [38], and DRAEM [53].

Original: Under the separate case, the baselines’ metric is reported in their papers except the metric of US borrowed from [53].
Revised: Under the separate case, the baselines’ metric is reported in their papers except the metric of US borrowed from [53].

Original: Under the unified case, US,
Revised: Under the unified case, US,

Original: PSVDD, PaDiM, CutPaste, MKD, and DRAEM are run with the publicly available implementations.
Revised: PSVDD, PaDiM, CutPaste, MKD, and DRAEM are run with the publicly available implementations.

Original: Quantitative results of anomaly detection on MVTec-AD [4] are shown in Tab. 1.
Revised: Quantitative results of anomaly detection on MVTec-AD [4] are shown in Tab. 1.

Original: Though all baselines achieve excellent performances under the separate case, their performances drop dramatically under the unified case.
Revised: Though all baselines achieve excellent performances under the separate case, their performances drop dramatically under the unified case.

Original: The previous SOTA, DRAEM, a reconstruction-based method trained by pseudo-anomaly, suffers from a drop of near 10%.
Revised: The previous SOTA, DRAEM, a reconstruction-based method trained by pseudo-anomaly, suffers from a drop of near 10%.

Original: For another strong baseline, CutPaste, a pseudo-anomaly approach, the drop is as large as 18.6%.
Revised: For another strong baseline, CutPaste, a pseudo-anomaly approach, the drop is as large as 18.6%.

Original: However, our UniAD has almost no performance drop from the separate case (96.6%) to the unified case (96.5%).
Revised: However, our UniAD has almost no performance drop from the separate case (96.6%) to the unified case (96.5%).

Original: Moreover, we beat the best competitor, DRAEM, by a dramatically large margin (8.4%), demonstrating our superiority.
Revised: Moreover, we beat the best competitor, DRAEM, by a dramatically large margin (8.4%), demonstrating our superiority.

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: The approach to visualizing reconstruction is the same as the one used in Fig
Revised: The approach to visualizing reconstruction is the same as the one used in Fig

Original: 4.3 Anomaly localization on MVTec-AD
Revised: 4.3 Anomaly localization on MVTec-AD

Original: Setup and baselines .
Revised: Setup and baselines .

Original: Anomaly localization aims to localize anomalous regions in an anomalous image.
Revised: Anomaly localization aims to localize anomalous regions in an anomalous image.

Original: MVTec-AD [4] is chosen as the benchmark dataset.
Revised: MVTec-AD [4] is chosen as the benchmark dataset.

Original: The setup is the same as that in Sec. 4.2.
Revised: The setup is the same as that in Sec. 4.2.

Original: Besides the competitors in Sec.
Revised: Besides the competitors in Sec.

Original: 4.2, FCDD [28] is included, whose metric under the separate case is reported in its paper.
Revised: 4.2, FCDD [28] is included, whose metric under the separate case is reported in its paper.

Original: Under the unified case, we run FCDD with the implementation: FCDD.
Revised: Under the unified case, we run FCDD with the implementation: FCDD.

Original: Quantitative results of anomaly localization on MVTec-AD [4] are reported in Tab. 2.
Revised: Quantitative results of anomaly localization on MVTec-AD [4] are reported in Tab. 2.

Original: Similar to Sec.
Revised: Similar to Sec.

Original: 4.2, switching from the separate case to the unified case, the performance of all competitors drops significantly.
Revised: 4.2, switching from the separate case to the unified case, the performance of all competitors drops significantly.

Original: For example, the performance of US, an important distillation-based baseline, decreases by 12.1%.
Revised: For example, the performance of US, an important distillation-based baseline, decreases by 12.1%.

Original: FCDD, a pseudo-anomaly approach, suffers from a dramatic drop of 28.7%, reflecting the pseudo-anomaly is not suitable for the unified case.
Revised: FCDD, a pseudo-anomaly approach, suffers from a dramatic drop of 28.7%, reflecting the pseudo-anomaly is not suitable for the unified case.

Original: However, our UniAD even gains a slight improvement from the separate case (96.6%) to the unified case (96.8%), proving the suitability of our UniAD for the unified case.
Revised: However, our UniAD even gains a slight improvement from the separate case (96.6%) to the unified case (96.8%), proving the suitability of our UniAD for the unified case.

Original: Moreover, we significantly surpass the strongest baseline, PaDiM, by 7.3%.
Revised: Moreover, we significantly surpass the strongest baseline, PaDiM, by 7.3%.

Original: This significant improvement reflects the effectiveness of our model.
Revised: This significant improvement reflects the effectiveness of our model.

Original: Qualitative results for anomaly localization on MVTec-AD [4] are illustrated in Fig.
Revised: Qualitative results for anomaly localization on MVTec-AD [4] are illustrated in Fig.

Original: For both global (Fig. 6a) and local (Fig. 6b) structural anomalies, both scattered texture perturbations (Fig.
Revised: For both global (Fig. 6a) and local (Fig. 6b) structural anomalies, both scattered texture perturbations (Fig.

Original: 6c) and multiple texture scratches (Fig.
Revised: 6c) and multiple texture scratches (Fig.

Original: 6d), our method could successfully reconstruct anomalies to their corresponding normal samples, then accurately localize anomalous regions through reconstruction differences.
Revised: 6d), our method could successfully reconstruct anomalies to their corresponding normal samples, then accurately localize anomalous regions through reconstruction differences.

Original: More qualitative results are given in Appendix .
Revised: More qualitative results are given in Appendix .

Original: 4.4 Anomaly detection on CIFAR- 
Revised: 4.4 Anomaly detection on CIFAR-

Original: Setup .
Revised: Setup .

Original: To further verify the effectiveness of our UniAD, we extend CIFAR-10 [24] to the unified case, which consists of four combinations.
Revised: To further verify the effectiveness of our UniAD, we extend CIFAR-10 [24] to the unified case, which consists of four combinations.

Original: For each combination, five categories together serve as normal samples, while other categories are viewed as anomalies.
Revised: For each combination, five categories together serve as normal samples, while other categories are viewed as anomalies.

Original: The class indices of the four
Revised: The class indices of the four

Original: [Table]
Revised: [Table]

Original: [Table]
Revised: [Table]

Original: combinations are { 01234 } , { 56789 } , { 02468 } , { 13579 } .
Revised: combinations are { 01234 } , { 56789 } , { 02468 } , { 13579 } .

Original: Here, { 01234 } means the normal samples include images from class 0 , 1 , 2 , 3 , 4 , and similar for others.
Revised: Here, { 01234 } means the normal samples include images from class 0 , 1 , 2 , 3 , 4 , and similar for others.

Original: Note that the class index is obtained by sorting the class names of 10 classes.
Revised: Note that the class index is obtained by sorting the class names of 10 classes.

Original: The setup of the model is detailed in Appendix .
Revised: The setup of the model is detailed in Appendix .

Original: Baselines .
Revised: Baselines .

Original: US [6], FCDD [28], FCDD+OE [28], PANDA [34], and MKD [38] serve as competitors.
Revised: US [6], FCDD [28], FCDD+OE [28], PANDA [34], and MKD [38] serve as competitors.

Original: US, FCDD, FCDD+OE, PANDA, and MKD are run with the publicly available implementations.
Revised: US, FCDD, FCDD+OE, PANDA, and MKD are run with the publicly available implementations.

Original: Quantitative results of anomaly detection on CIFAR-10 [24] are shown in Tab. 3.
Revised: Quantitative results of anomaly detection on CIFAR-10 [24] are shown in Tab. 3.

Original: When five classes together serve as normal samples, two recent baselines, US and FCDD, almost lose their ability to detect anomalies.
Revised: When five classes together serve as normal samples, two recent baselines, US and FCDD, almost lose their ability to detect anomalies.

Original: When utilizing 10000 images sampled from CIFAR-100 [24] as auxiliary
Revised: When utilizing 10000 images sampled from CIFAR-100 [24] as auxiliary

Original: Outlier Exposure (OE), FCDD+OE improves the performance by a large margin.
Revised: Outlier Exposure (OE), FCDD+OE improves the performance by a large margin.

Original: We still stably outperform FCDD+OE by 8.3% without the help of OE, indicating the efficacy of our UniAD.
Revised: We still stably outperform FCDD+OE by 8.3% without the help of OE, indicating the efficacy of our UniAD.

Original: 4.5 Comparison with transformer-based competitors
Revised: 4.5 Comparison with transformer-based competitors

Original: As described in Sec. 2, some attempts [32, 29, 51] also try to utilize transformer for anomaly detection.
Revised: As described in Sec. 2, some attempts [32, 29, 51] also try to utilize transformer for anomaly detection.

Original: Here we compare our UniAD with existing transformer-based competitors on MVTec-AD [4].
Revised: Here we compare our UniAD with existing transformer-based competitors on MVTec-AD [4].

Original: Recall that, we choose transformer as the reconstruction model considering its great potential in preventing the model from learning the “identical shortcut” (refer to Sec. 3.1).
Revised: Recall that, we choose transformer as the reconstruction model considering its great potential in preventing the model from learning the “identical shortcut” (refer to Sec. 3.1).

Original: Concretely, we find that the learnable query embedding is essential for avoiding such a shortcut but is seldom explored in existing transformer-based approaches.
Revised: Concretely, we find that the learnable query embedding is essential for avoiding such a shortcut but is seldom explored in existing transformer-based approaches.

Original: As shown in Tab. 4, after introducing even only one query embedding, our baseline already outperforms existing alternatives by a sufficiently large margin in the unified setting.
Revised: As shown in Tab. 4, after introducing even only one query embedding, our baseline already outperforms existing alternatives by a sufficiently large margin in the unified setting.

Original: Our proposed three components further improve our strong baseline .
Revised: Our proposed three components further improve our strong baseline .

Original: Recall that all three components are proposed to avoid the model from directly outputting the inputs.
Revised: Recall that all three components are proposed to avoid the model from directly outputting the inputs.

Original: 4.6 Ablation studies
Revised: 4.6 Ablation studies

Original: To verify the effectiveness of the proposed modules and the selection of hyperparameters, we implement extensive ablation studies on MVTec-AD [4] under the unified case.
Revised: To verify the effectiveness of the proposed modules and the selection of hyperparameters, we implement extensive ablation studies on MVTec-AD [4] under the unified case.

Original: Layer-wise query .
Revised: Layer-wise query .

Original: Tab.
Revised: Tab.

Original: 5a verifies our assertion that the query embedding is of vital significance.
Revised: 5a verifies our assertion that the query embedding is of vital significance.

Original: 1) Without query embedding, meaning the encoder embeddings are directly input to the decoder, the performance is the worst. 2)
Revised: 1) Without query embedding, meaning the encoder embeddings are directly input to the decoder, the performance is the worst. 2)

Original: Adding only one query embedding to the first decoder layer ( i.e. , vanilla transformer [43]) promotes the performance dramatically by 13.4% and 18.1% in anomaly localization and detection, respectively.
Revised: Adding only one query embedding to the first decoder layer ( i.e. , vanilla transformer [43]) promotes the performance dramatically by 13.4% and 18.1% in anomaly localization and detection, respectively.

Original: With layer-wise query embedding in each decoder layer, pixel-level and image-level AUROC is further improved by 3.7% and 7.4%, respectively.
Revised: With layer-wise query embedding in each decoder layer, pixel-level and image-level AUROC is further improved by 3.7% and 7.4%, respectively.

Original: Layer number .
Revised: Layer number .

Original: We conduct experiments to investigate the influence of layer number, as shown in Tab. 5b.
Revised: We conduct experiments to investigate the influence of layer number, as shown in Tab. 5b.

Original: 1) No matter with which combination, our model outperforms vanilla transformer by a large margin, reflecting the effectiveness of our design.
Revised: 1) No matter with which combination, our model outperforms vanilla transformer by a large margin, reflecting the effectiveness of our design.

Original: The best performance is achieved with a
Revised: The best performance is achieved with a

Original: [Table] [Table]
Revised: [Table] [Table]

Original: moderate layer number: 4Enc+4Dec.
Revised: moderate layer number: 4Enc+4Dec.

Original: A larger layer number like 6Enc+6Dec does not bring further promotion, which may be because more layers are harder to train.
Revised: A larger layer number like 6Enc+6Dec does not bring further promotion, which may be because more layers are harder to train.

Original: Neighbor masked attention .
Revised: Neighbor masked attention .

Original: The effectiveness of NMA is proven in Tab.
Revised: The effectiveness of NMA is proven in Tab.

Original: 5a.
Revised: 5a.

Original: Under the case of one query embedding, adding NMA brings promotion by 3.5% for localization and 8.5% for detection.
Revised: Under the case of one query embedding, adding NMA brings promotion by 3.5% for localization and 8.5% for detection.

Original: The neighbor size of NMA is selected in Tab. 5c.
Revised: The neighbor size of NMA is selected in Tab. 5c.

Original: 1 × 1 neighbor size is the worst, because 1 × 1 is too small to prevent the information leak, thus the recovery could be completed by copying neighbor regions.
Revised: 1 × 1 neighbor size is the worst, because 1 × 1 is too small to prevent the information leak, thus the recovery could be completed by copying neighbor regions.

Original: A larger neighbor size ( ≥ 5 × 5) is obviously much better, and the best one is selected as 7 × 7. 3)
Revised: A larger neighbor size ( ≥ 5 × 5) is obviously much better, and the best one is selected as 7 × 7. 3)

Original: We also study the place to add NMA in Tab. 5d.
Revised: We also study the place to add NMA in Tab. 5d.

Original: Only adding NMA in the encoder (Enc) is not enough.
Revised: Only adding NMA in the encoder (Enc) is not enough.

Original: The performance could be stably improved when further adding NMA in the first or second attention in the decoder (Enc+Dec1, Enc+Dec2) or both (All).
Revised: The performance could be stably improved when further adding NMA in the first or second attention in the decoder (Enc+Dec1, Enc+Dec2) or both (All).

Original: This reflects that the full attention of the decoder also contributes to the information leak.
Revised: This reflects that the full attention of the decoder also contributes to the information leak.

Original: Feature jittering .
Revised: Feature jittering .

Original: 1) Tab.
Revised: 1) Tab.

Original: 5a confirms the efficacy of FJ.
Revised: 5a confirms the efficacy of FJ.

Original: With one query embedding as the baseline, introducing FJ could bring an increase of 3.0% for localization and 7.4% for detection, respectively.
Revised: With one query embedding as the baseline, introducing FJ could bring an increase of 3.0% for localization and 7.4% for detection, respectively.

Original: According to Tab.
Revised: According to Tab.

Original: 5e, the jittering scale, α , is chosen as 20.
Revised: 5e, the jittering scale, α , is chosen as 20.

Original: A larger α ( i.e. , 30) disturbs the feature too much, degrading the results.
Revised: A larger α ( i.e. , 30) disturbs the feature too much, degrading the results.

Original: In Tab.
Revised: In Tab.

Original: 5f, the jittering probability, p , is studied.
Revised: 5f, the jittering probability, p , is studied.

Original: In essence, the task would be a denoising task with feature jittering, and be a reconstruction task without feature jittering.
Revised: In essence, the task would be a denoising task with feature jittering, and be a reconstruction task without feature jittering.

Original: The results show that the full denoising task ( i.e. , p = 1 ) is the best.
Revised: The results show that the full denoising task ( i.e. , p = 1 ) is the best.

Original:  Conclusion
Revised:  Conclusion

Original: In this work, we propose UniAD that unifies anomaly detection regarding multiple classes.
Revised: In this work, we propose UniAD that unifies anomaly detection regarding multiple classes.

Original: For such a challenging task, we assist the model against learning an “identical shortcut” with three improvements.
Revised: For such a challenging task, we assist the model against learning an “identical shortcut” with three improvements.

Original: First, we confirm the effectiveness of the learnable query embedding and carefully tailor a layer-wise query decoder to help model the complex distribution of multi-class data.
Revised: First, we confirm the effectiveness of the learnable query embedding and carefully tailor a layer-wise query decoder to help model the complex distribution of multi-class data.

Original: Second, we come up with a neighbor masked attention module to avoid the information leak from the input to the output.
Revised: Second, we come up with a neighbor masked attention module to avoid the information leak from the input to the output.

Original: Third, we propose feature jittering that helps the model less sensitive to the input perturbations.
Revised: Third, we propose feature jittering that helps the model less sensitive to the input perturbations.

Original: Under the unified task setting, our method achieves state-of-the-art performance on MVTec-AD and CIFARdatasets, significantly outperforming existing alternatives.
Revised: Under the unified task setting, our method achieves state-of-the-art performance on MVTec-AD and CIFARdatasets, significantly outperforming existing alternatives.

Original: Discussion .
Revised: Discussion .

Original: In this work, different kinds of objects are handled without being distinguished.
Revised: In this work, different kinds of objects are handled without being distinguished.

Original: We have not used the category labels that may help the model better fit multi-class data.
Revised: We have not used the category labels that may help the model better fit multi-class data.

Original: How to incorporate the unified model with category labels should be further studied.
Revised: How to incorporate the unified model with category labels should be further studied.

Original: In practical uses, normal samples are not as consistent as those in MVTec-AD, often manifest themselves in some diversity.
Revised: In practical uses, normal samples are not as consistent as those in MVTec-AD, often manifest themselves in some diversity.

Original: Our UniAD could handle all 15 categories in MVTec-AD, hence would be more suitable for real scenes.
Revised: Our UniAD could handle all 15 categories in MVTec-AD, hence would be more suitable for real scenes.

Original: However, anomaly detection may be used for video surveillance, which may infringe personal privacy.
Revised: However, anomaly detection may be used for video surveillance, which may infringe personal privacy.

Original: Acknowledgments and Disclosure of Funding
Revised: Acknowledgments and Disclosure of Funding

Original: Acknowledgement.
Revised: Acknowledgement.

Original: This work is sponsored by the National Key Research and Development Program of China (2021YFB1716000) and National Natural Science Foundation of China (62176152).
Revised: This work is sponsored by the National Key Research and Development Program of China (2021YFB1716000) and National Natural Science Foundation of China (62176152).

Original: Checklist
Revised: Checklist

Original: For all authors..
Revised: For all authors..

Original: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
Revised: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?

Original: [Yes] (b) Did you describe the limitations of your work?
Revised: [Yes] (b) Did you describe the limitations of your work?

Original: [Yes] See Discussion in Sec.
Revised: [Yes] See Discussion in Sec.

Original: (c) Did you discuss any potential negative societal impacts of your work?
Revised: (c) Did you discuss any potential negative societal impacts of your work?

Original: [Yes] See Discussion in Sec.
Revised: [Yes] See Discussion in Sec.

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
Revised: (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Original: [Yes] 2.
Revised: [Yes] 2.

Original: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
Revised: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?

Original: [N/A] (b) Did you include complete proofs of all theoretical results?
Revised: [N/A] (b) Did you include complete proofs of all theoretical results?

Original: [N/A] 3.
Revised: [N/A] 3.

Original: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
Revised: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original: [Yes] See Sec. 4 and Appendix .
Revised: [Yes] See Sec. 4 and Appendix .

Original: Also, we have released the code.
Revised: Also, we have released the code.

Original: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
Revised: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?

Original: [Yes] See Sec.
Revised: [Yes] See Sec.

Original: 4.1 & Sec.
Revised: 4.1 & Sec.

Original: 4.4 for data splits, and Sec. 4.6 for the choice of hyperparameters.
Revised: 4.4 for data splits, and Sec. 4.6 for the choice of hyperparameters.

Original: (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
Revised: (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

Original: [Yes] See Tab. 1, Tab. 2, and Tab. 3.
Revised: [Yes] See Tab. 1, Tab. 2, and Tab. 3.

Original: (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
Revised: (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?

Original: [Yes] See Setup in Sec.
Revised: [Yes] See Setup in Sec.

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?

Original: [Yes] See Sec.
Revised: [Yes] See Sec.

Original: 4.1 and Baselines in Sec.
Revised: 4.1 and Baselines in Sec.

Original: (b) Did you mention the license of the assets?
Revised: (b) Did you mention the license of the assets?

Original: [N/A] (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: [N/A] (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [N/A] (d) Did you discuss whether
Revised: [N/A] (d) Did you discuss whether

Original: and how consent was obtained from people whose data you’re using/curating?
Revised: and how consent was obtained from people whose data you’re using/curating?

Original: [N/A] (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?
Revised: [N/A] (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?

Original: [N/A] 5.
Revised: [N/A] 5.

Original: If you used crowdsourcing or conducted research with human subjects...
Revised: If you used crowdsourcing or conducted research with human subjects...

Original: (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
Revised: (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Original: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
Revised: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?

Original: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
Revised: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

Original: [N/A]
Revised: [N/A]

Original: Appendix A Discussions about the unified setting
Revised: Appendix A Discussions about the unified setting

Original: Semantic AD [1, 10] also involves multiple classes, which mainly focus on anomaly detection extended from classification datasets ( e.g. , CIFAR-10 [24]).
Revised: Semantic AD [1, 10] also involves multiple classes, which mainly focus on anomaly detection extended from classification datasets ( e.g. , CIFAR-10 [24]).

Original: Semantic AD treats one class as anomalous and the remaining classes as normal.
Revised: Semantic AD treats one class as anomalous and the remaining classes as normal.

Original: Our task setting clearly differs from semantic AD.
Revised: Our task setting clearly differs from semantic AD.

Original: First, we focus more on the industrial anomaly detection dataset, MVTec-AD [4], which is of more practical usage.
Revised: First, we focus more on the industrial anomaly detection dataset, MVTec-AD [4], which is of more practical usage.

Original: Unlike CIFAR-10, each category has normal and abnormal samples in MVTec-AD.
Revised: Unlike CIFAR-10, each category has normal and abnormal samples in MVTec-AD.

Original: We would like to model the joint distribution of normal samples across all categories .
Revised: We would like to model the joint distribution of normal samples across all categories .

Original: It requires the model to learn “what normal samples from each category look like” instead of “what categories are normal”.
Revised: It requires the model to learn “what normal samples from each category look like” instead of “what categories are normal”.

Original: The latter is the main focus of semantic AD.
Revised: The latter is the main focus of semantic AD.

Original: Second, we also differ from semantic AD regarding the CIFAR-10 task setting.
Revised: Second, we also differ from semantic AD regarding the CIFAR-10 task setting.

Original: Semantic AD studies the many-versus-one setting, which treats 9 classes as normal and the remaining class as anomalous.
Revised: Semantic AD studies the many-versus-one setting, which treats 9 classes as normal and the remaining class as anomalous.

Original: In contrast, we study the many-versus-many setting, which treats 5 classes as normal and the otherclasses as anomalous.
Revised: In contrast, we study the many-versus-many setting, which treats 5 classes as normal and the otherclasses as anomalous.

Original: We use such a setting to simulate the real scenario, where both normal and anomalous samples contain multiple classes .
Revised: We use such a setting to simulate the real scenario, where both normal and anomalous samples contain multiple classes .

Original: B Network architecture and training configurations B.1 Reconstruction baselines
Revised: B Network architecture and training configurations B.1 Reconstruction baselines

Original: We present the architectures of the three reconstruction baselines as follows.
Revised: We present the architectures of the three reconstruction baselines as follows.

Original: These baselines share the same assistant modules and training configurations as our UniAD, which are provided in appendix B. and appendix B.4.
Revised: These baselines share the same assistant modules and training configurations as our UniAD, which are provided in appendix B. and appendix B.4.

Original: CNN is designed based on the ResNet-34 [20] by revising the followings.
Revised: CNN is designed based on the ResNet-34 [20] by revising the followings.

Original: We remove the operations before stage-1 (a 7 × 7 convolutional layer followed by batch normalization, ReLU activation, and max-pooling).
Revised: We remove the operations before stage-1 (a 7 × 7 convolutional layer followed by batch normalization, ReLU activation, and max-pooling).

Original: All strides from stage-1 to stage-4 are 1, meaning the size of the feature map is the same.
Revised: All strides from stage-1 to stage-4 are 1, meaning the size of the feature map is the same.

Original: The channel dimensions from stage-1 to stage-4 are respectively C , C/ 2 , C/ 2 , and C , where C is the reduced channel dimension that is 256.
Revised: The channel dimensions from stage-1 to stage-4 are respectively C , C/ 2 , C/ 2 , and C , where C is the reduced channel dimension that is 256.

Original: Transformer follows the architecture of the vanilla transformer [43] with a 4-layer encoder and a 4-layer decoder.
Revised: Transformer follows the architecture of the vanilla transformer [43] with a 4-layer encoder and a 4-layer decoder.

Original: Each encoder layer is composed of a self-attention layer and a feed-forward network.
Revised: Each encoder layer is composed of a self-attention layer and a feed-forward network.

Original: Each decoder layer consists of a self-attention layer, a cross-attention layer, and a feedforward network.
Revised: Each decoder layer consists of a self-attention layer, a cross-attention layer, and a feedforward network.

Original: For the first decoder layer, the inputs of the self-attention layer are the learnable query embeddings.
Revised: For the first decoder layer, the inputs of the self-attention layer are the learnable query embeddings.

Original: While, for other decoder layers, the inputs of the self-attention layer are the outputs of the previous decoder layer.
Revised: While, for other decoder layers, the inputs of the self-attention layer are the outputs of the previous decoder layer.

Original: The outputs of the self-attention layer serve as the query of the cross-attention layer, and the encoder embeddings are used as the key and value of the cross-attention layer.
Revised: The outputs of the self-attention layer serve as the query of the cross-attention layer, and the encoder embeddings are used as the key and value of the cross-attention layer.

Original: Like the vanilla transformer, the residual connection is utilized in the attention module and the feed-forward network.
Revised: Like the vanilla transformer, the residual connection is utilized in the attention module and the feed-forward network.

Original: The learnable position embeddings [15] are added in all attention modules to inform the spatial information.
Revised: The learnable position embeddings [15] are added in all attention modules to inform the spatial information.

Original: The feed-forward network is the same as Tab. A1.
Revised: The feed-forward network is the same as Tab. A1.

Original: MLP is revised from the transformer by substituting all attention layers.
Revised: MLP is revised from the transformer by substituting all attention layers.

Original: The self-attention is replaced by a linear projection, followed by layer normalization and ReLU activation.
Revised: The self-attention is replaced by a linear projection, followed by layer normalization and ReLU activation.

Original: The crossattention between two sets of inputs is changed to a concatenation along the channel dimension and a linear projection, followed by layer normalization and ReLU activation.
Revised: The crossattention between two sets of inputs is changed to a concatenation along the channel dimension and a linear projection, followed by layer normalization and ReLU activation.

Original: The position embeddings are removed because MLP could keep spatial information.
Revised: The position embeddings are removed because MLP could keep spatial information.

Original: B.2 UniAD
Revised: B.2 UniAD

Original: The whole architecture of our UniAD including the neighbor masked encoder and the layer-wise query decoder has been described in the main paper.
Revised: The whole architecture of our UniAD including the neighbor masked encoder and the layer-wise query decoder has been described in the main paper.

Original: Here, the detailed architecture of the feed-forward network is provided in Tab. A1.
Revised: Here, the detailed architecture of the feed-forward network is provided in Tab. A1.

Original: [Table]
Revised: [Table]

Original: B.3 Assistant modules
Revised: B.3 Assistant modules

Original: Feature extraction .
Revised: Feature extraction .

Original: As stated in the main paper, the selected features are resized to the same size, and concatenated along the channel dimension to form a feature map, f org ∈ R C org
Revised: As stated in the main paper, the selected features are resized to the same size, and concatenated along the channel dimension to form a feature map, f org ∈ R C org

Original: × H × W .
Revised: × H × W .

Original: Afterward, this feature map is tokenized to H × W feature tokens with C org channels (no tokenization for CNN).
Revised: Afterward, this feature map is tokenized to H × W feature tokens with C org channels (no tokenization for CNN).

Original: Channel reduction .
Revised: Channel reduction .

Original: A linear projection (or 1 × 1 convolution for CNN) is first applied to reduce C org to a smaller channel, C .
Revised: A linear projection (or 1 × 1 convolution for CNN) is first applied to reduce C org to a smaller channel, C .

Original: Then these features are processed by the reconstruction model, followed by another linear projection to recover the channel from C to C org .
Revised: Then these features are processed by the reconstruction model, followed by another linear projection to recover the channel from C to C org .

Original: Through reshape (CNN does not need reshape), the reconstructed feature map, f rec ∈ R C org × H × W , is finally obtained.
Revised: Through reshape (CNN does not need reshape), the reconstructed feature map, f rec ∈ R C org × H × W , is finally obtained.

Original: Visualization of reconstructed features .
Revised: Visualization of reconstructed features .

Original: We employ a feature reconstruction paradigm, which is harder to visualize compared with image reconstruction.
Revised: We employ a feature reconstruction paradigm, which is harder to visualize compared with image reconstruction.

Original: To intuitively explain the problem of “identical shortcut”, we must render these reconstructed features into images.
Revised: To intuitively explain the problem of “identical shortcut”, we must render these reconstructed features into images.

Original: Therefore, we pre-train a decoder on both normal and anomalous samples to recover the backbone-extracted features to images, then use this decoder to render reconstructed features.
Revised: Therefore, we pre-train a decoder on both normal and anomalous samples to recover the backbone-extracted features to images, then use this decoder to render reconstructed features.

Original: Note that this decoder is only used for visualization.
Revised: Note that this decoder is only used for visualization.

Original: The decoder follows a reversed architecture of ResNet-34 [20].
Revised: The decoder follows a reversed architecture of ResNet-34 [20].

Original: The down-sampling in the original network is replaced by up-sampling implemented by the transposed convolution.
Revised: The down-sampling in the original network is replaced by up-sampling implemented by the transposed convolution.

Original: B.4 Training configurations on CIFAR- 
Revised: B.4 Training configurations on CIFAR-

Original: CIFAR-10 [24].
Revised: CIFAR-10 [24].

Original: The image size and feature size are set as 224 × 224 and 14 × 14 , respectively.
Revised: The image size and feature size are set as 224 × 224 and 14 × 14 , respectively.

Original: Considering that the anomalies in CIFAR-10 are semantically different objects (not structural damages or texture perturbations in MVTec-AD [4]), the features in deep layers containing more semantic information must be helpful.
Revised: Considering that the anomalies in CIFAR-10 are semantically different objects (not structural damages or texture perturbations in MVTec-AD [4]), the features in deep layers containing more semantic information must be helpful.

Original: Therefore, the feature maps from stage-1 to stage-5 are selected.
Revised: Therefore, the feature maps from stage-1 to stage-5 are selected.

Original: These features are resized and concatenated together to form a 720-channel feature map.
Revised: These features are resized and concatenated together to form a 720-channel feature map.

Original: The reduced channel dimension is set as 256.
Revised: The reduced channel dimension is set as 256.

Original: Our model is trained for 1000 epochs on 8 GPUs (NVIDIA Tesla
Revised: Our model is trained for 1000 epochs on 8 GPUs (NVIDIA Tesla

Original: V100 16GB) with batch size 128 by AdamW optimizer [22] (with weight decay 1 × 10 − 4 ).
Revised: V100 16GB) with batch size 128 by AdamW optimizer [22] (with weight decay 1 × 10 − 4 ).

Original: The learning rate is 1 × 10 − 4 initially, and dropped by 0.1 after 800 epochs.
Revised: The learning rate is 1 × 10 − 4 initially, and dropped by 0.1 after 800 epochs.

Original: The layer numbers of the encoder and decoder are both 4.
Revised: The layer numbers of the encoder and decoder are both 4.

Original: The neighbor size, jittering scale, and jittering probability are chosen as 7 × 7, 20, and 1, respectively.
Revised: The neighbor size, jittering scale, and jittering probability are chosen as 7 × 7, 20, and 1, respectively.

Original: The evaluation is run with 5 random seeds.
Revised: The evaluation is run with 5 random seeds.

Original: C Ablation studies
Revised: C Ablation studies

Original: In this part, we make comprehensive analyses on different components of our UniAD.
Revised: In this part, we make comprehensive analyses on different components of our UniAD.

Original: All experiments are implemented on MVTec-AD [4] and evaluated with AUROC under the unified case.
Revised: All experiments are implemented on MVTec-AD [4] and evaluated with AUROC under the unified case.

Original: C.1 Full ablation studies of our three designs
Revised: C.1 Full ablation studies of our three designs

Original: Because of the page limitation, we do not include the full ablation experiments in the main paper.
Revised: Because of the page limitation, we do not include the full ablation experiments in the main paper.

Original: Here we provide the full ablation studies regarding layer-wise query embedding, neighbor masked attention (NMA), and feature jittering (FJ) in Tab. A2.
Revised: Here we provide the full ablation studies regarding layer-wise query embedding, neighbor masked attention (NMA), and feature jittering (FJ) in Tab. A2.

Original: Based on the vanilla transformer with one query embedding (1 query), adding NMA or FJ both could obviously improve the results.
Revised: Based on the vanilla transformer with one query embedding (1 query), adding NMA or FJ both could obviously improve the results.

Original: NMA together with FJ provides a quite strong performance (96.6% for localization and 96.2% for detection).
Revised: NMA together with FJ provides a quite strong performance (96.6% for localization and 96.2% for detection).

Original: Therefore, the effectiveness and the combined effect of
Revised: Therefore, the effectiveness and the combined effect of

Original: NMA and FJ are verified.
Revised: NMA and FJ are verified.

Original: When it comes to layer-wise query embedding, adding NMA brings slight promotion, while adding FJ makes the performance slightly worse.
Revised: When it comes to layer-wise query embedding, adding NMA brings slight promotion, while adding FJ makes the performance slightly worse.

Original: Adding both NMA and FJ could achieve the best results (96.8% for localization and 96.5% for detection).
Revised: Adding both NMA and FJ could achieve the best results (96.8% for localization and 96.5% for detection).

Original: These reflect that, under layer-wise query embedding,
Revised: These reflect that, under layer-wise query embedding,

Original: [Table]
Revised: [Table]

Original: FJ must cooperate with NMA to function, and combining all three components brings the best performance.
Revised: FJ must cooperate with NMA to function, and combining all three components brings the best performance.

Original: C.2 Layer-wise query decoder
Revised: C.2 Layer-wise query decoder

Original: For each decoder layer (except the first one), there are three inputs, the learnable query embedding, the encoder embedding, and the outputs of the previous layer.
Revised: For each decoder layer (except the first one), there are three inputs, the learnable query embedding, the encoder embedding, and the outputs of the previous layer.

Original: These three sources of information should be fused by two attention modules.
Revised: These three sources of information should be fused by two attention modules.

Original: The learnable query embedding must serve as the query of an attention module, while others are not sure.
Revised: The learnable query embedding must serve as the query of an attention module, while others are not sure.

Original: Therefore, there are six combinations in total, as illustrated in Fig.
Revised: Therefore, there are six combinations in total, as illustrated in Fig.

Original: A1.
Revised: A1.

Original: The performances of the six architectures are given in Tab. A3.
Revised: The performances of the six architectures are given in Tab. A3.

Original: First, we focus on the situation where the Neighbor Masked Attention (NMA) and Feature Jittering (FJ) are not adopted.
Revised: First, we focus on the situation where the Neighbor Masked Attention (NMA) and Feature Jittering (FJ) are not adopted.

Original: As shown in Tab. A3, (a) achieves the best results, and (a), (b), and (f) all outperform the vanilla transformer [43].
Revised: As shown in Tab. A3, (a) achieves the best results, and (a), (b), and (f) all outperform the vanilla transformer [43].

Original: These three designs share some common characteristics as followings.
Revised: These three designs share some common characteristics as followings.

Original: The outputs of the previous layer should be input as the key and value of the attention module.
Revised: The outputs of the previous layer should be input as the key and value of the attention module.

Original: The reason might be that being key and value helps aggregate the information layer by layer.
Revised: The reason might be that being key and value helps aggregate the information layer by layer.

Original: If the query embedding is input to the first attention, the results of the first attention should serve as the query of the second attention.
Revised: If the query embedding is input to the first attention, the results of the first attention should serve as the query of the second attention.

Original: We assert that the results of the first attention are semantic-instructed query embedding, which functions the same as the query embedding and should be the query of the second attention.
Revised: We assert that the results of the first attention are semantic-instructed query embedding, which functions the same as the query embedding and should be the query of the second attention.

Original: Then, we add NMA and FJ to the three designs including (a), (b), and (f) that have been proven effective by above experiments.
Revised: Then, we add NMA and FJ to the three designs including (a), (b), and (f) that have been proven effective by above experiments.

Original: Adding NMA and FJ promotes the performance stably.
Revised: Adding NMA and FJ promotes the performance stably.

Original: (a) keeps the best performance, and is our final choice.
Revised: (a) keeps the best performance, and is our final choice.

Original: Attention
Revised: Attention

Original: Attention  3 K, V
Revised: Attention  3 K, V

Original: K, V
Revised: K, V

Original: Q
Revised: Q

Original: Q 1 query embedding 2 encoder embedding 3 outputs of previous LQD layer
Revised: Q 1 query embedding 2 encoder embedding 3 outputs of previous LQD layer

Original: Attention
Revised: Attention

Original: Attention  2 K, V
Revised: Attention  2 K, V

Original: K, V
Revised: K, V

Original: Q Q
Revised: Q Q

Original: Q K, V
Revised: Q K, V

Original: Attention Attention
Revised: Attention Attention

Original: K, V
Revised: K, V

Original: Q
Revised: Q

Original: Q K, V
Revised: Q K, V

Original: Attention Attention
Revised: Attention Attention

Original: K, V
Revised: K, V

Original: Q
Revised: Q

Original: Q K, V
Revised: Q K, V

Original: Attention Attention
Revised: Attention Attention

Original: K, V
Revised: K, V

Original: Q
Revised: Q

Original: Q K, V
Revised: Q K, V

Original: Attention Attention
Revised: Attention Attention

Original: K, V
Revised: K, V

Original: Q
Revised: Q

Original: [Figure]
Revised: [Figure]

Original: Concept of each design can be found in
Revised: Concept of each design can be found in

Original: Vanilla
Revised: Vanilla

Original: NMA FJ (a) (b) (c) (d) (e) (f)
Revised: NMA FJ (a) (b) (c) (d) (e) (f)

Original: [Table]
Revised: [Table]

Original: ✓
Revised: ✓

Original: [Table] [Table]
Revised: [Table] [Table]

Original: C.3 Object function
Revised: C.3 Object function

Original: This section studies the loss function used for feature regression.
Revised: This section studies the loss function used for feature regression.

Original: Here we denote the original features as f org ∈ R C org × H × W , the reconstructed features as f rec ∈ R C org × H × W .
Revised: Here we denote the original features as f org ∈ R C org × H × W , the reconstructed features as f rec ∈ R C org × H × W .

Original: MSE loss is one of the most popular regression or reconstruction loss functions.
Revised: MSE loss is one of the most popular regression or reconstruction loss functions.

Original: It is represented as,
Revised: It is represented as,

Original: [Equation]
Revised: [Equation]

Original: A1)
Revised: A1)

Original: Normalized MSE loss adds a normalization to both feature maps, such that each feature vector in the two feature maps is a unit vector.
Revised: Normalized MSE loss adds a normalization to both feature maps, such that each feature vector in the two feature maps is a unit vector.

Original: The normalized MSE loss is written as,
Revised: The normalized MSE loss is written as,

Original: [Equation]
Revised: [Equation]

Original: A
Revised: A

Original: [Equation]
Revised: [Equation]

Original: Also, when we adopt this loss function, the anomaly localization results should be changed to the L norm of the differences between two normalized features.
Revised: Also, when we adopt this loss function, the anomaly localization results should be changed to the L norm of the differences between two normalized features.

Original: Cosine distance loss is used to minimize the cosine distance between the original features and the reconstructed features.
Revised: Cosine distance loss is used to minimize the cosine distance between the original features and the reconstructed features.

Original: It is denoted as,
Revised: It is denoted as,

Original: [Equation]
Revised: [Equation]

Original: Similarly, to get compatible with the loss function, the anomaly localization results are also obtained through cosine distance.
Revised: Similarly, to get compatible with the loss function, the anomaly localization results are also obtained through cosine distance.

Original: The performances of the three loss functions are provided in Tab. A4.
Revised: The performances of the three loss functions are provided in Tab. A4.

Original: The three loss functions achieve similar results, proving the universality of our UniAD with different loss functions.
Revised: The three loss functions achieve similar results, proving the universality of our UniAD with different loss functions.

Original: We finally choose MSE loss because it is the most commonly adopted regression or reconstruction loss function.
Revised: We finally choose MSE loss because it is the most commonly adopted regression or reconstruction loss function.

Original: [Table]
Revised: [Table]

Original: .4 Backbone
Revised: .4 Backbone

Original: Trainable or frozen .
Revised: Trainable or frozen .

Original: As shown in Tab. A5, if we train the backbone like other modules, the performance drops dramatically on both anomaly detection (-30.6%) and localization (-31.3%).
Revised: As shown in Tab. A5, if we train the backbone like other modules, the performance drops dramatically on both anomaly detection (-30.6%) and localization (-31.3%).

Original: We speculate that training backbone would lead the backbone to extract some indiscriminative features that are easy to reconstruct, which however does not help detect anomalies.
Revised: We speculate that training backbone would lead the backbone to extract some indiscriminative features that are easy to reconstruct, which however does not help detect anomalies.

Original: [Table]
Revised: [Table]

Original: Backbone architecture .
Revised: Backbone architecture .

Original: We evaluate two types of backbones, i.e. , ResNet [20] and EfficientNet [41].
Revised: We evaluate two types of backbones, i.e. , ResNet [20] and EfficientNet [41].

Original: Both are pre-trained on ImageNet.
Revised: Both are pre-trained on ImageNet.

Original: From Tab.
Revised: From Tab.

Original: A6, we have the following observations: 1) EfficientNet performs obviously better than ResNet, especially in anomaly detection.
Revised: A6, we have the following observations: 1) EfficientNet performs obviously better than ResNet, especially in anomaly detection.

Original: The backbone with moderate parameter size is more suitable for anomaly detection, like ResNet-34 in ResNet, and
Revised: The backbone with moderate parameter size is more suitable for anomaly detection, like ResNet-34 in ResNet, and

Original: EfficientNet-b2 or EfficientNet-b4 in EfficientNet.
Revised: EfficientNet-b2 or EfficientNet-b4 in EfficientNet.

Original: The reason might be that too shallow networks could not extract discriminative features, while too deep networks focus more on semantic features, rather than the structural damages or texture perturbations in anomaly detection.
Revised: The reason might be that too shallow networks could not extract discriminative features, while too deep networks focus more on semantic features, rather than the structural damages or texture perturbations in anomaly detection.

Original: We select
Revised: We select

Original: EfficientNet-b4 as the backbone by default.
Revised: EfficientNet-b4 as the backbone by default.

Original: [Table]
Revised: [Table]

Original: D More results D.1 Complexity comparison
Revised: D More results D.1 Complexity comparison

Original: With the image size fixed as 224 × 224 , we compare the inference FLOPs and learnable parameters with all competitors.
Revised: With the image size fixed as 224 × 224 , we compare the inference FLOPs and learnable parameters with all competitors.

Original: We conclude that the advantage of UniAD does not come from a larger model capacity.
Revised: We conclude that the advantage of UniAD does not come from a larger model capacity.

Original: D.2 Visualization results
Revised: D.2 Visualization results

Original: Visualization results of reconstructed features are given in Fig.
Revised: Visualization results of reconstructed features are given in Fig.

Original: A2.
Revised: A2.

Original: The feature visualization follows the approach described in appendix B.3.
Revised: The feature visualization follows the approach described in appendix B.3.

Original: MLP, CNN, and transformer all tend to learn an “identical shortcut”, where the anomalous regions would also be well recovered.
Revised: MLP, CNN, and transformer all tend to learn an “identical shortcut”, where the anomalous regions would also be well recovered.

Original: In contrast, our
Revised: In contrast, our

Original: UniAD overcomes such a problem and manages to reconstruct anomalies as normal samples .
Revised: UniAD overcomes such a problem and manages to reconstruct anomalies as normal samples .

Original: Qualitative results for anomaly localization on MVTec-AD are given in Fig.
Revised: Qualitative results for anomaly localization on MVTec-AD are given in Fig.

Original: A3.
Revised: A3.

Original: All 15 categories are handled by a unified model.
Revised: All 15 categories are handled by a unified model.

Original: For both global (Fig.
Revised: For both global (Fig.

Original: A3i-left, Fig.
Revised: A3i-left, Fig.

Original: A3m-left) and local (Fig.
Revised: A3m-left) and local (Fig.

Original: A3b,
Revised: A3b,

Original: Fig.
Revised: Fig.

Original: A3h) structural anomalies, both additional (Fig.
Revised: A3h) structural anomalies, both additional (Fig.

Original: A3h-left, Fig.
Revised: A3h-left, Fig.

Original: A3f-right) and missing (Fig.
Revised: A3f-right) and missing (Fig.

Original: A3cright, Fig.
Revised: A3cright, Fig.

Original: A3m) anomalies, both tightly aligned objects (Fig.
Revised: A3m) anomalies, both tightly aligned objects (Fig.

Original: A3a, Fig.
Revised: A3a, Fig.

Original: A3b) and randomly placed objects (Fig.
Revised: A3b) and randomly placed objects (Fig.

Original: A3f, Fig.
Revised: A3f, Fig.

Original: A3j), both texture bumps (Fig.
Revised: A3j), both texture bumps (Fig.

Original: A3e-left, Fig.
Revised: A3e-left, Fig.

Original: A3g-left) and texture scratches (Fig.
Revised: A3g-left) and texture scratches (Fig.

Original: A3k-left, Fig.
Revised: A3k-left, Fig.

Original: A3n), both color perturbations (Fig.
Revised: A3n), both color perturbations (Fig.

Original: A3k-right, Fig.
Revised: A3k-right, Fig.

Original: A3h-right) and uneven surface disturbances (Fig.
Revised: A3h-right) and uneven surface disturbances (Fig.

Original: A3d, Fig.
Revised: A3d, Fig.

Original: A3g)
Revised: A3g)

Original: , our method could successfully reconstruct anomalies to their corresponding normal samples, then accurately localize anomalous regions through reconstruction differences.
Revised: , our method could successfully reconstruct anomalies to their corresponding normal samples, then accurately localize anomalous regions through reconstruction differences.

Original: This reflects the effectiveness of our UniAD.
Revised: This reflects the effectiveness of our UniAD.

Original: [Figure]
Revised: [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure]
Revised: [Figure]

Original: ) (n) (
Revised: ) (n) (

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: GT
Revised: GT

Original: Pred
Revised: Pred

Original: [Figure]
Revised: [Figure]

Original: Anomaly Recon
Revised: Anomaly Recon

Original: GT
Revised: GT

Original: Pred
Revised: Pred