Dinomaly: The Less Is More Philosophy in Multi-Class Unsupervised Anomaly Detection

Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-07 11:59:38

Abstract: Recent studies highlighted a practical setting of unsupervised anomaly detection (UAD) that builds a unified model for multi-class images, serving as an alternative to the conventional one-class-one-model setup. Despite various advancements addressing this challenging task, the detection performance under the multi-class setting still lags far behind state-of-the-art class-separated models. Our research aims to bridge this substantial performance gap. In this paper, we introduce a minimalistic reconstruction-based anomaly detection framework, namely Dinomaly, which leverages pure Transformer architectures without relying on complex designs, additional modules, or specialized tricks. Given this powerful framework consisted of only Attentions and MLPs, we found four simple components that are essential to multi-class anomaly detection: (1) _Foundation_ _Transformers_ that extracts universal and discriminative features, (2) _Noisy_ _Bottleneck_ where pre-existing Dropouts do all the noise injection tricks, (3) _Linear_ _Attention_ that naturally cannot focus, and (4) _Loose_ _Reconstruction_ that does not force layer-to-layer and point-by-point reconstruction. Extensive experiments are conducted across three popular anomaly detection benchmarks including MVTec-AD, VisA, and the recently released Real-IAD. Our proposed Dinomaly achieves impressive image AUROC of 99.6\%, 98.7\%, and 89.3\% on the three datasets respectively, which is not only superior to state-of-the-art multi-class UAD methods, but also surpasses the most advanced class-separated UAD records.

Main paper: 
 introduction: 
 Unsupervised anomaly detection (UAD) aims to detect abnormal patterns from normal images and further localize the anomalous regions. Because of the diversity of potential anomalies and their scarcity, this task is proposed to model the accessible training sets containing only normal samples as an unsupervised paradigm. UAD has a wide range of applications, e.g., industrial defect detection, medical disease screening, and video surveillance, addressing the difficulty of collecting and labeling all possible anomalies in these scenarios.Efforts on UAD attempt to learn the distribution of available normal samples. Most advanced methods utilize networks pre-trained on large-scale datasets, e.g. ImageNet [1] , for extracting discriminative and informative feature representations. Specifically, Feature reconstruction [2; 3; 4] and feature distillation methods [5; 6] are proposed to reconstruct features of pre-trained encoders, based on the hypothesis that the networks trained on normal images can only construct normal regions, but fail for unseen anomalous regions. Feature statistics methods [7; 8; 9] memorize and model all anomaly-free features extracted from pre-trained networks in training, and compare them with the test features during inference. Pseudo-anomaly methods [10; 11] generate pseudo defects or noises Submitted to 38th Conference on Neural Information Processing Systems (NeurIPS 2024). Do not distribute. reconstruction objects. Second, as an alternative to carefully designed pseudo anomaly and feature noise, we propose to activate the out-of-the-box Dropout in an MLP to prevent the network from restoring both normal and anomalous patterns, which is previously referred to as identity mapping.Third, we propose to utilize the "side effect" of Linear Attention (a computation-efficient counterpart of Softmax Attention) that makes it hard to focus on local regions, to further alleviate the issue of identity mapping. Fourth, previous methods adopt layer-to-layer and region-by-region reconstruction schemes, distilling a decoder that can well mimic the behavior of the encoder even for anomalous regions. Therefore, we propose to loosen the reconstruction constraints by grouping multiple layers as a whole and discarding well-reconstructed regions during optimization.To validate the effectiveness of the proposed Dinomaly under MUAD setting, we conduct extensive experiments on three widely used industrial defect detection benchmarks, i.e., MVTec AD [20] (15 classes), VisA [21] (12 classes), and recently released Real-IAD (30 classes) . Notably, we achieve unprecedented image-level AUROC of 99.6%, 98.7%, and 89.3% on MVTec AD, VisA, and Real-IAD, respectively, which surpasses previous SoTA methods by a large margin.Related works are presented in Appendix A.1. 
dinomaly framework: 
 "What I cannot create, I do not understand"--Richer FeynmanThe ability to recognize anomalies from what we know is an innate human capability, serving as a vital pathway for us to explore the world. Similarly, we construct a reconstruction-based framework that relies on the epistemic characteristic of artificial neural networks. Dinomaly consists of an encoder, a bottleneck, and a reconstruction decoder, as shown in Figure 2 . Without loss of generality, a standard ViT-Base/14 network [28] with 12 Transformer layers is used as the encoder, extracting informative feature maps with different semantic scales. The bottleneck is a simple MLP (a.k.a. feed-forward network, FFN) that collects the feature representations of the encoder's 8 middle-level layers. The decoder is similar to the encoder, consisting of 8 Transformer layers. During training, the decoder learns to reconstruct the middle-level features of the encoder by maximizing the cosine similarity between feature maps. During inference, the decoder is expected to reconstruct normal regions of feature maps but fails for anomalous regions as it has never seen such samples.Foundation Transformers. Foundation models, especially ViTs [28; 31] pre-trained on large-scale datasets, serve as a basis and starting point for specific computer vision tasks. Such networks employ self-supervised learning schemes such as contrastive learning (MoCov3 [32] , DINO [29] ), masked image modeling (MAE [33] , SimMIM [34] , BEiT [35] ), and their combination (iBOT [36] , DINOv2 [30] ), producing universal features suitable for image-level visual tasks (image classification, instance retrieval) and pixel-level visual tasks (depth estimation, semantic segmentation).Because of the lack of supervision in UAD, most advanced methods adopt pre-trained networks to extract discriminative features. Recent works [37; 17; 38] have discovered the advantage of robust and universal features of self-supervised models over domain-specific ImageNet features in anomaly detection tasks. In this work, we further utilize the up-to-date Transformer foundation, i.e., DINOv2 with registers [39] , as the encoder of Dinomaly."Dropout is all you need."Generalization ability is a merit of neural networks, allowing them to perform equally well on unseen test sets. However, generalization is not so wanted in the context of unsupervised anomaly detection that leverages the epistemic nature of neural networks. With the increasing diversity of images and their patterns due to multi-class UAD settings, the decoder can generalize its reconstruction ability to unseen anomalous samples, resulting in the failure of anomaly detection using reconstruction error. This phenomenon is called "identity mapping" in previous works of literature [3; 23; 18] . A direct solution for identity mapping is to shift "reconstruction" to "restoration". Specifically, instead of directly reconstructing the normal images or features given normal inputs, previous works propose to add perturbations as pseudo anomalies on input images [10; 40; 12] or feature representations [3; 25] during network forward propagation; meanwhile, still let the decoder restore anomaly-free images or features, formulating a denoising-like framework. However, such methods employ heuristic and hand-crafted anomaly generation strategies, that are not universal across domains, datasets, and methods.In this work, we propose to activate the pre-existing Dropout in an MLP layer. Dropout, a popular network element introduced by Hinton et al. [41] in 2014 to prevent overfitting, flourished in nearly all neural network architectures to the present day, including Transformers. In Dinomaly, Dropout is used to discard neural activations in the MLP bottleneck randomly. Instead of alleviating overfitting, the role of Dropout in Dinomaly can be explained as feature noise and pseudo feature anomaly.Although the decoder takes noisy features during training, it is encouraged to restore clean features from the encoder. Without introducing any novel modules, this paradigm forces the decoder to restore normal features given a test image with anomalies, in turn, mitigating identical mapping."One man's poison is another man's meat" Softmax Attention is the key mechanism of Transformers, allowing the model to attend to different parts of its input token sequence. Formally, given an input sequence X ∈ R N ×d with length N , Attention first transforms it into three matrices: the query matrix Q ∈ R N ×d , the key matrix K ∈ R N ×d , and the value matrix V ∈ R N ×d :EQUATIONwhere W Q , W K , W V ∈ R d×d are learnable parameters. By computing the attention map by the query-key similarity, the output of Attention is given as:EQUATION)1 The full form of Attention is Softmax( QK T √ d )V. The constant denominator is omitted for narrative simplicity. The multi-head mechanism that concatenates multiple Attentions is also omitted. Because the attention map is obtained by computing the similarity between all query-key pairs followed by row-wise Softmax, the computation complexity is O(N 2 d).Linear Attention was proposed as a promising alternative to reduce the computation complexity of vanilla Softmax Attention concerning the number of tokens [42] . By substituting Softmax operation with a simple activation function ϕ(•) (usually ϕ(x) = elu(x) + 1), we can change the computation order from (QK T )V to Q(K T V). Formally, Linear Attention is given as:EQUATIONwhere the computation complexity is reduced to O(N d 2 ). The trade-off between complexity and expressiveness is a dilemma. Previous studies [43; 44] attribute Linear Attention's performance degradation on supervised tasks to its incompetence in focusing. Due to the absence of non-linear attention reweighting by Softmax operation, Linear Attention cannot concentrate on important regions related to the query, such as foreground and neighbors.Back to MUAD, previous methods [3; 24] suggest adopting Attentions instead of Convolutions because Convolutions can easily learn identical mappings. Nevertheless, both operations are in danger of forming identity mapping by over-concentrating on corresponding input locations for producing the outputs:Conv Kernel = 0 0 0 0 1 0 0 0 0 , Attention Map =    1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1    .In Dinomaly, we turn to leverage the "unfocusing ability" of Linear Attention. In order to probe how Attentions propagate information, we train two variants of Dinomaly using vanilla Softmax Attention or Linear Attention as the spatial mixer in the decoder and visualize their attention maps.As shown in Figure 3 , Softmax Attention tends to focus on the exact region of the query, while Linear Attention spreads its attention across the whole image. This implies that Linear Attention, forced by its incompetence to focus, utilizes more long-range information to restore features at each position, reducing the chance of passing identical information of unseen patterns to the next layer However, according to the intuition of knowledge distillation, the student (decoder) can better mimic the behavior of the teacher (encoder) given more layer-to-layer supervision, which is harmful for UAD models that detect anomalies by encoder-decoder discrepancy. This phenomenon is also embodied as identity mapping. Thanks to the top-to-bottom consistency of columnar Transformer layers, we propose to loosen the layer-to-layer constraint by adding up all feature maps of interested layers as a whole group, as shown in Figure 4(d) . This scheme can be seen as loosening the layer-to-layer correspondence, so that the decoder is allowed to act much more differently from the encoder when the input pattern is unseen. Because features of shallow layers contain low-level visual characters that are helpful for precise localization, we can further group the features into the low-semantic-level group and high-semantic-level group, as shown in Figure 4 (e).Loose Loss. Following the above analysis, we also loosen the point-by-point reconstruction loss function by discarding some points in the feature map. Here, we simply borrow the hard-mining global cosine loss [18] that detaches the gradients of well-restored feature points with low cosine distance during training. Let f E and f D denotes (grouped) feature maps of encoder and decoder:EQUATIONEQUATIONwhere F( and then center-cropped to 3922 , so the feature map ( 282 ) is large enough for anomaly localization.StableAdamW optimizer [46] with AMSGrad [47] (more stable than AdamW [48] in training) is utilized with lr=2e-3, β=(0.9,0.999) and wd=1e-4. The network is trained for 10,000 iterations (steps) on MVTec-AD and VisA, and 50,000 iterations on Real-IAD. More details are available in Appendix A.2. 
comparison to multi-class uad sotas: 
 We compare the proposed Dinomaly with the most advanced methods. Among them, RD4AD [2] based on feature reconstruction, SimpleNet [13] based on feature-level pseudo-anomaly, and DeST-Seg [12] We also compare our Dinomaly with class-separated SoTAs, as shown in Table 2 . On MVTec-AD and VisA, our Dinomaly under MUAD setting is comparable to conventional SoTAs that build individual models for each class [2; 13; 8; 15]. In addition, Dinomaly is subjected to nearly no performance drop compared to its class-separated counterpart on these datasets. On the complicated Real-IAD that involves more images, classes, and views, class-separated Dinomaly sets new SoTA records.Multi-class Dinomaly presents moderate performance drop but is still comparable to class-separated SoTAs.Overall Ablation. We conduct experiments to verify the effectiveness of the proposed elements, i.e., Noisy Bottleneck (NB), Linear Attention (LA), Loose Constraint (LC), and Loose Loss (LL). The already-powerful baseline is Dinomaly with noiseless MLP bottleneck, Softmax Attention, dense layer-to-layer supervision, and global cosine loss [18] . Results on MVTec-AD and VisA are shown in Table 3 and Table A1 , respectively. NB and LL can directly contribute to the model performance. LA and LC boost the performance with the presence of NB. The use of LC is not solely beneficial because or generative adversarial network (GAN) [54; 55] are used to restore normal pixels. However, pixel reconstruction models may also succeed in restoring unseen anomalous regions if they resemble normal regions in pixel values or the anomalies are barely noticeable [2] . Therefore, feature reconstruction is proposed to construct features of pre-trained encoders instead of raw pixels [2; 3; 4]. To prevent the whole network from converging to a trivial solution, the parameters of the encoders are frozen during training. In feature distillation [5; 6] , the student network is trained from scratch to mimic the output features of the pre-trained teacher network with the same input of normal images, also based on the similar hypothesis that the student trained on normal samples only succeed in mimicking features of normal regions.Pseudo-anomaly methods generate handcrafted defects on normal images to imitate anomalies, converting UAD to supervised classification [11] or segmentation tasks [10] . Specifically, CutPaste [11] simulates anomalous regions by randomly pasting cropped patches of normal images. DRAEM [10] constructs abnormal regions using Perlin noise as the mask and another image as the additive anomaly. DeTSeg [12] employs a similar anomaly generation strategy and combines it with feature reconstruction. SimpleNet [13] introduces anomaly by injecting Gaussian noise in the pre-trained feature space. These methods deeply rely on how well the pseudo anomalies match the real anomalies, which makes it hard to generalize to different datasets. Multi-Class UAD. UniAD [3] first introduced multi-class anomaly detection, aiming to detect anomalies for different classes using a unified model. In this setting, conventional UAD methods often face the challenge of "identical shortcuts", where both anomaly-free and anomaly samples can be effectively recovered during inference [3] . It is caused by the diversity of multi-class normal patterns that drive the network to generalize on unseen patterns. This contradicts the fundamental assumption of epistemic methods. Many current researches focus on addressing this challenge [3; 24; 18; 57; 25] .UniAD [3] employs a neighbor-masked attention module and a feature-jitter strategy to mitigate these shortcuts. HVQ-Trans [24] proposes a vector quantization (VQ) Transformer model that induces large feature discrepancies for anomalies. LafitE [25] utilizes a latent diffusion model and introduces a feature editing strategy to alleviate this issue. DiAD [26] also employs diffusion models to address multi-class UAD settings. OmniAL [23] focuses on anomaly localization in the unified setting, preventing identical reconstruction by using synthesized pseudo anomalies. ViTAD [58] abstracts a unified feature-reconstruction UAD framework and employ Transformer building blocks. MambaAD [19] explores the recently proposed State Space Model (SSM), Mamba, in the context of multi-class UAD.In this work, we focus on sensory AD that detects regional or structural anomalies (common in practical applications such as industrial inspection, medical disease screening, etc.), which is distinguished from semantic AD. In sensory AD, normal and anomalous samples are the same objects except for anomaly, e.g. good cable vs. spoiled cable. In semantic AD, the class of normal samples and anomalous samples are semantically different, e.g. animals vs. vehicles.Semantic AD methods usually utilize and compare the global representation of images, which generally do not suffer from the issues of multi-class setting discussed in this paper..ViT-Base/14 (patch size=14) pre-trained by DINOv2 with registers (DINOv2-R) [39] is utilized as the encoder. The discard rate of Dropout in Noisy Bottleneck is 0.2 by default, which is increased to 0.4 for the diverse Real-IAD. Loose constraint with 2 groups and L global-hm loss are used by default.The input image is first resized to 448 2 and then center-cropped to 392 2 , so that the feature map (28 2 ) is large enough for localization. StableAdamW optimizer [46] with AMSGrad [47] is utilized with lr (learning rate)=2e-3, β=(0.9,0.999), wd (weight decay)=1e-4 and eps=1e-10. The network is trained for 10,000 iterations for MVTec-AD and VisA and 50,000 iterations for Real-IAD under MUAD setting. The network is trained for 5,000 iterations on each class under the class-separated UAD setting. The lr warms up from 0 to 2e-3 in the first 100 iterations and cosine anneals to 2e-4 throughout the training. The discarding rate in Equation 5 linearly rises from 0% to 90% in the first 1,000 iterations as warm-up (500 iters for class-separated setting). The anomaly map is obtained by upsampling the point-wise cosine distance between encoder and decoder feature maps (averaging if more than one pair or group). The mean of the top 1% pixels in an anomaly map is used as the image anomaly score. All experiments are conducted with random seed=1 with cuda deterministic for invariable weight initialization and batch order. Codes are implemented with Python 3.8 and PyTorch 1.12.0 cuda 11.3, and run on NVIDIA GeForce RTX3090 GPUs (24GB).Ablations on VisA. Similar to Scalability. Previous works [3; 2; 17] reported that AD methods do not follow the model "scaling law", i.e., larger models do not necessarily produce better performance. For example, RD4AD [2] found WideResNet50 better than WideResNet101 as the encoder backbone. ViTAD [17] Pre-Trained Foundations. The representation quality of the frozen backbone Transformer is of great significance to unsupervised anomaly detection. We conduct extensive experiments to probe the impact of different pre-training methods, including supervised learning and self-supervised learning. DeiT [59] is trained on ImageNet [1] in a supervised manner by distilling CNNs. MAE [33] ,BEiTv2 [35] , and D-iGPT [60] are based on masked image modeling (MIM). Given input images with masked patches, MAE [33] is optimized to restore raw pixels; BEiTv2 [35] is trained to predict the token index of VQ-GAN and CLIP; D-iGPT [60] is trained to predict the features of CLIP model. DINO [29] is based on positive-pair contrastive learning (CL), which is also referred to as self-distillation. It trains the network to produce similar feature representations given two views (augmentations) of the same image. iBot [36] and DINOv2 [30] combine MIM and CL strategies, marking the SoTA of self-supervised foundation models. DINOv2-R [39] is a variation of DINOv2 that employs 4 extra register tokens. Vision Transformers are known for their high computation cost, which can be a barrier to lowcomputation scenarios that require inference speed. Future research can be conducted on the efficiency of Transformer-based methods, such as distillation, pruning, and hardware-friendly attention mechanism (such as FlashAttention). As discussed in section A.1, Dinomaly is used for sensory AD that aims to detect regional anomalies in normal backgrounds. It is not suitable for semantic AD. Previous works have shown that methods designed for sensory AD usually fail to be competitive under semantic AD tasks [3; 2] . Conversely, methods designed for semantic AD do not perform well on sensory AD tasks [62; 37] . Future work can be conducted to unify these two tasks, but according to the "no free lunch" theorem, we believe that methods designed for specific anomaly assumption are likely to be more convincing.Other special UAD settings, such as zero-shot UAD (vision-language model based) [63] , few-shot UAD [64] , UAD under noisy training set [65] , are not included in this work.For future research, we report the per-class results of MVTec-AD [20] , VisA [21] , and Real-IAD [22] .The performance of compared methods is drawn from MambaAD [19] . Thanks for their exhaustive reproducing. The results of image-level anomaly detection and pixel-level anomaly localization on MVTec-AD are presented in Table A8 and Table A9 , respectively. The results of image-level anomaly detection and pixel-level anomaly localization on VisA are presented in Table A10 and Table A11 ,Real-IAD are presented in Table A12 and Table A13 , respectively.We visualize the output anomaly maps of Dinomaly on MVTec-AD, VisA, and Real-IAD, as shown in Figure A1 , Figure A2 , and Figure A3 . It is noted that all visualized samples are randomly chosen without artificial selection. • If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.• If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.• Depending on the contribution, reproducibility can be accomplished in various ways.For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.• While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example (d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility.In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results. Justification: Yes, and code is in supplementary material.Guidelines:• The answer NA means that paper does not include experiments requiring code.• Please see the NeurIPS code and data submission guidelines (https://nips.cc/ public/guides/CodeSubmissionPolicy) for more details.• While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).• The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: //nips.cc/public/guides/CodeSubmissionPolicy) for more details.• The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.• The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.• At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).• Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 
experimental setting/details: 
 Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes] Justification: Yes.Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.• The full details can be provided either with the code, in appendix, or as supplemental material. 
experiment statistical significance: 
 Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [Yes]Justification: Results with mean and std are presented in Appendix.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.• The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).• The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)• The assumptions made should be given (e.g., Normally distributed errors).• It should be clear whether the error bar is the standard deviation or the standard error of the mean.• It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.• For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).• If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text. 
experiments compute resources: 
 Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?Answer: [Yes] Justification: Yes.Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.Complete stop-gradient causes optimization instability occasionally. 
 
 Appendix:  


Decision: This paper presents a minimalistic reconstruction-based anomaly detection framework for multi-class unsupervised anomaly detection. The framework comprises four key components: foundation transformers, noisy bottleneck, linear attention, and loose reconstruction. The proposed method has been extensively tested on MVTec-AD, VisA, and Real-IAD datasets, demonstrating state-of-the-art performance in multi-class unsupervised anomaly detection scenarios.

The paper receives mixed scores, including three positive and two borderline. All reviewers concur that the paper introduces a simple and effective method for multi-class anomaly detection, and achieves state-of-the-art performance (i.e., anomaly detection accuracy) on multiple datasets. The paper is well-written, well-structured, and easy to understand. However, there are contentious points primarily concerning the novelty of the proposed method, the fairness of the comparison, and the omission of a discussion on the number of parameters, computational complexity, or time complexity.

**Limited Novelty**. The proposed method uses the existing RD4AD framework with several new modules including foundation transformers, noisy bottleneck, linear attention, group loss, and loose loss. Notably, the noisy bottleneck is an MLP with dropout. And it is, in fact, very similar to some existing reconstruction works that randomly mask input pixels or features. The difference lies in the proposed method performing random dropout in the channel dimension while existing methods perform random mask in the spatial dimension. Furthermore, linear attention is an existing work, while group loss and loose loss are simple improvements to the existing reconstruction loss.  

**Unfair Comparisions**. Compared to existing methods (e.g., RD4AD and UniAD), this paper employs a more powerful pre-trained model (DINO-v2 vs WideResNet50 or EfficientNet-b4) and a larger input resolution (448$\times$448 vs. 224$\times$224 or 256$\times$256). Consequently, it is currently hard to directly evaluate the actual performance improvement due to the unfair comparison. I suggest that the authors first establish strong baselines using the same backbone and input resolution, then compare the proposed method and the state-of-the-art. Additionally, using a larger backbone and higher input resolution implies more hidden computational costs, such as the number of parameters, memory, and inference time. However, the detailed comparisons are absent.

Although the author has partially addressed the above contentious points in the rebuttal period, I believe it will be challenging to fully resolve these issues in the camera-ready version. Therefore, I lean towards rejecting it and encourage the author to fully consider the reviewer's suggestions and revise the paper for submission to other venues.

Reviews: review: paper_summary: paper_summary: Dinomaly simplifies the anomaly detection process by eliminating the need for complex designs, additional modules, or specialized techniques. It relies solely on basic Transformer components such as self attention mechanisms and multi-layer perceptrons (MLPs) to perform anomaly detection for multi class images.
strength_weakness: strength_weakness: I hope to provide a specific explanation of the information provided by the decision-making process for identifying anomalies in the model.
questions: questions: Is the viewpoint presented in this article too sharp? Large scale models may not necessarily lack their advantages. I acknowledge this work and would like to see other people's questions and the author's response to determine the final score.
limitations: limitations: The author candidly acknowledged the limitations of the work and provided the problems that need to be addressed.
paper_summary: paper_summary: This paper introduces Dinomaly, a minimalistic unsupervised anomaly detection (UAD) method designed to bridge the performance gap between multi-class UAD and class-separated UAD. Utilizing pure Transformer architectures with key components such as Foundation Transformers, Noisy Bottleneck, Linear Attention, and Loose Reconstruction, Dinomaly achieves superior performance on MVTec-AD, VisA, and Real-IAD benchmarks, surpassing state-of-the-art methods.
strength_weakness: strength_weakness: 1. The method might be perceived as too application-oriented, lacking broader theoretical contributions.
questions: questions: See the weaknesses.
limitations: limitations: See the weaknesses.
paper_summary: paper_summary: This paper introduces Dinomaly, a simple yet effective anomaly detection framework using pure Transformer architectures. It identifies four key components essential for multi-class anomaly detection: Foundation Transformers, Noisy Bottleneck, Linear Attention, and Loose Reconstruction. Extensive experiments on MVTec-AD, VisA, and Real-IAD datasets show that Dinomaly achieves superior performance, surpassing both state-of-the-art multi-class and class-separated anomaly detection methods.
strength_weakness: strength_weakness: 1. L53-55 'In addition, previous...': The authors should provide relevant evidence rather than subjective assumptions.
2. L77: Placing the Related Works section in the appendix is unconventional.
3. The first component proposed by the authors, Foundation Transformers, was already introduced in the ViTAD paper, which diminishes the overall contribution of the paper. 
4. The input resolution used in the authors' experiments is 448x448, while other comparison methods use 256x256 or 224x224. This is an extremely unfair comparison. Please include results with a 256 resolution in table for a fair comparison.
5. In the proposed Loose Loss, 90% of the feature points were selected. How was this 90% hyperparameter determined? Please provide ablation study results.
6. The authors have employed Linear Attention to reduce computational load while maintaining similar performance. It is recommended that the authors compare the parameter count and FLOPs of their method with those of the baseline methods to demonstrate its efficiency. Additionally, it is suggested to conduct ablation studies to verify the computational efficiency of Linear Attention. 
7. Other methods, such as RD4AD, SimpleNet, and UniAD, perform under the proposed settings. The authors can conduct a more equitable comparison.
questions: questions: See the Weaknesses.

=== After Rebuttal ===
I decide to raise my score from 4 to 5.
limitations: limitations: It is recommended that the authors evaluate the performance of different methods under fair settings.
paper_summary: paper_summary: This paper focuses on the Multi-class Unsupervised Anomaly Detection task and proposes a minimalistic reconstruction-based anomaly detection framework — Dinomaly that consists of only vanilla Transformer blocks. In this framework, four key components (Foundation Transformers, Noisy Bottleneck, Linear Attention, and Loose Reconstruction) are introduced to alleviate the performance gap between multi-class and class-separated models. The paper conducts extensive experiments on three major datasets: MVTec-AD, VisA, and Real-IAD. Results show that Dinomaly outperforms current state-of-the-art methods.
strength_weakness: strength_weakness: (1)The method relies heavily on transformer architectures, which might limit its applicability to other types of models.

(2)Transformers can be resource-intensive, and the paper does not fully address the computational cost of training and inference.

(3)The method's generalization to other domains or types of anomaly detection is not fully explored.
questions: questions: (1) Please specify the computational overhead of your work.

(2) There have been various recent works that attempt to perform anomaly detection in a more general zero-shot/few-shot setting, where the model trained on multiple classes is used to test samples from unseen classes. A discussion of these recent works and how the zero-shot (cross-dataset) performance of this approach should be added. e.g.,

"WinCLIP: Zero-/few-shot anomaly classification and segmentation" CVPR'2023

"AnomalyCLIP: Object-agnostic Prompt Learning for Zero-shot Anomaly Detection" ICLR'2024

"Toward Generalist Anomaly Detection via In-context Residual Learning with Few-shot Sample Prompts" CVPR'2024

"PromptAD: Learning Prompts with only Normal Samples for Few-Shot Anomaly Detection" CVPR'2024

(3) If I understand correctly, the best and second-best results in each table should be highlighted, with the best results in bold and the second-best results underlined. However, in Table 2, the best results for MVTec-AD (P-AP) and VisA (P-AP) are incorrectly marked. Additionally, could you clarify why the Dinomaly (MUAD) model outperforms the Dinomaly (class-separated) model in this metric?
limitations: limitations: Limitations are discussed in Supplementary Sec. A.4.
paper_summary: paper_summary: The paper "Dinomaly: The Less Is More Philosophy in Multi-Class Unsupervised Anomaly Detection" introduces a minimalist reconstruction-based framework for unsupervised anomaly detection (UAD) in multi-class settings. The framework focuses on four main components: Foundation Transformers, Noisy Bottleneck, Linear Attention, and Loose Reconstruction. Extensive experiments on MVTec-AD, VisA, and Real-IAD datasets show that Dinomaly achieves superior performance compared to state-of-the-art multi-class and even some class-separated UAD methods.
strength_weakness: strength_weakness: 1. The paper provides a detailed explanation of the proposed framework but lacks important justification and discussion, it is difficult for readers to realize the novelty and improvements brought by Dinomaly. The author may need to compare Dinomaly to specific previous methods, highlighting the differences and improvements. Discuss how the minimalist approach contrasts with more complex architectures and why this improvement is significant.
2. The motivations for choosing Noisy Bottleneck and Loose Reconstruction are not deeply explored. For instance, explain in more detail why Noisy Bottleneck helps prevent identity mapping.
3. The paper claims simplicity but there was no discussion of parameter number, computational complexity, or time complexity in the experiment. 
4. In Loose Constraint, the author claims that 1-group LC mixes low-level and high-level features which is harmful for anomaly localization. How to group the features into the low-semantic-level group and high-semantic-level group in 2-group LC?
questions: questions: 1. Can you provide more detailed justifications for the choice of Noisy Bottleneck and Loose Reconstruction? Why do these help in preventing identity mapping and improving anomaly detection performance?
Suggestion: A detailed theoretical explanation or additional references would clarify the rationale of the design of Dinomaly and strengthen the argument for its effectiveness.
2. Provide a more explicit comparison to recent methods in terms of performance and conceptual differences. Highlight any specific limitations of prior work that Dinomaly addresses.
3. Increase discussion on simplicity and scalability.
4. The credibility of the paper may benefit from deeper experimental analysis. Known that Dinomaly surpasses compared methods by a large margin on all datasets and all metrics, if its performance is limited under certain conditions, such as when the input is video?
limitations: limitations: N/A

Comments: title: Response
For Authors
Response
Response (2/2)
Response (1/2)
For Authors
Revision of the Last Table of Rebuttal
comment: Sincere thanks for your reply. We will do so. Again, thank you for your thorough review and valuable discussion.
Thanks for the authors' response, and it basically addressed my concerns. I will maintain my score.
Thank you for the detailed response. I suggest that the author add the fairness resolution experiments to the revised version and include results from the VisA and Real-IAD datasets. I have also reproduced the Dinomaly-ViT-Small experimental results using the official code. The author could consider uploading the logs of the main experiments (excluding ablation studies) to the website to facilitate replication and comparison by future researchers. 

Although I still maintain a somewhat negative view of the paper's technological contributions, the solid experimental results could positively impact the community. Therefore, I have decided to raise my score, but the author should adhere to their commitment to address the issues in the revised version.
Thank you for your valuable review and comments. 

As discussed in the 4th paragraph of the rebuttal of W1,2&Q1,2, we have attributed the harms of dense layer-wise distillation to its generalization ability.  Apart from the listed [d] and [e], there were plenty of works that utilized dense layer-wise distillation schemes for better generalization in the context of knowledge distillation. E.g., In [f]: "Given that the teacher’s layerwise representations often contain rich semantic knowledge, they can significantly improve the student’s generalizability". Because we have established that over-generalization is a curse in the context of UAD, such dense layer-wise distillation paradigm is self-evidently harmful to reconstruction/distillation-based UAD. In addition, as discussed in L165-L166, "the student (decoder) can better mimic the behavior of the teacher (encoder) given more layer-to-layer supervision", which is also clearly unwanted in UAD, because an "Identical" student cannot detect anomalies based on the reconstruction error. We believe such analysis can be a strong conceptual principle for the proposed Loose Constraint.

[f] Liang, Chen, et al. "Module-wise adaptive distillation for multimodality foundation models." Advances in Neural Information Processing Systems 36 (2023).

Due to the page limit, we have to put extensive ablation studies in the Appendix, while leaving the main paper for presenting contributions.  We will present at least two more ablation experiments in the final version, while others can be included in Appendix with proper references.

Again, thank you for your thorough review. Looking forward to further discussion.
I agree with the author's answer to “Why choose Noisy Bottleneck and differentiation”, but still do not explain why dense layer-wise distillation is harmful to UAD. 
Thanks for the valuable comments by reviewer #LXBB. I agree with the statement that “it is challenging to include the suggested experiments in the main text without allowing major changes in subsequent submissions”. There are indeed some writing shortcomings in innovative ideas, motivations, clarity of principles, and related work. But in terms of technical contribution, I acknowledge the good performance and comprehensive experimental design of this work, as the author replied, “We provide a variety of selections for downstream users to choose from according to their budget”, so I will keep my decision.
Response (2/2)

__3)__ All metrics are shown as follows. For RD4AD, P-AP and P-F1 increase with resolution at 320, but still do no further benefits from 384; P-AUROC and P-AURPO do not increase with resolution. For ViTAD, P-AP and P-F1 increase with resolution;  P-AUROC and P-AURPO do not increase with resolution. For ReContrast, pixel-level metrics generally do not benefit from increasing resolution. In addition, it is worth considering whether such pixel-level improvement is worth a large decrease in image-level detection performance, as image-level detection is usually more relevant in real applications.

| Methods | Input size | I-AUROC | I-AP | I-F1 | P-AUROC | P-AP | P-F1 | P-AUPRO |
|---|---|---|---|---|---|---|---|---|
| RD4AD | 256 | 94.6 | 96.5 | 95.2 | 96.1 | 48.6 | 53.8 | 91.1 |
| RD4AD | 320 | 93.2 | 96.9 | 95.6 | 95.7 | 55.1 | 57.5 | 91.1 |
| RD4AD | 384 | 91.9 | 96.2 | 95.0 | 94.9 | 52.1 | 55.3 | 90.8 |
| ViTAD | 256 | 98.3 | 99.4 | 97.3 | 97.7 | 55.3 | 58.7 | 91.4 |
| ViTAD | 320 | 98.3 | 99.2 | 97.1 | 97.6 | 61.3 | 63.3 | 92.4 |
| ViTAD | 384 | 97.8 | 98.9 | 96.3 | 97.5 | 62.5 | 63.7 | 92.4 |
| ReContrast | 256 | 98.3 | 99.4 | 97.6 | 97.1 | 60.2 | 61.5 | 93.2 |
| ReContrast | 320 | 98.2 | 99.2 | 97.5 | 96.8 | 61.8 | 62.6 | 93.3 |
| ReContrast | 384 | 95.2 | 98.0 | 96.4 | 96.5 | 57.7 | 59.5 | 92.6 |


__4)__ MAE was also tested as the backbone of ViTAD in their paper, resulting in worse performances (I-AUROC, MAE=95.3 vs. DINO=98.3, Table 1 in ViTAD), which was attributed to the weak semantic expression caused by its pretraining strategy (also note MAE is bad in other unsupervised tasks such as ImageNet kNN). Comparatively, Dinomaly with MAE achieves much better results (I-AUROC=97.3) than ViTAD with MAE.

According to your suggestion, we reproduce ViTAD (original DINO) with the stronger DINOv2 and DINOv2-Register as the backbone. Arming ViTAD with stronger backbones results in slightly better performance, but still lags behind our Dinomaly. It is expected because ViTAD with DINOv2-R is very similar to Dinomaly baseline (Table 3 first row). 

| Methods | Backbone | I-AUROC | I-AP | I-F1 | P-AUROC | P-AP | P-F1 | P-AUPRO |
|---|---|---|---|---|---|---|---|---|
| ViTAD | MAE | 95.3 | 97.7 | 95.2 | 97.4 | 53.0 | 56.2 | 90.6 |
| ViTAD | DINO (original) | 98.3 | 99.4 | 97.3 | 97.7 | 55.3 | 58.7 | 91.4 |
| ViTAD | DINOv2 | 98.7 | 99.4 | 98.1 | 97.6 | 55.3 | 59.1 | 92.7 |
| ViTAD | DINOv2-R | 98.5 | 99.3 | 97.8 | 97.4 | 54.5 | 59.2 | 92.8 |


__5)__ Sorry for the confusion. We did not mean that we do not care about computational cost and building models at all costs. We mean that we did not design modules or techniques aiming to reduce computational cost. 

We believe scaling ability is important in this decade, so we armed Dinomaly from ViT-Small to ViT-Large. On the contrary, previous methods did not show this obeying of scaling law (e.g., ViTAD favors ViT-Small over ViT-Base. RD4AD favors ResNet50 over ResNet101). We provide a variety of selections for downstream users to choose from according to their budget.

In Table A2, we have shown that Dinomaly can be scaled down to ViT-Small with the computation cost (26.3G) given 396x396 input, which is already lower than RD4AD given 256x256 input (28.4G). This computation cost is very promising for real industrial applications, yielding latency of 6.8ms per image (~147 FPS) on home-use-level NVIDIA 3090. 

Moreover, we further scale down Dinomaly-ViT-Small by decreasing input size, as shown below, where Dinomaly produces SoTA results given more limited computation budgets. 


| Methods | Parameters | MACs | I-AUROC | I-AP | I-F1 | P-AUROC | P-AP | P-F1 | P-AUPRO |
|---|---|---|---|---|---|---|---|---|---|
| RD4AD | 80.6M | 28.4G | 94.6 | 96.5 | 95.2 | 96.1 | 48.6 | 53.8 | 91.1 |
| ViTAD | 39.0M | 9.7G | 98.3 | 99.4 | 97.3 | 97.7 | 55.3 | 58.7 | 91.4 |
| ReContrast | 154.2M | 67.4G | 98.3 | 99.4 | 97.6 | 97.1 | 60.2 | 61.5 | 93.2 |
| Dinomaly-ViT-Small-396x396 | 37.4M | 26.2G | 99.3 | 99.7 | 98.7 | 98.1 | 68.3 | 67.8 | 94.4 |
| Dinomaly-ViT-Small-280x280 | 37.4M | 14.5G | 99.3 | 99.7 | 98.7 | 98.0 | 65.1 | 65.7 | 93.4 |
| Dinomaly-ViT-Small-252x252 | 37.4M | 11.6G | 99.1 | 99.6 | 98.7 | 98.0 | 63.9 | 64.9 | 93.1 |


We believe scaling ability is important in this decade. By setting the current upper-bounds on various benchmarks, we aim to (and can) provide a scalable framework that can accommodate different demands of application needs. If someone's main concern is detection performance and has abundant computation resources, they are given the option to choose Dinomaly-ViT-Large/Base for ultimate performance. If someone has limited computation resources or cares about FPS/latency, they can choose Dinomaly-ViT-Small while still producing state-of-the-art performance. 

Again, thank you for your thorough comments and review. Looking forward to further discussion with you.
Response(1/2)

Thank you for your valuable comments. We by no means intended to evade any concerns.  We spare no effort to clarify any raised concerns.

__R-Q1:__

NeurIPS allows one more page for the final version (10 pages), which will fit a condensed Related Work. 

In addition, though there is no explicit Related Work section in our submission, we merged such content directly in Introduction (L28-L55). We discussed categories of conventional UAD methods, why conventional methods fail in MUAD, and recent methods of MUAD, which elaborates the background and history of MUAD. Such narrative will also be partly moved to the final Related Work. 

Moreover, we have checked the NeurIPS formatting guidelines carefully and found no such guideline that strictly demands an explicit Related Work section. Many NeurIPS papers do not have an explicit Related Work section, some not even in the final version.

To name a few, see:

https://openreview.net/pdf?id=DP2lioYIYl

https://openreview.net/pdf?id=aExAsh1UHZo

https://openreview.net/forum?id=hgLMht2Z3L

https://openreview.net/forum?id=wxkBdtDbmH&noteId=JKLz5pP5sJ

https://openreview.net/attachment?id=4VAF3d5jNg&name

__R-Q3:__

In L99-L101, we have acknowledged that ViTs have been used for UAD in recent works. As discussed in Rebuttal, we consider foundation ViTs as an important and bedstone component of our Dinomaly from a narrative perspective to emphasize the importance of a well-pretrained ViT backbone in UAD context. 

According to the "less is more" essence of this paper, we did not intend to propose "novel" or "never seen" technologies (L124-L125), but to propose simple pre-exists elements that have been long ignored for achieving SoTA performance for MUAD.

Most proposed elements (especially noisy MLP, Linear Attention, Loose Constraint) are closely bounded to modern ViTs. Applying such elements to the only previous ViT-based method (ViTAD) would just convert it to Dinomaly. Such elements are extensively evaluated on various ViT variants in Table A4 to show their generalizability. Loose Loss can be directly applied to previous CNN-based methods. Noisy Bottleneck can be adapted to RD4AD with minor modifications (apply dropout before MFF layer). Following your new suggestions, we apply these modules to a different framework to validate the effectiveness of our contributions. The results are shown as follows, where these two elements boost RD4AD to a whole new level.

| Methods | I-AUROC | I-AP | I-F1 | P-AUROC | P-AP | P-F1 | P-AUPRO |
|---|---|---|---|---|---|---|---|
| RD4AD | 94.6 | 96.5 | 95.2 | 96.1 | 48.6 | 53.8 | 91.1 |
| RD4AD+Loose Loss | 98.4 | 99.4 | 97.9 | 97.2 | 58.6 | 60.4 | 92.9 |
| RD4AD+Noisy Bottleneck | 98.2 | 99.2 | 97.5 | 96.8 | 60.0 | 61.1 | 92.7 |
| RD4AD+Both | 98.5 | 99.4 | 97.8 | 97.2 | 59.6 | 61.2 | 93.0 |

__R-Q4/6:__

__1)__ As previously discussed, Dinomaly with the resolution of 224x224 has already been presented in Table A4 (last row), in which Dinomaly also achieved SoTA results (I-AUROC=99.3). 256 is not feasible for ViT-Base/14 as 256 is not divisible by 14.  As mentioned in the previous rebuttal, we followed the common comparison strategy based on "optimum vs. optimum" in the manuscript. We will include both settings in the main Table 1.

| Methods | Input Size| I-AUROC | I-AP | I-F1 | P-AUROC | P-AP | P-F1 | P-AUPRO |
|---|---|---|---|---|---|---|---|---|
| RD4AD | 256 | 94.6 | 96.5 | 95.2 | 96.1 | 48.6 | 53.8 | 91.1 |
| ViTAD | 256 | 98.3 | 99.4 | 97.3 | 97.7 | 55.3 | 58.7 | 91.4 |
| ReContrast | 256 | 98.3 | 99.4 | 97.6 | 97.1 | 60.2 | 61.5 | 93.2 |
| Dinomaly | 224 | 99.3 | 99.7 | 99.0 | 98.1 | 63.0 | 64.5 | 92.6 |

__2)__ Most prior UAD methods use ConvNets. ConvNets have much smaller downsample stride than ViTs. Therefore, such methods can utilize 64x64 feature maps under 256 input. Empirically, 64x64 feature maps are already saturated for UAD tasks. Further enlarging causes over-focusing on non-semantic low-level noises. (that is why pixel-reconstruction methods are no longer used) On the contrary, ViT/16 can only get 16x16 feature maps given the same input size, which is the reason why ViTAD does not suffer from performance drop when increasing input size. From another perspective, fairness is also questioned given the same image size as CNN-based methods and ViT-based methods have different feature map sizes.

__Questions 3), 4) and 5) are discussed in the next official comments.__
The authors have resolved a few issues, but most concerns remain unaddressed or have been evaded:

Q1. Placing related works in the appendix violates the formatting guidelines, potentially extending the main text to 10 pages instead of 9, which is unfair to other works. Considering that revised papers generally increase in length, I believe it will be challenging for the authors to include the Related Work section in the main text as required.

Q3. The authors list ViT as a contribution point in lines L57-59 and Section 2.1, particularly the use of DINOv2 weights, which offers no technical contribution to AD (as also noted by Reviewer **#m7bP**). However, I believe this is a core aspect of the work. The authors should apply other modules to different frameworks to validate the effectiveness of their contributions.

Q4/Q6. 1) A comparison of model efficiency at the standard resolution of 256 is necessary to ensure a fair comparison of different methods under the same parameter and computation constraints. Please provide the comparison results. 2) For perception tasks, pixel-level metrics should increase with resolution, but the results provided by the authors show the opposite trend. Please explain this. 3) AUROC is an unreliable metric; providing results with additional metrics would be beneficial. 4) Notably, the model performance under several weights in Table A4, such as MAE, significantly deteriorates, weakening the evidence for the effectiveness of other contributions. How does the performance compare when using stronger pre-trained weights for the comparison methods? 5) The reviewers do not accept the authors' claim of focusing solely on model performance without considering computational cost, which is unreasonable for model applications. This issue was also raised by reviewers **#hJXx** and **#m7bP**. The authors should provide a comparison of the proposed method scaled to a similar level as the comparison methods.
In the last table of our rebuttal, the computation cost (FLOPs) of compared methods (RD4AD, SimpleNet, ViTAD, DiAD) is drawn from the benchmark paper ADer [a]. However, we just found that they confused MACs(multiply–accumulate operations) with FLOPs(floating point operations). MACs~0.5*FLOPs. The FLOPs they reported are acutally MACs, which are around half of the real FLOPs.

Therefore, we report GMACs for unification.

[a] ADer: A Comprehensive Benchmark for Multi-class Visual Anomaly Detection. arXiv preprint arXiv:2406.03262 2024.

| Method | Parameters | MACs | I-AUROC | P-AUROC |
|---|---|---|---|---|
| RD4AD | 80.6M | 28.4G | 94.6 | 96.1 |
| SimpleNet | 72.8M | 17.2G | 95.3 | 96.9 |
| DRAEM | 97.4M | 198G | 54.5 | 47.6 |
| ViTAD | 39.0M | 9.7G | 98.3 | 97.7 |
| DiAD | 1331M | 451.5G | 97.2 | 96.8 |
| Dinomaly/ViT-Small-384x384 | 37.4M | 26.2G | 99.3 | 98.1 |
| Dinomaly/ViT-Base-280x280 | 148M | 53.7G | 99.5 | 98.4 |
| Dinomaly/ViT-Base-384x384 (default) | 148M | 104.6G | 99.6 | 98.4 |
| Dinomaly/ViT-Base-Softmax-384x384 | 148M | 112.3G | 99.5 | 98.2 |
| Dinomaly/ViT-Large-384x384 | 413.5M | 285.3G | 99.8 | 98.5 |