Dual-Modeling Decouple Distillation for Unsupervised Anomaly Detection

Authors: xinyue  liu
jianyuan  wang
biao  leng
shuo  zhang
Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-09 10:13:00

Abstract: Knowledge distillation based on student-teacher network is one of the mainstream solution paradigms for the challenging unsupervised Anomaly Detection task, utilizing the difference in representation capabilities of the teacher and student networks to  implement anomaly localization.  However, over-generalization of the student network to the teacher network may lead to negligible differences in representation capabilities of anomaly, thus affecting  the detection effectiveness.  Existing methods address the possible  over-generalization by using differentiated students and teachers  from the structural perspective or explicitly expanding distilled information from the content perspective, which inevitably result in an increased likelihood of underfitting of the student network and poor anomaly detection capabilities in anomaly center or edge.  In this paper, we propose Dual-Modeling Decouple Distillation (DMDD)  for the unsupervised anomaly detection. In DMDD, a Decouple Student-Teacher Network is proposed to decouple the initial student features into normality and abnormality features. We further introduce Dual-Modeling Distillation based on normal-anomaly image pairs, fitting normality features of anomalous image and the teacher features of the corresponding normal image, widening the distance between abnormality features and the teacher features in anomalous regions.  Synthesizing these two distillation ideas, we achieve anomaly detection which focuses on both edge and center of anomaly.  Finally, a Multi-perception Segmentation Network is  proposed to achieve focused anomaly map fusion based on multiple  attention. Experimental results on MVTec AD show that DMDD surpasses SOTA localization performance of previous knowledge distillation-based methods, reaching 98.85% on pixel-level AUC and 96.13% on PRO.

Main paper: 
 introduction: 
 Anomaly Detection (AD) is an important task in the field of computer vision, aiming to detect and locate anomalous regions in images, which has wide applications such as industrial quality inspection [3, 4] , medical disease screening [35, 43] , and video surveillance [19, 22, 29, 36] . Considering the scarcity of anomalous samples, AD is usually unsupervised, relying only on normal samples to train the model.Most existing unsupervised AD methods rely on the difference in the feature distribution or reconstruction ability of the network for normal pixels and anomalous pixels to locate anomalies in the image. Inspired by this idea, Knowledge Distillation (KD) based on Student-Teacher (S-T) Network gradually becomes one of the mainstream paradigms of unsupervised AD in recent years. In the training process, the student network learns the feature representation of normal images by the pre-trained teacher network, so that only the normality representation ability is obtained. During inference, the features extracted by the teacher network are used as comparison benchmarks. The similarity between the student features and the teacher benchmarks indicates anomaly localization results. That is, regions with larger feature similarity are more likely to be normal, while regions with small similarity are more likely to be anomalous. These KD-based methods assume that the student network cannot obtain the representation ability of the teacher network for anomalous pixels in the images during training. However, due to the generalization ability of neural networks, there is possibility that the student network may learn to generate abnormality representations in practice, thus reversing the above assumption. Existing KD-based methods work on the over-generalization problem from both structural and content perspectives, as shown in Fig. 1 .Structurally, Asymmetric Distillation [11, 26, 27] is proposed, whose main idea is to take advantage of asymmetry to differentiate the information capacity of the teacher and student networks and prevent over-generalization. However, the different capacities and network information transmission directions are likely to cause the student network to underfit the teacher network in terms of normality features [11, 15] . As for content, some methods improve the basic S-T network in respect to the distillation of information from normality and abnormality. Normality Distillation [13] focuses on "normality forgetting" issue, and indirectly enlarges the feature differences in anomalous regions by guiding the student's normality feature generation using the teacher features through memory banks and other means. Abnormality Distillation [46] introduces anomaly synthesis into KD paradigm, and explicitly distances the student's features from the teacher's ones in the anomalous regions. However, for anomaly centers, anomalous pixels in the receptive field are not conducive to normality feature generation guidance; for anomaly edges, interference with normal pixels in the receptive field is not favorable for abnormality feature differentiation. Consequently, improving distillation content only from normality or abnormality has limitation in capturing the full scope of anomalies.To tackle the above problems, in structure, our intuition is to construct a decoupled S-T network with the same capacity of teacher and student networks. In content, we propose to implement a dualbranch design by combining the ideas of Normality Distillation and Abnormality Distillation. Through fully aligned teacher and student networks, the adequacy of the representation ability of the student network is guaranteed. The design of dual-branch decoupling of the student network in turn ensures the difference between the student network and the teacher network. Moreover, with the feature decoupling idea, the ideas of normality and abnormality modeling are able to be introduced into the same distillation framework. In this way, anomaly detection is synthetically realized from two aspects of fine (edge localization) and coarseness (center localization), overcoming problems of previous content solutions.Following the above ideas, as in Fig. 1 , we propose a Dual-Modeling Decouple Distillation framework to implement dualbranch coarseness-fine distillation. Our method mainly includes three components: Decoupled Student-Teacher Network, Dual-Modeling Distillation and Multi-perception Segmentation Network. First, letting the same pre-trained network be the student and teacher networks, we propose a Decoupled Student-Teacher Network, which aims to decouple normality feature and abnormality feature from the student output for subsequent distillation. In addition, a Dual-Modeling Distillation is innovatively proposed, which relies on normal-anomalous image pairs and contains two distillation modules: Normality Guidance Modeling and Abnormality Inverse Mimicking to remodel normality features and abnormality features in different directions. Finally, after obtaining the anomaly maps calculated based on the student's decoupled features and the teacher's features, a Multi-perception Segmentation Network is put forward for achieving precise fusion of the anomaly maps in a differentiated manner. We conduct unsupervised AD experiments on multiple datasets, and the results show that our method achieves SOTA performance compared with existing KD-based methods. 
related works: 
 Unsupervised image Anomaly Detection has been rapidly developed due to the difficulty in obtaining anomalous images. The image reconstruction-based methods are widely accepted for Anomaly Detection using autoencoders [6, 30] , variational autoencoders [30, 48] , generative adversarial networks [9, 23, 28] and diffusion models [21, 33, 34, 41, 44] . Memory-based methods [1, 24, 37] are also commonly employed for unsupervised AD by designing memory bank to store normal features during training. In addition, memory bank is often utilized as a mean of guiding normality feature generation in other paradigms [13, 15] . Besides, some methods use parametric density estimation [10, 16] to determine anomalous outliers, and by introducing the idea of Normalizing Flow [14, 38, 47] , the detection effectiveness of this type of methods has reached a high level. Recently, some other methods [18, 40] propose anomaly synthesis to transform the unsupervised anomaly detection into a supervised problem, which greatly improves the detection performance.In addition, in recent years, knowledge distillation has gradually become a mainstream solution paradigm for unsupervised Anomaly Detection. The hypothesis is that a student network trained using teacher features on normal samples only obtains the teacher's representation of normality, but cannot simulate the teacher's abnormal representation. US [5] first introduces knowledge distillation into unsupervised AD. MKD [27] and STPM [32] use multi-scale features, where differentiated teacher and student network structures are also proposed to solve the over-generalization problem of the student network. Similarly, using the idea of differential teacher and student structures [2, 26, 45] , RD [11] and RD++ [31] design reverse distillation with an encoder and a decoder as teacher and student respectively. There are other methods [7, 42] addressing over-generalization in terms of distillation content by introducing information such as synthetic anomalies [46] , memory banks [13] , and so on, to ensure that the student network generates different features in the anomalous regions from the teacher network.  
overall framework: 
 Based on the student-teacher framework of KD, we propose Dual-Modeling Decouple Distillation (DMDD) for unsupervised AD. The left part of Fig. 2 shows the proposed Decoupled Student-Teacher Network (Sec 3.2), which is the basis of distillation. Take the first 4 stages of WideResNet50 [39] pre-trained on ImageNet [12] as the teacher and student network. The two teacher networks share the same weights. During the training process, the input of the whole network is a normal-anomalous image pair (𝐼 𝑛 , 𝐼 𝑛 𝑎 ) containing a normal image 𝐼 𝑛 and a synthetic anomalous image 𝐼 𝑛 𝑎 . The weights of the teacher network are fixed, and the student network is continuously optimized by Dual-Modeling Distillation (Sec 3.3). In addition, Due to the size limitation of the receptive field of Convolutional Neural Network (CNN), distillation methods based on normality construction (Normality Distillation) often have stronger perception of the edge pixels of anomalous regions, and distillation methods based on synthetic anomalies (Abnormality Distillation) often have stronger perception of the center pixels of anomalous regions. This is because the receptive field at the edge of the anomalous region receives more normality information, which enables better reconstruction of normality. On the contrary, the convolution kernel receives more abnormality information in the center of the anomaly region, which prevents interference from normal pixels, and thus being able to better expand the distance between the student features and the teacher features. Fig. 3 illustrates the above reasoning with a diagram. Existing methods only use one of the above methods, leading to insufficient perception of the anomalies' center or edge, resulting in inaccurate anomaly localization results.To solve this problem, we propose to decouple and remodel the features of student network to perceive anomalies from both the edge and center perspectives. Our idea is that the features of the student network corresponding to the anomalous region are able to be decomposed into the normality feature and abnormality feature. Among them, normality feature refers to the feature generated by the teacher network assuming that this region is normal, and abnormality feature is hoped to be different from the anomalous features of the teacher network.Therefore, we carry out the dual-branch design including Normality Branch and Abnormality Branch for the student network and propose a Decouple Student-Teacher Network. In addition to providing decoupling features for the subsequent distillation process, this design also differentiates the teacher and student architectures to some extent, and structurally avoids over-generalization of the student network. We design to add a 1 × 1 convolution layer after each stage of the student network to expand the channel by 2 times, and divide the output features into two parts along the channel dimension, which are used as initial normality feature and abnormality feature. Among them, the normality feature are used as the input of the next stage.For deep CNNs, as in Fig. 3 , the receptive field of high-level features is larger, while the receptive field of low-level features is smaller. That is, high-level features are greatly affected by surrounding pixels, while low-level features are less affected. For Normality Branch, the goal is to generate normality features on anomalous pixels. Therefore, considering that the larger the receptive field corresponding to the anomalous pixel is, the more normal information it contains, we believe that the normality features of higher stages are better generated. On the contrary, it is necessary for Abnormality Branch to generate features on anomalous pixels that are as different as possible from the teacher features. When the receptive field is large, the normal information that may appear in the receptive field corresponding to the anomalous pixel is likely to have an impact on the remodeling of the abnormality features. As a result, the normal information is more unlikely to be contained in the smaller receptive field, which means the abnormality feature remodeling at the lower stages is better. Based on the analysis above, we introduce multi-scale feature modeling into the student network, and propose a bidirectional dual-path Pyramid Modeling Network, as shown in Fig. 4 . For Normality Branch, a top-down-top pyramid feature modeling path is introduced. And a down-top-down feature modeling path is designed for Abnormality Branch. On one hand, Pyramid Modeling Network uses the better decoupled features to guide features with poor decoupling capabilities (inner path), and on the other hand, it increases the information contained in features by adding an opposite feature fusion path (outer path).By designing Normality Branch and Abnormality Branch, we get the initially decoupled normality features and abnormality features from the student network. To further utilize the features output by the teacher network to optimize the normality and abnormality features of the student network in the specific directions, we design Dual-Modeling Distillation.The entire distillation framework is divided into two modules: Normality Guidance Modeling (NGM) and Abnormality Inverse Mimicking (AIM), which remodel the normality features and abnormality features from two aspects. The features optimized by NGM and AIM remain similar to the teacher's features in normal regions, but are different from the teacher's features in anomalous regions.3.3.1 Normality Guidance Modeling. Since anomalous regions usually have different pixel distributions from normal regions, the features extracted by CNN in normal regions and anomalous regions are obviously different. Based on this experience, some methods use the similarity of the features between pixels or patches to determine whether there occurs anomaly. The key to this type of method is how to remember the normality features of the images.Most of the previous methods [13, 15, 45] use memory bank or generative network to store or simulate normality features. However, memory bank requires a large storage space, and the operation of searching the memory bank has greater computational complexity. The generative network requires both a encoder and a decoder, which requires a large amount of calculation to generate normality features. In addition, designing the generative network as the student leads to a large structural difference between the teacher network, facing the risk of underfitting. Therefore, for the task of modeling normality features, we draw on the idea of Mask Image Modeling (MIM) used in self-supervised learning. Based on the synthetic anomalies as the masks and the normality features generated by the teacher network as the regression goal, we propose a novel distillation method Normality Guidance Modeling.During training, input each normal image 𝐼 𝑛 to the teacher and student networks at the same time. In addition, the input of the student network also includes the corresponding synthetic anomalous image 𝐼 𝑛 𝑎 . NGM uses the feature 𝐹 𝑇 𝑛 corresponding to 𝐼 𝑛 extracted by the teacher as distillation indicator, and minimizes the distances from the features generated by the student on 𝐼 𝑛 and 𝐼 𝑛 𝑎 to 𝐹 𝑇 𝑛 . Let 𝐹 𝑆 𝑁 𝐺𝑀 𝑛 𝑖 and 𝐹 𝑆 𝑁 𝐺𝑀 𝑎 𝑖 represent the output normality features of the 𝑖-th stage of the student network for a pair of normal and synthetic anomalous images respectively. Let 𝐹 𝑇 𝑛 𝑖 represent the feature of the 𝑖-th stage corresponding to the normal image output by the teacher network. Then the loss 𝐿 𝑁 𝐺𝑀 used to train NGM distillation is calculated based on cosine similarity asEQUATIONEQUATIONwhere 𝐻 𝑖 and 𝑊 𝑖 respectively represent the height and width of the features output by 𝑖-th stage.Mimicking. Distillation for unsupervised AD generally refers to reducing the distances between the features output by the student and teacher networks on normal images. By default, the student does not learn the feature representation ability of the teacher in anomalous regions. Therefore, the conventional distillation method does not directly train the model on the abnormality features, but only implicitly differentiates the features of the student network and the teacher network in the anomalous regions through forward distillation in the normal regions. However, this kind of forward distillation can easily lead to over-generalization of abnormality features between student and teacher networks because of the neglect of the training of anomalous regions. Following the idea of Pull & Push [46] , we introduce the distillation method Abnormality Inverse Mimicking, which explicitly gives the meaning of the abnormality features of the student network by maximizing the cosine distances between the features of the student and teacher networks on synthetic anomalies. After remodeling by Abnormality Inverse Mimicking, the student network is able to output abnormality features that are significantly different from the teacher network in anomalous regions, especially anomaly centers.With the help of anomaly mask and L1 distance, we unify the AIM distillation training of normal images and synthetic anomalous images. Let the input image be 𝐼 , which may be a normal image 𝐼 𝑛 or an anomalous image 𝐼 𝑛 𝑎 . For each image 𝐼 , let the 𝑖-th stage output of the teacher network be 𝐹 𝑇 𝑖 , and let the 𝑖-th stage output of Abnormality Branch of the student network be 𝐹 𝑆 𝐴𝐼 𝑀 𝑖 . Then the AIM distillation loss 𝐿 𝑁𝐺𝑀 is expressed asEQUATIONwhere 𝑀 𝑔𝑡 𝑖 represents the ground-truth anomaly segmentation mask which is downsampled to the feature size of the 𝑖-th stage, 𝑀 𝐴𝐼 𝑀 𝑖 represents the cosine distance map between teacher and student features of the 𝑖-th stage, calculated asEQUATIONIn most previous AD algorithms based on knowledge distillation, such as RD [11] and RD++ [31] , the cosine distances between features of each stage of the teacher and student networks are added or multiplied as the final anomaly map. However, since the weights used in fusing the anomaly maps of different stages are the same, the difference in correctness of the anomaly maps is ignored. Besides, due to the structural characteristics of objects in the images, the likelihood of anomaly occurrence varying from region to region. Therefore, anomaly map fusion methods used in DeSTSeg [45] where each pixel in the same anomaly map uses the same weight when fusing do not fully utilize the pattern information of the images, resulting in inaccurate anomaly localization.To fuse anomaly maps in an optimal way and improve the location accuracy of the fused anomaly map, we propose Multiperception Segmentation Network in this section. First, we design a data enhancement method Foreground-aware Anomaly Synthesis to control the synthetic anomalies within the foreground of the object images, thereby optimizing the subsequent training and making the fusion network pay more attention to the foreground areas where anomalies are more likely to occur. After that, we propose Multi-perception Segmentation Head to obtain the optimal anomaly map based on multi-perception mechanism.3.4.1 Foreground-aware Anomaly Synthesis. In recent years, anomaly synthesis methods have been gradually introduced into the design of unsupervised AD algorithms, providing synthetic anomalous images and corresponding binary anomaly masks as the ground truth for the model training process. Among them, the simulated anomaly generation method proposed in DRAEM [40] that uses random two-dimensional Perlin noise and the images from the external dataset Describable Textures Dataset (DTD) [8] is widely recognized and introduced into a variety of AD algorithms. However, considering that anomalies in some industrial images may only occur in the foreground where objects are located, synthesizing anomalies based on random full-image noise is likely to result in large differences between synthetic anomalies and real anomalies.To solve this problem, we make some improvements based on DRAEM and put forward Foreground-aware Anomaly Synthesis. Utilizing the foreground extraction method in traditional image processing field, we realize anomaly synthesis only in the foreground areas of the images, aiming to generate synthetic anomalous images that are closer to real ones. The detailed process is shown in the Fig. 5 . First, employ GrabCut [25] algorithm to extract the foreground 𝑀 𝑓 𝑜𝑟𝑒 . Then, use binarized Perlin noise 𝑀 𝑛𝑜𝑖𝑠𝑒 restricted to the foreground to superimpose the external texture image as random noise. Finally, add the noise to the normal image 𝐼 𝑛 and obtain the synthesized anomalous image 𝐼 𝑛 𝑎 . First, for each test image 𝐼 , a total of eight anomaly maps are calculated based on the pixel-wise cosine similarity using the intermediate features obtained after NGM and AIM distillation in the student network and the features of the teacher network.Second, based on the ideas mentioned in 3.2, we also introduce Pyramid Upsampling to achieve the fusion of high-level and lowlevel anomaly maps. For the anomaly maps 𝑀 𝑁𝐺𝑀 𝑖 , 𝑖 = {1, 2, 3, 4} corresponding to NGM distillation, a top-down fusion path is added. For the abnormal maps 𝑀 𝐴𝐼 𝑀 𝑖 , 𝑖 = {1, 2, 3, 4} corresponding to AIM distillation, a down-top fusion path is added. For the fused anomaly maps, we uniformly perform bilinear interpolation upsampling operation to expand them to the input image size.Third, we utilize Multi-perception Mechanism to process and fuse the concatenated eight-channel anomaly map, as in Fig. 6 . At the start, to enable anomaly segmentation to focus more on stages with stronger anomaly detection capabilities, we introduce a channel attention mechanism based on the eight-channel tensor. Besides, a spatial attention mechanism is added, which effectively makes the anomaly localization process pay more attention to the most likely regions where anomalies occur. Considering that the convolution kernel is unchanged for different regions of the images, we innovatively propose a global attention parameter, to distinguish the weights on different regions of different channels during averaging the processed eight-channel tensor along the channel dimension Finally, after channel compression, pass the tensor through a sigmoid layer to obtain the fused anomaly segmentation map 𝑀, average the values of the maximum 100 pixels in 𝑀 with sigmoid to calculate the anomaly score 𝑆. Here, we optimize segmentation map and the image anomaly score using Binary Cross-Entropy loss (BCE) as EQUATION 
implementation details: 
 Experiment Setting. Consistent with previous unsupervised AD algorithms, we train corresponding detection models for each category of image in the dataset separately. All images input to the network during the training and inference process are resized to a fixed resolution of 256 × 256, and no other image augmentation Evaluation Metrics. We report area under the ROC curve (AUC) as the evaluation metrics for image-level detection and pixel-level localization, abbreviated as I-AUC and P-AUC. For anomaly localization, per-region-overlap (PRO) [5] is also used for comparison.Anomaly Detection on MVTec AD. Table 1 reports the anomaly detection and localization results of the advanced KD-based unsupervised AD methods and our proposed DMDD on MVTec AD. The average pixel-level anomaly localization results of DMDD are proved to outperform other KD methods to reach SOTA. Notably, our method surpasses the previous methods in all metrics on texture images, and exceeds RD++ [31] which is the current KD-based SOTA method for unsupervised AD task by 0.55% and 1.13% on total average P-AUC and PRO. The anomaly detection qualitative results are visualized in Fig. 7 .Anomaly Detection on BTAD. We exhibit the experimental results of DMDD on BTAD as in Table 2 . Compared with previous unsupervised AD methods, DMDD achieves the best average localization performance, reaching 98.06% and 80.28% on P-AUC and PRO. Anomaly Detection on MPDD. We perform experimental validation of anomaly localization on MPDD, and the related results are reported in Table 3 . To better compare with the KD-based methods, we re-conduct the experiments related to RD [11] and RD++ [31] on MPDD and record the results exactly according to the original settings of these two methods. Obviously, our method is more outstanding in anomaly detection and localization capability.Ablation Study on Student-Teacher Network Architecture. We first conduct ablation experiments on the structure of Decoupled Student-Teacher (S-T) Network to investigate the necessity of the dualbranch design and the effectiveness of the proposed Pyramid Modeling Network (PMN). Table 4 is divided into three parts from top to bottom, showing the anomaly detection performance of only Normality Branch with NGM distillation, only Abnormality Branch with AIM distillation, and the whole Decoupled Student-Teacher Network with Dual-Modeling Distillation (DMD). The significant improvement of PRO with the similar AUCs proves the localization superiority of the dual-branch design over the single branch. In addition, we experimentally evaluate the inclusion of PMN, and it can be seen from Table 4 that the anomaly detection effect is significantly improved by simultaneously adding two multi-scale fusion paths, inner and outer, of PMN. Some relevant visualization results are exhibited in Fig. 8 , mainly including the anomaly maps of each stage output by the two branches with or without PMN. It is evident that the detection of anomaly maps of all stages is improved by the addition of PMN, which proves that PMN is capable of increasing the accuracy of anomaly localization. In addition, it is intuitively clear from Fig. 8 that Normality Branch and Abnormality Branch pay attention to different regions of anomalies. Compared to Normality Branch, Abnormality Branch only focuses on the center region of anomalies, which explains why the dual-branch design yields the best detection performance.Ablation Study on Multi-perception Segmentation Network Components. As shown in Table 5 , compared to anomaly map fusion using only plain summation (w/o MSN), our proposed Multi-perception Segmentation Network (MSN) improves I-AUC, P-AUC and PRO by 0.22%, 0.13% and 0.43% respectively. In addition, we conduct ablation studies on the components of Multi-perception Fusion Network. It turns out that the removal of Foreground-aware Anomaly Synthesis (FAS), i.e.,using full-image anomaly synthesis, or the subtraction of Pyramid Upsampling (PU) or Multi-perception Mechanism (MM) in Multi-perception Segmentation Head, all make DMDD less effective at detecting and localizing anomalies, which squarely justifies the need for these modules. 
conclusion: 
 In this paper, we propose a novel unsupervised anomaly detection method based on knowledge distillation paradigm DMDD. A Decoupled Student-Teacher network is employed to differentiate the network structures while ensuring the same information capacity for the teacher and student networks. Dual-Modeling Distillation is proposed to distill student features from both normality simulation and abnormality distancing, allowing the anomaly maps derived from the feature similarity of student and teacher to focus on both the anomaly edge and center. Besides, to optimize the anomaly segmentation process, Multi-perception Segmentation Network is presented. Experimental outcomes demonstrate that comparing with the previous knowledge distillation methods, our method significantly improves the image anomaly localization effect. 
 
 Appendix:  


Decision: This paper originally received scores of 2 Weak Accept, 1 Borderline Reject, and 1 Weak Reject.

According to the reviewers and my evaluation, the paper has clear strengths in its interesting and well-designed model and its good performance. However, the issues and limitations are mainly related to the representations of some claims and discussions. These need to be addressed in the final version according to the reviews.

Reviews: review: paper_summary: paper_summary: The paper 'Dual-Modeling Decouple Distillation for Unsupervised Anomaly Detection' proposes an interesting approach for AD in images, based on reverse distillation. It is a well written paper, with minor details that need correction.
-  The paper introduces a new approach, Dual-Modeling Decouple Distillation (DMDD). It decouples the student & teacher networks for abnormality and normality learning. 
- The paper presents a robust set of experiments, and ablation studies. The proposed approach outperforms the SoTa, with multiple metrics.
strength_weakness: strength_weakness: 1. Decoupled Student Teacher Network: The underlying ideation and motivation is crisply explained (Sec 3.2). 
2. Pyramid Modeling Network : design is well thought out. The motivation of the abnormatity and the normality branch makes sense. The overall idea of receptive field behaving differently, for normal pixels vs anomalous pixels is correct. 
3. Dual-Modeling Distillation: the loss function, and the overall architecture conceptually differentiates mining the anomalies from normal samples.
4. the set of experiments are exhaustive and comprehensive. Its good that the paper visits more than one metric.
5. the ablation study provided is useful
limitations: limitations: 1.Minor errors:
- Line 84 : KD-base methods assumpt -> assume or make assumption
- Line 344: features , not feature. It is a collection of features, not a single one.  
- 
2. The core idea discussed in lines 370 - 380 lacks a clarity
3. the apper is overall well organized, but clarity in some parttscould be improved. Long complex sentences are a hindrance to comprehension.
4. the conclusion section could have been informative, with a bit further light on future work.
5. Lines: 95-98 : This is a  strong assumption. A paper should present a hypothesis,  not a strong assumption as a premise.  Introduction section should avoid as such.
paper_summary: paper_summary: This paper proposes a Dual-Modeling Decouple Distillation method (DMDD) for the unsupervised Anomaly Detection. The experimental results demonstrate the effectiveness of the author's method. However, the experimental results seem to be incomplete. Representative comparison methods (PatchCore [24]) were not run on all datasets.
strength_weakness: strength_weakness: 1.The experimental results demonstrate that the performance of DMDD exceeds previous state-of-the-art (SOTA) methods based on knowledge distillation.

2.The author employs a dual-branch dual-modeling decoupling distillation approach to perform distinct distillation.
limitations: limitations: 1.The comparison with the key method, PatchCore [24], should encompass all datasets in order to enhance the persuasiveness of the proposed method.

2.This paper may benefit from outlining the key contributions in the introduction in a concise manner, in order to help readers grasp the differences between the authors' method and other existing approaches quickly.

3.The MemKD column in Table 3 is empty, yet there is an average result for it. Additionally, the textual description does not mention MemKD. The same situation occurs in Table 1.The author should provide complete results.

4.The author should emphasize the distinctiveness of the proposed method in comparison to existing methods, as the current discussion lacks this aspect.

5.The author has not released the code, lacking reproducibility.

6.What is “improving distillation content from normality or abnormality only has the limitation of anomaly focus”? Is it referring to the limitation caused by favoring one aspect too much?
paper_summary: paper_summary: This paper presents a novel unsupervised image anomaly detection method called DMDD. Student-Teacher network is employed to decouple student features into normality and abnormality features. Dual-Modeling Distillation is used to distill student features. Multi-perception Segmentation Network is used to optimize the anomaly segmentation process. Experiments on three AD datasets demonstrate its performance over existing approaches.
strength_weakness: strength_weakness: Unsupervised image anomaly detection is an interesting research topic with important applications in various fields. The strengths of this paper are:

1. Proposing a new model based on knowledge distillation to solve the problem.
2. The discussion of difference between edge and center for normal and abnormal samples is interesting, but needs justification.
limitations: limitations: There are several key drawbacks for this paper:

1. The writing needs to be improved to make it easier to understand and more logically sound. For example, the introduction section is hard to follow when it comes to the authors' method design. It is suggested that the authors rephrase it to make it clearer.
2. S3.2, "Existing methods only use one of the above methods, which leads to insufficient perception of the anomalies’ center or edge, resulting in inaccurate anomaly localization results." What does the anomalies' center and edge mean? How does it affect the anomaly detection process? The statements needs clarification and experiments to prove this point.
3. The author names the proposed method as an unsupervised one. However, the training used synthesized abnormal samples by stacking normal samples and noise. The question is, if you have no knowledge about how actual abnormal samples look like, how can you generate good synthetic abnormal samples? Are all the methods being compared also used synthesized abnormal samples for training? If no, then the comparison is unfair.
4. The citation is insufficient, e.g., GrabCut (line 589, left column)

A few typos:

1. Line 84, right column, "KD-based" instead of "KD-base". There are also a few other "KD-base" in the paper, which should be modified.
2. Line 84, right column, "assume" instead of "assumpt"
3. Line 89, right column, "in practice" instead of "in practical"
paper_summary: paper_summary: This paper proposes a novel Dual-Modeling Decouple Distillation for the unsupervised anomaly detection. In DMDD, a Decouple Student Network is proposed to decouple the initial student features into normality and abnormality features. The decoupled features are then distilled through Normality Guidance Modeling and Abnormality Inverse Mimicking to highlight their normality and abnormality.
strength_weakness: strength_weakness: Decoupling features of the student network into normality feature and abnormality feature is interesting. The experimental results show the proposed method can achieve SOTA anomaly localization results on popular AD datasets.
limitations: limitations: [Major]
1. In line 344-347, “feature of anomalous region to be decomposed into the normality feature and abnormality feature”. Decoupling feature into normality and abnormality feature is the core idea in your paper. But, what do normality and abnormality represent respectively? Why can a feature be decomposed into normality and abnormality features? This is not well explained in the paper, a more intuitive explanation is needed.

2. During training, does the method have to have anomalies or pseudo anomalies? If only with normal samples, is it still usable?

3. In Figure 6, it would be better to compare with other SOTA methods, while not just provided the qualitative results of your method.

4. Implementing the proposed method is non-trivial, will the code be open source? As a good work, open source code is required to ensure reproducibility.

[Minor] 

1.In line 226 and 229, the $I_1, I_2$, … needs to be wrapped by {…}. In line 504, there should be $L_{AIM}$. In figure 2, the right part feature map F is hard to see clearly.