AR-Pro: Counterfactual Explanations for Anomaly Repair with Formal Properties

Authors: xiayan  ji
anton  xue
eric  wong
oleg  sokolsky
insup  lee
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-14 04:26:15

Abstract: Anomaly detection is widely used for identifying critical errors and suspicious behaviors, but current methods lack interpretability.
We leverage common properties of existing methods and recent advances in generative models to introduce counterfactual explanations for anomaly detection.
Given an input, we generate its counterfactual as a diffusion-based repair that shows what a non-anomalous version $\textit{should have looked like}$.
A key advantage of this approach is that it enables a domain-independent formal specification of explainability desiderata, offering a unified framework for generating and evaluating explanations.
We demonstrate the effectiveness of our anomaly explainability framework, AR-Pro, on vision (MVTec, VisA) and time-series (SWaT, WADI, HAI) anomaly datasets. The code used for the experiments is accessible at: https://github.com/xjiae/arpro.

Main paper: 
 introduction: 
 Anomaly detectors measure how much their inputs deviate from an established norm, where too much deviation implies an instance that warrants closer inspection [15, 52] . For example, unusual network traffic may indicate potential malicious attacks that necessitate a review of security logs [16, 30] , irregular sensor readings in civil engineering may suggest structural weaknesses [40, 49, 53] , and atypical financial transactions point to potential fraud [3, 47] . As a benign occurrence, unexpected anomalies in scientific data can also lead to new insights and discoveries [25, 45] .Although state-of-the-art anomaly detectors are good at catching anomalies, they often rely on black-box models. This opacity undermines reliability: inexperienced users might over-rely on the model without understanding its rationale, while experts may not trust model predictions that are not backed by well-founded explanations. This limitation of common machine learning techniques has led to growing interest in model explainability [12] , particularly in domains such as medicine [50] and law [14] . We refer to [36, 44] and the references therein for recent surveys on explainability.While many anomaly detection methods can localize which parts of the input are anomalous, this may not be a satisfactory explanation, especially when the data is complex. In medicine [20] , heart sound recordings [17] and EEG brain data [5] can differ greatly between individuals; in industrial manufacturing, it can be hard to understand subtle defects of PCB for inexperienced workers [73] . Thus, even when the location of the anomaly is known, it may be hard to articulate why it is anomalous. In such cases, it can be helpful to ask the counterfactual question: "What should a non-anomalous version look like?" For example, a doctor might ask what changes in a patient's chest X-ray might improve diagnostic outcomes [58] , while a quality assurance engineer may ask what changes would fix the defect [6] . To answer such counterfactual questions, we begin with two observations. First, anomalies are commonly localized to small regions of the input [13, 38, 73] . Second, recent generative models, e.g., diffusion [24] models, can be trained to produce high-quality, non-anomalous examples. These observations motivate us to repair anomalies as a counterfactual explanation. However, simply generating a repair is not sufficient; we must also ensure its quality. For example, it may be undesirable for a repair to significantly improve the anomaly score but barely resemble the original input. Therefore, it is important to measure the quality of repairs using formal properties.However, formalizing broadly applicable properties is challenging, as different domains (e.g., vision, time series) and tasks (e.g., manufacturing, security) have unique considerations. To overcome this, we further observe that many anomaly detectors in practice [4] satisfy a condition that we call linear decomposability: the overall anomaly score is an aggregation of feature-wise anomaly scores. Importantly, this is a strong but common condition with which we may formalize the desiderata of counterfactual explanations. Conveniently, many of these desiderata are, in fact, domainindependent. We leverage these conditions to formalize a general, domain-independent framework for counterfactual anomaly explanation: we use a generative model to produce a repair and then evaluate this repair with respect to the detector model.We present an overview of our anomaly explainability framework, AR-Pro, in Figure 1 . While anomaly repair [21, 31, 70] and counterfactual explanations [56] have been explored in the literature, we are the first to study this in a unified context. We summarize our contributions as follows:• We observe that many anomaly detectors satisfy linear decomposability and use this condition to define general, domain-independent properties for counterfactual explanations. This approach lets us measure explanation quality with respect to the anomaly detector, as shown in Figure 1 .• We use these properties to guide diffusion models towards a high-quality repair of the anomalous input. Such a repair serves as the counterfactual explanation of the anomaly.• Our framework, AR-Pro, can produce semantically meaningful repairs that outperform off-theshelf diffusion models with respect to our explainability criteria. Our vision anomaly benchmarks include MVTec [13] and VisA [73] . Our time-series anomaly benchmarks include SWaT [38] , HAI [54] , and WADI [2] . 
overview and formal properties: 
 In Section 2.1, we first observe that many anomaly detector paradigms are linearly decomposable. Intuitively, this condition means that the overall anomaly score is an aggregation of feature-wise anomaly scores. Next, we use linear decomposability in Section 2.2 to formalize general, domain-independent properties of counterfactual explanations. We will use the terms counterfactual, explanation, and repair interchangeably throughout this paper. α i (x) = |x i -x i | 2 ∈ R n for i = 1, . . . , n.Then, the standard ℓ 2 reconstruction-based anomaly score is a linear combination of the feature scores:s(x) = α 1 (x) + • • • + α n (x).Many anomaly detection techniques use a scoring function s : R n → R to measure the anomalousness an input x ∈ R n , where s(x) is called the anomaly score of x. This is commonly done in a twostage process: the feature-wise anomaly scores α 1 (x), . . . , α n (x) ∈ R are first computed and then aggregated [15] . We observe that this aggregation often satisfies the following form:Definition 2.1 (Linear Decomposition). The feature-wise anomaly scores α : R n → R n and regularizer β : R n → R linearly decomposes the anomaly score s : R n → R if for all x ∈ R n :s(x) = α 1 (x) + • • • + α n (x) + β(x).While linear decomposability appears to be a strong assumption, it is common in practice. We show a number of examples below, where we also flexibly refer to α : R n → R n as the feature scores.Example 2.2. In reconstruction-based anomaly detection [8, 29] , the input x is passed through an encoder-decoder architecture to generate a reconstruction x. The motivation is that it should be harder to reconstruct out-of-distribution (anomalous) inputs. Empirically, it is observed that if x is anomalous, then x will have a large reconstruction error when feature i is in the anomalous region, e.g., a pixel in the defect area of a manufacturing artifact image. A typical example of such an anomaly score is:s(x) = |x 1 -x 1 | 2 + • • • + |x n -x n | 2 ,which is also known as the ℓ 2 reconstruction error. This lets us define the feature-wise anomaly scores by α i (x) = |x i -x i | 2 , and we illustrate this example in Figure 2 .Example 2.3. In maximum likelihood-based anomaly detection [15] , one measures the likelihood of a test input x with respect to a set of non-anomalous training examples. Intuitively, an out-ofdistribution (anomalous) x should be unlikely with respect to the non-anomalous training examples and will thus have a lower likelihood. In variants such as normalizing flow-based anomaly detection for images [66] , it is common to define a joint probability distribution over all the features:s(x) = -log p 1 (x) + • • • + log p n (x) + log|det J(x)| ,where p 1 (x), . . . , p n (x) are the probabilities of each feature that lets us define α i (x) = -log p i (x), while the change-of-variable Jacobian log|det J(x)| may be viewed as a regularization term.Example 2.4. In language modeling [61] , it is common to measure the anomaly of a token sequence based on the likelihood of each token [7] . Although similar to the vision case, the standard formulation for language models is different: given a token sequence x 1 , . . . , x n ∈ {1, . . . , vocab_size}, its measure of unlikeliness (anomalousness) may be defined as:s(x) = - 1 n log p(x 1 ) + log p(x 2 |x 1 ) + log p(x 3 |x 1 , x 2 ) + • • • + log p(x n |x 1 , . . . , x n-1 ) ,where p is a probabilistic generative model, and soα i (x) = -(1/n) log p(x i |x 1 , . . . , x i-1). This is also known as a preplexity measure, which has been used to detect jailbreaks against LLMs [64] .Beyond the above examples, anomaly detectors for time-series data [29, 60, 63] and text [35] also commonly use this convention. We further remark that the feature-wise anomaly scores α are related to feature attribution scores in explainability literature [33, 51, 55] .We now present the formal properties of counterfactual explanations. We will assume a given anomaly score function s that is linearly decomposed by the feature-wise score function α and regularizer β.Given some input x, it is common to convert α(x) ∈ R n into a binary-valued vector ω(x) ∈ {0, 1} n to classify which input feature is anomalous, and this is commonly done by a threshold:ω(x) = (α 1 (x) ≥ τ 1 , . . . , α n (x) ≥ τ n ), for some feature-wise threshold τ ∈ R n .A binarization of the feature-wise scores suggests the need for region-specific anomaly scores, which we implement with the following indexing scheme on s(x):s z (x) = β(x) + i:zi=1 α i (x), for all z ∈ {0, 1} n .Then, the anomalous and non-anomalous regions have scores s ω(x) (x) and s ω(x) (x), respectively, where ω(x) = 1 -ω(x) denotes non-anomalous region. We next enumerate some common desiderata of counterfactual explanations, where we will refer to the anomalous input as x bad and the repaired version as x fix .Property 1 (Overall Improvement): The anomaly score should improve. Because a "repaired" version should fix the anomaly by definition, one would reasonably expect that:s(x fix ) < s(x bad ). (P1)Property 2 (Similarity): The repair should resemble the original. When s is generated by a complex machine-learning model, it may be the case that it has a value x fix where s(x fix ) ≪ s(x bad ), but x fix and x bad bear little resemblance. In the case of vision models, x fix may even resemble static noise. Such extreme dissimilarities between x fix and x bad are not desirable because a user cannot be expected to feasibly interpret this information. Thus, we desire a similarity condition as:EQUATIONwhere ⊙ denotes element-wise vector multiplication. This similarity condition states that the nonanomalous regions of the original and the repair, as given by ω(x bad ), should remain similar.Property 3 (Localized Improvement): The anomalous region should improve. However, P1 and P2 are not sufficient. For example, one might have s(x fix ) < s(x bad ), but have a higher score on the anomalous region ω(x bad ). This is not desirable because it means that x fix has not actually fixed the anomalous region of x bad . To ensure progress, we would like:s ω(x bad ) (x fix ) < s ω(x bad ) (x bad ). (P3)Property 4 (Non-degradation): The non-anomalous region should not significantly worsen. Even when the above properties are satisfied, it is possible that the proposed repair could inadvertently increase the anomaly score on the non-anomalous region ω(x bad ). This would mean repairing the anomalous region at the cost of corrupting the non-anomalous parts. We thus state the property against this as follows, where δ 4 > 0 is a given tolerance threshold:EQUATION)The benefit of our above formulation is that it encapsulates general, domain-independent desiderata of anomaly repairs. Importantly, this is achieved under the mild assumption of a linearly decomposable anomaly detector. We comment that some overlap among our proposed formal properties may arise in certain scenarios, and alternative sets could be more tailored for specific applications. Our goal, however, is to provide a foundational set of properties that ensures broad applicability, allowing further customization to suit individual application needs.Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes] Justification: We discuss the limitation in Section 6. Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
property-guided generation of counterfactual explanations: 
 We now outline the process of conducting a property-guided repair for an anomalous input. In Section 3.1, we first introduce the generalized setup, where we define the four previously outlined properties as objective functions and frame the problem using risk-constrained optimization. Although this formulation clarifies the objectives, it is generally intractable. Therefore, in Section 3.2, we propose a diffusion-based algorithm to approximate the solution and achieve high-quality repairs.Figure 3 : We run property-guided diffusion with masked in-filling.We first present a generalized setup for generating repairs. We conceptualize this in terms of a repair model R s,ω parametrized by an anomaly score function s : R n → R with known linear decompositions α, β and a feature-wise binarization ω. To generate a repair, we sample the model:x fix ∼ R s,ω (x bad ),where a probabilistic formulation is relevant in the context of variational auto-encoders [27] or diffusion models [24] . However, it is important that x fix obeys the formal properties as outlined in Section 2.2. To do this, we cast these properties as loss functions listed below:L 1 = s(x fix ) (P1 loss) L 2 = ∥ω(x bad ) ⊙ (x fix -x bad )∥ 2 (P2 loss) L 3 = max 0, s ω(x bad ) (x fix ) -s ω(x bad ) (x bad ) (P3 loss) L 4 = max 0, s ω(x bad ) (x fix ) -s ω(x bad ) (x bad ) -δ 4 (P4 loss)Our rationale is as follows. First, because the primary objective of anomaly repair is to reduce the anomaly score, we set L 1 as simply the score of x fix . Second, we would like to ensure that x fix and x bad are similar in the non-anomalous region, and so formulate L 2 as the ℓ 2 distance between x fix and x bad over ω(x bad ). Third, we formulate L 3 to apply a penalty when the anomalous region degrades in performance. Fourth, we allow for a degradation of score in the non-anomalous region ω(x bad ), up to some tolerance threshold δ 4 . We cast these as a risk-constrained optimization problem as follows:minimize θ Ex fix ∼Rs,ω(x bad ;θ) s(x fix )subject to Px fix ∼Rs,ω(x bad ;θ)EQUATIONwhere θ is a parameter of the repair model, δ 2 > 0 is a given threshold for L 2 and δ > 0 is a given failure probability for the violation of at least one of (P2), (P3), or (P4). We acknowledge that multiple formulations for property-based losses are valid; however, the chosen approach is optimally suited to our context.We adopt techniques from guided diffusion [19] to generate repairs x fix . We first give a brief overview of a standard diffusion process and then adapt it to perform property-guided generation. We refer to [62] for a comprehensive guide on diffusion but attempt to make the exposition self-contained.Background. A basic variant of diffusion models takes the form:EQUATIONwhere µ θ is the denoising model with parameters θ, and b 1 < • • • < b T is the variance schedule. When µ θ is trained on non-anomalous data (see [62] for training details), one can generate non-anomalous samples of the training distribution by running the following iterative process:EQUATIONwhere x T is the initial noise and x 0 is the output sample. The idea is to repeatedly remove noise from a Gaussian x T using µ θ until the final x 0 resembles a high-resolution image without defects, for instance. The iterations (3) is also known as backward process.Property-guided Diffusion. We next show how to adapt the denoising iterations (3) to produce repairs. We use two main ideas: first, we use guidance [19] to slightly nudge the iterates x t-1 of (3) at every step using a property-based loss, to encourage that the final iterate x 0 , which we take to be x fix , is more amenable to our properties. Second, we used masked-infilling [39] to ensure that the non-anomalous region ω(x bad ) is generally preserved by the iterates. We implement this modified iteration as follows: beginning from the initial noise x fix,T ∼ N (0, I), let:EQUATIONEQUATIONEQUATIONfor t = T, . . . , 1, wherea t = t i=1 (1 -b i ) and η 1 < • • • < η Tis the guidance schedule. The property-based loss is given by:EQUATIONwith L 1 , L 2 , L 3 , L 4 as in Section 3.1 and weights λ 1 , λ 2 , λ 3 , λ 4 > 0. In the above, (4) first generates xfix,t-1 from x fix,t by combining a standard denoising step with a guidance term. Then, ( 6) combines xfix,t-1 with xbad,t , from ( 5), in a masked-infilling operation [39] to yield x fix,t-1 . This masked in-filling ensures similarity between x bad and x fix (i.e., x fix,0 ) over the non-anomalous region ω(x bad ).We emphasize that the diffusion iterations given by ( 4), ( 5), ( 6), and do not guarantee the satisfaction of our formal properties. Rather, these iterations tend toward an output that better respects these properties -as we later shown in our experiments. In particular, our property-based losses define a way to evaluate the quality to which each property is satisfied or violated.Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?Answer: [Yes] Justification: Although we did not have proofs, we listed the assumptions in Section 2 and Section 3. 
experiments: 
 Our experiments evaluate the performance of AR-Pro across vision and time-series datasets. In particular, we aim to address the following research questions:• (RQ1) Empirical Validation: How well does AR-Pro repair anomalies for different domains? In particular, we investigate how well the four properties are satisfied when the diffusion process is guided or unguided, as in the baseline. • (RQ2) Ablation Study: How do the different hyper-parameters affect the repair quality? We focus on the weights λ 1 , λ 2 , λ 3 , λ 4 for the four property-based losses. Vision Anomaly Models and Datasets. For anomaly detectors, we used the anomalib [4] implementation of Fastflow [66] (with ResNet-50-2 backbone [68] ) and Efficient-AD [11] . For datasets, we used the VisA [73] and MVTec-AD [13] datasets. VisA and MVTec-AD involve anomaly detection in the context of industrial manufacturing, where VisA consists of 12 image classes, and MVTec-AD consists of 15 image classes. Both FastFlow and Efficient-AD were trained with AdamW and a learning rate of 10 -4 until convergence.Time-series Anomaly Datasets and Models. For anomaly detectors, we used the GPT-2 [48] and Llama2 [59] architectures for time-series anomaly detection. In particular, we use only the first 6 layers of GPT-2 and the first 4 layers of Llama-2 (with an embedding dimension of 1024) to accelerate training. For datasets, we used the SWaT (51 features) [38] , HAI (86 features) [54] , and WADI (127 features) [2] datasets, split into sliding windows of size 100. Both our versions of GPT-2 and Llama-2 were trained with AdamW and a learning rate of 10 -5 until convergence.Diffusion-based Repair Models. We used the HuggingFace implementation of DDPM [24] for vision data and Diffusion-TS [67] for time-series data. Both models were trained on the non-anomalous instances of their respective datasets using AdamW and a learning rate of 10 -4 until convergence.Evaluation Metrics. We use the four property-based loss functions defined in Section 3.1 as our evaluation metrics. In particular, we measure the improvement of property-guided diffusion over un-guided diffusion. We adapt these metrics below, where we write ω to mean ω(x bad ) for brevity:• Property 1 (Overall Improvement): M s ≡ s(x fix ).• Property 2 (Similarity): Table 1 : AR-Pro outperforms a non-guided diffusion baseline across our four metrics. We show the results for all VisA and MVTec-AD categories, where ∆ is the median percentage improvement.M d ≡ ∥ω ⊙ (x fix -x bad )∥ 2 • Property 3 (Localized Improvement): M ω ≡ s ω (x fix ) -s ω (x bad ) • Property 4 (Non-degradation): M ω ≡ s ω (x fix ) -s ω (x bad ) Dataset Class Ms(↓) Md(↓) Mω(↓) Mω(↓)Each experiment employs a representative anomaly detector and dataset with predefined train-test splits. The performance of the anomaly detectors is detailed in Appendix B. For ω, each feature-wise threshold τ i is taken to be the 90% quantile of the training set's feature-wise anomaly scores.We now evaluate the performance of AR-Pro for generating anomaly repairs with respect to the four evaluation metrics. We report the mean and standard deviation values. Because there may be a large range of values across classes, we report the median improvement (∆) of the guided generation over the baseline. We present results for the FastFlow anomaly detector on the VisA and MVTec dataset in Table 1 , and refer to Appendix C for additional results with Efficient-AD.Quantitative Results. For all the categories, the guided results show significant improvement over the baseline on the formal criteria, with an average improvement of 84.27%. There is a wide range of M s values for baseline models in certain classes, possibly due to deviations in the color scheme of the generated images from their training distribution. Hence, we report the median improvement across the classes in the last row to mitigate the impact of outliers.We show results for time-series data in Table 2 . AR-Pro achieves lower error on M d , M ω , and M ω , with an overall improvement of 60.03% over the baseline. Specifically, Llama-2 achieves an average improvement of 67.17% in formal metrics, while GPT-2 increases by 53.90% on average. Compared to image data, the performance on M d is not as competitive and exhibits considerable variability, likely due to the broader range of adjustments required for repairs to ensure smooth signals.In addition, we evaluate whether the guided repairs generate non-anomalous samples by comparing them against a conformity threshold with 95% confidence derived from the training set [9] . Treating the non-anomalous class as the negative, we report the True Negative Rate (TNR) in Table 3 . The TNR of most VisA and MVTec-AD categories reaches 100%, surpassing the majority of baseline values. As a result, the guided repair achieves a median TNR of 100% for both VisA and MVTec, representing an average improvement of 2.50% over the baseline. Across the three time-series datasets (SWaT, HAI, and WADI), the guided repair obtains an average TNR of 95.33%, which is 92.66% higher than the baseline average of 2.67%. Overall, 99.26% of guided repairs across both domains are classified as non-anomalous with 95% confidence, statistically confirming the effectiveness of AR-Pro.Qualitative Results. We present qualitative examples here to illustrate that our method can generate semantically meaningful repairs, as shown in Figure 4 for VisA and Figure 5 for MVTec-AD. For example in Figure 4 , we observe that for categories PCB 1, PCB 2, PCB 3, and PCB 4, the baseline fails to rigorously repair the anomaly: the orientation of the board is reversed for PCB 1; unintended white marks appear on the lower left of PCB 2; the dark lines in the potentiometer change for PCB 3; and the "FC-75" label is missing for PCB 4. However, by integrating formal property guidance, our approach accurately reconstructs all these details while effectively removing the anomalies. For the MVTec-AD examples, AR-Pro produces repairs that more closely resemble the original inputs compared to those generated by the baseline. This demonstrates that our method's generated repairs adhere more rigorously to the formal properties.In Figure 6 , we present examples from our time-series repair, where the anomalous time segment is shaded in red. Our results demonstrate that AR-Pro generates a signal that resembles the original signal better, as shown in the first two plots of Figure 6 . In addition, we recover the sensor time series to non-anomalous values when the baseline fails to repair the anomaly, as shown in the last two plots of Figure 6 .More repair examples are available in Appendix D. However, although the quality of the generated repairs has improved, we notice that this enhancement comes with a trade-off of increased inference time. Further details can be found in Appendix E.We randomly sampled 100 instances to compute the mean of each metric in order to evaluate the effect of hyper-parameters λ 1 , λ 2 , λ 3 , λ 4 associated with each property-based loss. Each line in the plots represents results obtained while keeping the other hyper-parameters at 1.0. The ablation results for the time series are presented in Figure 7 , with additional plots in Appendix F. Our observations indicate that variations in the scale of property hyperparameters do not significantly impact the formal metrics, as the range of change remains relatively small. In addition, no consistent trends were observed when varying the λ 1 , λ 2 , λ 3 , λ 4 hyper-parameters. This suggests that our framework demonstrates robust performance, and extensive tuning may be unnecessary.Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?Answer: [Yes] Justification: We attach the code for reproducing the result to the zip file. 
related work: 
 Numerous techniques have been developed for anomaly detection across various domains [3, 49] .Traditional approaches to anomaly detection include clustering [43, 57] and statistical methods [34] such as ARIMA [41] and Gaussian models [52] . However, these methods often struggle with high-dimensional data. More recently, deep learning-based anomaly detection [20, 28] , including autoencoders [27] and GANs [18] , can detect high-dimensional anomalies via reconstruction error. For time-series data, LSTMs [32] and transformer-based models [60, 63, 69, 72] have shown exceptional performance. Additionally, diffusion models are emerging as promising tools for visual anomaly detection [42, 71] . While these methods vary in strengths and are continually improving, explaining anomalies remains a challenge [10] . Most current methods rely on feature importance scores or visualizations [44] , such as gradients or reconstructions [46] , which often fail to provide actionable insights. The lack of formal frameworks [65] and consistent evaluation metrics [1] complicates this issue. For example, the absence of formal metrics leads to inconsistencies in evaluation [26] , underscoring the need for more rigorous approaches and reliable criteria [37] . Most similar to our work is [56] , which also performs time series-specific generation of counterfactual explanations in the form of anomaly repairs but considers a different set of properties and does not use diffusion.Our work also focuses on generative modeling to produce counterfactual explanations in the form of anomaly repairs. For vision, the current leading paradigms are diffusion models [24] and generative adversarial networks [23] . Diffusion models are also applicable to time-series data [67] , and we refer to [22] for a survey on other techniques. 
discussion: 
 The main theoretical contribution of our work is the identification of common counterfactual explanation desiderata for linearly decomposable anomaly detectors. While we have identified four formal properties, we acknowledge that other valid ones may also exist. Moreover, we recommend that practitioners evaluate and choose the properties necessary for the particular problem, and this is made possible by the form of our diffusion guidance function in (7) . The quality of anomaly repairs depends on the performance of the anomaly detector and the generative model. While there may have been limitations in our efforts, we found it challenging to use variational auto-encoders [27] for generating high-quality repairs. Furthermore, our implementation is focused on diffusion models, but the ideas presented can also be extended to other generative techniques. 
conclusion: 
 We present AR-Pro, a framework for generating and evaluating counterfactual explanations in anomaly detection. We use the fact that common anomaly detectors are linearly decomposable, which lets us define formal, general, domain-independent properties for explainability. Using these properties, we show how to generate high-quality counterfactuals using a property-guided diffusion setup. We demonstrate the effectiveness of AR-Pro on vision and time-series datasets and showcase our improvement over off-the-shelf diffusion models.Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments? Answer: [Yes] Justification: We reported the mean and standard deviation for our main results in Table 1 .We also report TNR with 95% statistical significance in Table 3 . Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper. • The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).• The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors). • It should be clear whether the error bar is the standard deviation or the standard error the mean. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed? 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [Yes]Justification: We call for users adhere to usage guidelines of GPT2 in Appendix B or implementing safety filters.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes] Justification: We cited the original paper that produced the code packages and datasets. We also state which version of the model is used.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.• If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset's creators. 
new assets: 
 Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?Answer: [Yes] Justification: We communicate the details of the code as part of our submissions in the zip file. This includes details about training, license, limitations, etc.Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc. • The paper should discuss whether and how consent was obtained from people whose asset is used. • At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file. 
crowdsourcing and research with human subjects: 
 Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?Answer: [NA]Justification: The paper does not involve crowdsourcing nor research with human subjects.Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper. • According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector. 
institutional review board (irb) approvals or equivalent for research with human subjects: 
 Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?Answer: [NA]Justification: The paper does not involve crowdsourcing nor research with human subjects.Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.• We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: All experiments were done on a server with three NVIDIA GeForce RTX 4090 GPUs.In this section, we report the performance of anomaly detectors. The results for FastFlow [66] are presented in Table 4 , and the results for GPT-2 [48] are shown in Table 5 . We did not perform extensive parameter tuning, as the performance of anomaly detectors is not the primary focus of our work. In addition, it is recommended that users adhere to usage guidelines of using GPT-2 or implementing safety filters.We include additional model EfficientAD [11] and dataset MVTec [13] . The results are shown in Table 6 . EfficientAD achieved an 88.24% average improvement across the four metrics on MVTec-AD and 74.85% improvement on VisA.Additional VisA examples can be found in Figure 8 , while additional MVTec-AD examples can be found in Figure 9 and Figure 10 . Additional time-series examples can be found in Figure 11 .We randomly sampled 50 instances from the testing set to compare inference times, as shown in Table 7 . We observed that although property guidance generates higher-quality repairs, it results in slightly longer inference times for time series data and significantly longer times for image data, possibly due to the higher dimensionality of the inputs. In total, it took about 40 hours to finish RQ1 for VisA and 25 hours to finish SWaT. For RQ2, it took 5 hours for SWaT. Improving computation time will be a focus of our future work.In addition to λ 1 to λ 4 , we also perform ablation on the overall guidance scale, which we denote using λ ϕ . The The ablation study for λ ϕ on time series can be found in Figure 12 . The ablation for image data, using cashew class as an example can be found in Figure 13 and Figure 14 . 1. Claims Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? Answer: [Yes] Justification: The main claims made in the abstract and introduction accurately reflect the paper's contributions and scope, as they clearly outline the development of a domainindependent framework for anomaly based on formal properties and the demonstration of its effectiveness through empirical results on VisA and SWaT datasets. This alignment ensures that the reader's expectations are met and that the paper delivers on its proposed objectives.Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.• The answer NA means that the paper does include theoretical results.• All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced. • All assumptions should be clearly stated or referenced in the statement of any theorems.• The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition. • Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material. • Theorems and Lemmas that the proof relies upon should be properly referenced.• The answer NA means that the paper does not include experiments.• If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not. • If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable. • Depending on the contribution, reproducibility can be accomplished in various ways.For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed. • While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. , with an open-source dataset or instructions for how to construct the dataset). (d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results. Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material.Justification: There is no noticeable negative societal impact of the work performed.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.• Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 


Decision: This paper proposed a method to convert anomalous to normal images under a set of properties such as making minimal changes to normal regions and actually reducing anomalous scores. The method uses diffusion methods for image AD, and also applies to time series anomalies. Overall, the method addressed an interesting problems and shows promising results. While there were some concerns, they were addressed in the rebuttal.

Reviewer DUtN was concerned the paper does not detail which AD segmentation approaches this method will work for. The authors explained this will not work for clustering based AD. The AC suggests also mentioning retrieval based AD e.g. SPADE or PatchCore. In either case the AC believes this can be fixed in one or two sentences in the final manuscript.

Reviewers k8Af and F7R5 pointed out there is some redundancy between the 4 formal conditions. The authors admitted that it is possible to remove condition 1, but it makes the exposition more natural. This satisfied reviewer k8Af. The AC believes this is reasonable but requests adding a clarification to the final version. Reviewers k8Af wanted additional experiments, the reviewers supplied a few experiments and promised further experiments, this addressed the concern.

Reviewer F7R5 initially had several additional concerns about i) the motivation for anomaly repair ii) that anomalous images do not come with anomaly maps. AC believes the concerns should not prevent acceptance as: i) fixing anomalies is interesting both as a way of obtaining pristine images and as a tool for explaining model decisions, which is very important for anomaly detector as they are essentially unsupervised  ii) there are many anomaly segmentation methods available and it is not essential to introduce another one. The reviewer also indicated post-rebuttal that they do not object to accepting this paper.

Overall, given the interesting research direction that this paper offers, the promising initial results and that rebuttal addressed the concerns of the reviewers, the AC recommends acceptance.

Reviews: review: paper_summary: paper_summary: Paper proposes a method for anomaly repair that goes one step beyond an anomaly detection and/or an anomaly localization method. While anomaly detection focuses on identifying which objects (images, time series, etc.) are anomalous, and anomaly localization focuses on identifying regions within the object (image or time series) which is anomalous, anomaly repair focuses on producing the normal object that the anomalous object is derived from. Authors identify properties for the repair, and develop a generative model that can take an anomalous object and produces the corresponding normal (and repaired) object.

Paper describes experimental results to demonstrate the effectiveness of the proposed approach on different data sets. Evaluation is done both quantitatively and qualitatively.
strength_weakness: strength_weakness: A primary weakness of this paper is that it does not state the assumptions regarding the scope of the methods upfront. The analysis holds for methods that follow the reconstruction-based anomaly detection recipe, i.e., each input is reconstructed, and the anomaly score is calculated using the difference between the input and reconstruction. While that is true for many methods, there is still a vast majority of methods for which this is not applicable. In fact, even in the reconstruction based methods, the analysis holds for those in which the scoring function can be decomposed over the individual features. Again, this is not true for all reconstruction-based methods. It would be good if the authors can make this clear in the beginning to avoid confusion.
questions: questions: - In Section 2, the definition of anomaly map has a term $\hat{x}$, which has not been defined. From the figure 2 it appears that $\hat{x}$ is the reconstructed version of the original data point. Does that mean that this analysis framework is applicable only to the class of anomaly detection methods that involve reconstruction. While those methods are certainly capable, there is a large class of anomaly detection methods that do not necessarily have a reconstruction step involved. In fact, does the whole approach rely on availability of a base anomaly detector?
- Can this method be applied of multivariate instances (no spatial or temporal relationships). It is unclear how a single threshold ($\tau$) in line 106 be applied to a case where each feature could have a different scale.
- What is $\alpha_i(x)$ in Definition 2.1? Is it the absolute difference between the actual value and the reconstructed value for the $i^{th}$ feature?
- In Section 3.1, what does a "normal input" mean? Does it refer to an entire observation (an image) that does not have any anomalous features or does it refer to the non-anomalous parts of an image?
- In Figure 5(a), why do the authors infer that the proposed method is not adding a spurious signal, when the plot shows that it is different from the normal signal
limitations: limitations: I think the paper does not quite scope the method - identify what kind of problems will it work on and what it won't.
paper_summary: paper_summary: The paper proposes an anomaly repair technique. Based on proposed four
properties, it trains a generative model that can fix anomaly data to a benign
one. The proposed properties include similar to the data, etc., which are
globally applicable to any dataset. The evaluation is performed on VisA dataset
and the GPT2 on SWat Dataset. The results show that the proposed method can
effectively repair the anomaly data.
strength_weakness: strength_weakness: * I do not get the motivation of fixing anomaly data. If it is detected and
  considered as anomaly, our typical action is to determine if it is a false
  positive and then improve the detection model and following downstream models.
  What is the rationale of fixing the anomaly data?

* Properties 3 and 4 seem to be generalized version of 1 and 2. Typically, this
  should be solved in the training of models, either a vision model or
  time-series. Why not directly embedding these properties into the training of
  final model instead of training a separate model?

* The method assumes the availability of anomaly map, which seems to be
  impractical in real-world scenarios. How can the method be applied to
  real-world scenarios where anomaly maps are not available? In the real world,
  we tend to only have individual anomaly data points, not a map. Similarly, it
  would be great if you can extend the discussion to the region selector.
questions: questions: What is the motivation of fixing the anomaly data?
limitations: limitations: The paper does not explicitly discuss the limitations and potential negative. As
for me, my concern is this can be used as a tool to evade anomaly detection
models. The authors should discuss this in the paper.
paper_summary: paper_summary: The paper addresses the anomaly explanation task by repairing the normal appearances of input inputs. Specifically, this paper designs four properties to guide the repair process that works in both image and time series domains. To demonstrate the effectiveness of the proposed method, this paper conducts experiments on the VisA and SWaT datasets.
strength_weakness: strength_weakness: This paper exploits fixed anomaly detection methods, i.e., Fastflow and GPT2, to guide the anomaly repair process. The two selected methods are out-of-date, and the influence of the utilized anomaly detection methods should also be investigated. 

More datasets should be included but not only VisA and SWaT. For example, for industrial image anomaly detection, MVTec AD should also be included. 

For me, Property 1 is a combination of Property 3 and 4, and maybe only Property 2, 3, and 4 are enough. 

The figure quality should be improved, especially for Figure 3.

For the fourth metric M_{1-w}, should it be the absolute value? Or it will encourage lower anomaly scores for normal regions in the fixed results. 

Sec 3.3 is quite important but is not clear. The author may condense Sec. 2 a lot and then extend Sec 3.3, at least elaborating on the details of the generation process and adding proper references.


------------post rebuttal response--------
the authors have addressed all my concerns, so I raise my rate from 5 to 6.
questions: questions: See the Weaknesses.
limitations: limitations: See the Weaknesses.

Comments: title: Explanation of why encoding as training objectives fail
Follow up
Response to Reviewer k8Af
Response
comment: Many thanks for your clarifications. I stand by my positive rating.
Thank you for your response. We found that augmenting the training loss often failed to produce effective counterfactuals, either due to limitations in the model architecture or because of the model's natural training loss.

After many attempts, we discovered that guided diffusion was a straightforward algorithm capable of achieving good performance. Our eventual choice of algorithm was primarily influenced by the available generative models for images (VAEs, GANs, Diffusion), and we elaborate on the challenges associated with each below.

We initially tried VAEs for image repair but found the generated counterfactuals were often blurry, which is a known limitation of VAE [1,2]. Although later work describes methods for high-resolution VAE-based image generation [3,4], these methods often fell short in producing the same level of sharpness and detail compared to GANs and diffusion models—especially within the anomalous regions. So why are VAEs commonly used for reconstruction-based anomaly detection despite the blurry outputs? This is likely because although their ELBO loss tends to favor images that are the "average" of a distribution (i.e., blurry), this often suffices for detecting anomalous regions.

Given the limitations of VAEs, we turned to GANs and diffusion models. However, GAN training was plagued by well-known issues of instability and non-convergence [5,6], thereby leading us to focus on diffusion models.

With diffusion models, we found that incorporating the four properties into the training loss did not enhance counterfactual quality. This was likely because the diffusion model's noise prediction loss often led to static-like reconstructions (very noisy images) that were unsuitable for evaluation against our properties, especially at larger time steps. It was with guided diffusion methods [7] that we achieved high-quality counterfactuals, and these methods now form the foundation of our present methodology.

We thank the reviewer for these questions, and we will update our manuscript to include a discussion of the lessons learned with various model architectures.

**Additional References**

[1] Tomczak, Jakub, and Max Welling. "VAE with a VampPrior." In International conference on artificial intelligence and statistics, pp. 1214-1223. PMLR, 2018.

[2] Dai, Bin, and David Wipf. "Diagnosing and enhancing VAE models." arXiv preprint arXiv:1903.05789 (2019).

[3] Razavi, Ali, Aaron Van den Oord, and Oriol Vinyals. "Generating diverse high-fidelity images with vq-vae-2." Advances in neural information processing systems 32 (2019).

[4] Liu, Zhi-Song, Wan-Chi Siu, and Yui-Lam Chan. "Photo-realistic image super-resolution via variational autoencoders." IEEE Transactions on Circuits and Systems for video Technology 31, no. 4 (2020): 1351-1365.

[5] Saxena, Divya, and Jiannong Cao. "Generative adversarial networks (GANs) challenges, solutions, and future directions." ACM Computing Surveys (CSUR) 54, no. 3 (2021): 1-42.

[6] Lu, Yuzhen, Dong Chen, Ebenezer Olaniyi, and Yanbo Huang. "Generative adversarial networks (GANs) for image augmentation in agriculture: A systematic review." Computers and Electronics in Agriculture 200 (2022): 107208.

[7] Dhariwal, Prafulla, and Alexander Nichol. "Diffusion models beat gans on image synthesis." Advances in neural information processing systems 34 (2021): 8780-8794.
Could you explain why encoding the properties as training objectives fail? Thanks.
We thank the reviewer for their encouragement and dataset suggestions. We will try to incorporate MVTec LOCO into our paper. Meanwhile, if the reviewer has any additional comments, questions, or requests, please do let us know.
Dear Authors,

Thank you so much for your efforts. Explaining anomalies by repairing them is an interesting direction, and your responses have addressed my concerns.  I would suggest trying to repair semantic anomalies, like those anomalies in MVTec LOCO in the future since the structural anomalies are typically easy for users to understand the reason to be anomalous, but in comparison, users can suffer from explaining semantic anomalies without proper prior information.