Interpretable Out-of-Distribution Detection using Pattern Identification
Authors: Romain Xu-Darme
Julien Girard-Satabin
Darryl Hond
Gabriele Incorvaia
Zakaria Chihani
Venue: Submitted to ICLR 2023 (2023)
Keywords: out-of-distribution detection
pattern detection
interpretable artificial intelligence
confidence
metric

Abstract: Out-of-distribution (OoD) detection for data-based programs is a goal of paramount importance. Common approaches in the literature tend to train binary classifiers requiring inside-of-distribution (IoD) and OoD validation samples, and/or implement confidence metrics that are often abstract and therefore difficult to interpret. In this work, we propose to use the PARTICUL pattern identification algorithm in order to build more interpretable and robust OoD detectors for visual classifiers. Crucially, this approach does not require retraining the classifier and is tuned directly to the IoD dataset, making it applicable to domains where OoD does not have a clear definition. Moreover, pattern identification allows us to provide images from the IoD dataset as reference points to better explain our confidence scores. We illustrate the generalization abilities of our approach through an extensive benchmark across four datasets and two definitions of OoD. Our experiments show that the robustness of all metrics under test does not solely depend on the nature of the IoD dataset or the OoD definition, but also on the architecture of the classifier, which stresses the need for thorough experimentations for future work in OoD detection.

Under review as a conference paper at ICLR 2023
INTERPRETABLE OUT-OF-DISTRIBUTION DETECTION USING PATTERN IDENTIFICATION
Anonymous authors Paper under double-blind review
ABSTRACT
Out-of-distribution (OoD) detection for data-based programs is goal of paramount importance. Common approaches the literature tend to train detectors requiring inside-of-distriuutie (in-distribution, or IoD) and OoD validation samples, and/or implement confidence metrics that are often abstract and therefore difficult to interpret. In this work, we propose to use existing work from the field of explainable AI, namely the PARTICUL pattern identification algorithm, in order to build more interpretable and robust OoD detectors for visual classifiers. Crucially, this approach does not require to retrain the classifier and is tuned directly to the IoD dataset, making applicable domains where OoD does have clear definition. Moreover, pattern identification allows us to provide images from the IoD dataset as reference points better explain the confidence scores. We demonstrates that the detection capabilities of this approach are on par with existing methods through an extensive benchmark across four datasets and two definitions of Ool. In particular, we introduce new benchmark based perturbations of the IoD dataset which provides known and quantifiable evaluation the discrepancy between the IoD and OoD datasets that serves reference value for the comparison between various OoD detection methods. Our experiments show that the robustness of all metries under test does not solely depend the nature of the IoD dataset the OoD definition, but also on the architecture of the classifier, which stresses the need for thorough experimentations for future work OoD detection.
INTRODUCTION
fundamental aspect of software safety is arguably the modelling of its expected operational domain through formal semi-forma specification, giving clear boundaries when it is sensible to deploy the program, and when it is not. learning programs, especially for visual classifiers based on artificial neural networks (ANN), which are the subject of this paper. Indeed, such programs process high-dimensional data (images, videos) and are the result of complex optimization procedure, but they do not embed clear failure modes that could get trigged in the case of an unknown distribution, with potentially dire consequences in critical applications.
however difficult to define such boundaries for machine
Although it is difficult to characterize operational distribution, one could still measure its dissimi- larity with other distributions. this context, Out-of-Distriune (IoD) outside of it serves several
(OoD) detection which aims to
detect whether an input of an ANN is purposes. It helps characterize the extent to which the ANN can operate outside a bounded dataset (which is important due to the incompleteness of the training set w.r.f. the operational domain). It also constitutes a surrogate measure of the generalization abilities of the ANN. Finally, OoD detection can help assess when an input is too far away from the operational domain, which prevents misuses of the program and increases its safety.
2 RELATED WORK AND CONTRIBUTION
Out-of-distribution detection The maximum class probability (MCP) obtained after softmax nor- malization of the classifier logits already constitutes a good baseline for OoD detection Hendrycks Gimpel (2017). However, neural networks tend to be overconfident their predictions Szegedy et al.

---

Under review as a conference paper at ICLR 2023
(2014); Lee et al. (2018a); Hein et al. (2019), even when they are wrong, which may result in false claims of confidence. Hence, the development of enhancements like temperature scaling Liang et al. (2017), ensemble learning Nguyen et al. (2020) True- ClassCass Probability learning Corbiere et al. (2021) (assuming such information is known). Other types of confidence measures have also been developed, with various operational settings. Note that in this work, we focus on methods that can apply to pre-trained classifiers. Therefore, we exclude methods such as Lee et al. (2018a); Hein et al. (2019); Hendrycks al. (2019) - which integrate the learning of the confidence measure within the training objective of the model or specific architectures from the field of Bayesian Deep-Learnig that aim at capturing uncertainty by design Gal & Ghahramani (2016). While efficient, these ap- proaches may prove costly or impractical in an industrial context where lot of resources might have already been dedicated to obtain an accurate model for the task at hand.
Moreover, we make a distinction between methods that require validation set composed of Ool samples for the calibration of validation set and are therefore "OoD-agnosti" (Liu et al. (2020)).
(OoD-specific), and methods that do not require such
OoD-specific methods. ODIN Liang et al. (2018) extends the effect of temperature scaling with the use of small adversarial perturbations, applied on the input sample, that aim at increasing the maximum softmax score. OoD detection is performed by measuring the gain in softmax score after calibrating the temperature value and the perturbation intensity on validation set, that perturbations lead to a greater margin for loD data than for OoD data. Other approaches also attempt to capture the "normal" behaviour of the different layers of the classifier: Huang al. (2021) states that the latent representations of OoD samples through a CNN classifier are clustered around a point called the feature-space singularity (FSS) which serves as reference point for OoD detection. Similarly, Lee et al. (2018) 8b) proposes a confidence score based on the Mahalanobis distance between a new sample and class conditional Gaussian distributions inferred from the training set. Both approaches operate upon multiple layers of the network and require the use of set of OoD samples for calibrating the relative importance of each layer in the final confidence score.
OoD-agnostic methods. Liu et al. (2020) proposes framework based on energy scores (using in practice the denominator of the softmax normalization function) which can be used either during inference on a pre-trained model or to fine-tuned the model for more discriminative properties. The Fractional Neuron Region Distance Hond et al. (2021) (FNRD) computes the range of activations for each neuron on the training set, then provides score describing how many neurons are activated outside their boundaries for a given input. Finally, Attribution-Bant -ased Confidence Jha et al. (2019) (ABC) does not requires access to IoD data and equates the confidence of the network to its local stability by sampling the neighbourhood of a given input and measuring stability through attribution methods. Although this last method relies on fewer prerequisites, is computationally expensive (e.g., gradient computation) and may not be suited to runtime constraints.
As noted by Hendrycks et al. (2019), OoD-specific and OoD-agnostie methods are usually "not directly comparable" due to their different prerequisites. Therefore, in this work, we mainly compare DoD-agnottic methods, using FSSD Huang et al. (2021) only as a reference measure to illustrate possible detection gaps between OoD-agnostic and OoD- OoD-specific methods. We also exclude ABC Jha et al. (2019) from our experiments due to its computational cost.
It is also important to note that all the methods presented above are evaluated on different datasets and definitions of OoD (e.g., different datasets, distribution shifts), which only gives a partial picture on their robustness Tajwar et al. (2021). Although works such Open-OoD Yang et al. (2022) which aims at standardizing the evaluation of OoD detection, anomaly detection and open-set recognition into unified benchmark are invaluable for the community, most datasets commonly in use (MNIST Deng (2012), SEFAR-10/100 Krizhevsky (2009)) contain images with low resolution that may not reflect the detection capabilities of the methods under test in more realistic operational settings. Moreover, when evaluating the ability of an OoD detection method to discriminate between IoD and OoD datasets, it is often difficult to properly quantify the discrepancy between these two datasets, independently from the method under test, and therefore to exhibit a "ground truth" value of what this margin should be. Therefore, in this paper we propose a new type of OoD benchmark based on perturbations of the IoD dataset which aims at measuring the correlation between the OoD detection score of a given method on the perturbed dataset (OoD) and the intensity of the perturbation, under the hypothesis that the intensity of the perturbation can serve as a ground-truth measure of the discrepancy between the LoD and the OoD dataset.
2

---

Under review as a conference paper at ICLR 2023
Part detection Many object recognition methods have focused on part detection, in supervised (using annotations Farhadi ct al. (2009); Han et al. (2018); Zhao et al. (2019)), weakly-supervised (using class labels Li et al. (2020); Peng al. (2018)) or unsupervised settings (Zheng et al. (2017); Han et al. (2022); Xu-Darme et al. (2022)), primarily with the goal of improving accuracy on hard classification tasks such as fine-grained recognition Welinder al. (2010); Yang al. (2015). To our knowledge, the PARTICUL algorithm described in Xu-Darme et al. (2022) is the only method including a confidence measure associated with the detected parts (in the paper, it used to infer the visibility of given part). PARTICUL aims at identifying recurring patterns in the latent representation of a set of images processed through a CNN, in an unsupervised manner. It is however restricted to fine- mraiinde recognition datasets where all images belong to the same macro-category.
Our contributions In this paper, we aim to investigate the following research tracks: 1) How well can OoD detection methods generalize across multiple definitions of Ool?? 2) How to improve the interpretability of confidence measures in the context of visual classifier?? As an answer to those questions, we make the following contributions:
1. We show that recent work on pattern identification (Xu-Darme al. (2022)) can be adapted to build an OoD detection measure that does not require any fine-tuning of the original classifier, and with performance on par with other OoD-agnostic methods. More importantly, pattern identification allows us to provide images from the IoD dataset as reference points to better explain the confidence scores;
2. We demonstrate the ability of our approach to consistently capture various notions of OoD over an extensive benchmark comprising four datasets, two architectures and several evaluation metrics. In particular, we introduce a new benchmark based on perturbations of the IoD dataset which provides a known and quantifiable evaluation of the discrepancy between the IoD and OoD datasets that serves a reference value for the comparison between various OoD detection methods.
This paper is organized as follows: Sec. 3 formulates the problem and the two OoD modalities we will explore: Sec. 4 introduces our modifications to the PARTICUL algorithm with the goal of providing more interpretable and robust confidence measure: See. 5 describes our experimental setup and results. Finally, Sec. 6 presents our conclusions and future work.
PROBLEM FORMULATION
Let I2 be the space of all RGB images of size Let Dsod C I, be dataset approximating the distribution Djod of images belonging to N categories. Let M be a deep convolutional neural network (CNN) classifier trained on subset of Diod, Xtrain- For any image E Ig, M outputs vector of logits M(x) E R.," where the index of the highest value corresponds to the most probable category (or class) of x relatively all other categories. Since any image is assigned a category regardless of its similarity to the training distribution, (xx is often normalized using a softmax function in order to indicate a level of confidence in the prediction. More generally, a confidence measure Cm I, + [0, 1] is function assigning a score to each image x E I, processed through M. We denote (Dint) the distribution of values computed over Diod using the confidence measure CM- As indicated above, CM (Dood) approximated by (P(oooa) (Dod) = {C)((T) x E Diod) } Ideally, images belonging to the training distribution Diod should have a higher confidence score than images outside Diod- We propose to take an empirical approach, which will only attempt to provide evidence that a confidence measure tends to return greater score for images in D iod than images outside Diod However, it not possible to obtain a dataset representative of I, \ Diods i.e., encompassing all possible OoD inputs. Therefore, in this work we use two weaker modalities of OoD detection that serve as benchmarks for assessing the various proposed confidence measures.
Cross-dataset detection In this experiment, as in most related works, we evaluate the separability of the distributions Cm (Dioa) and CM where Dood is a dataset different (not drawn from the same distribution) from Diod- We use three complementary metrics that do not require determination of an acceptance threshold between IoD and OoD inputs, and that are suitable for comparing the different confidence measures proposed: the Area Under the ROC curve (AUROC); the Area Under the Precision-Rece Recall curve (AUPR). As in Huang et al. (2021), we also compute the False Positive Rate when the true positive rate is 80% (FPR80).
3

---

Under review as a conference paper at ICLR 2023
Sensitivity to dataset perturbation For this second experiment, we generate an OoD dataset Dood by applying a perturbation to all images of Dsad- More precisely, a perturbation Px is a function I, with variable magnitude À (e.g., for rotation, À
which applies transformation to image might correspond to the angle of the rotation). Note that there exist no general value for the magnitude of the perturbation that would result in Dood be OoD in all cases. In this paper, the maximum perturbation amplitude is chosen when the resulting image is qualitatively considered dissimilar to the base image. For a perturbation PA applied over Diod we define the average value of confidence measure CMd as
T(CM, Doodi P.
1
Σ (M(()(())
|Diod
REDiod
which is extended set of perturbation magnitudes A (An, (An....,AA) ,n) as T(CM, Diod, P) (T(CMs Diod, P. separability of LoD and OoD confidence distributions, perturbations of small intensities would result in almost identical distributions. Instead, evaluate the correlation between the intensity of the pertur- bation and the average confidence value of the perturbed dataset by computing the Spearman Rank cor- relation coefficient rs between A and T(CM, Dind; P, A))- Indeed, r's (A,T(CM, Drad, P, P.A)) = (resp. 1) indicates that the average confidence measure increases (resp. decreases) monotonically with the value of A, which indicates that the measure is least correlated with an increase in the perturbation. The use of the Spearman Rank correlation score, rather than another qualitative metric such as the average value of the derivative AT' is motivated by the following reasons: 1) Averaging the derivative across multiple intensity values might hide the fact that there is no correlation between the measure and the perturbation; 2) A higher sensitivity of a given measure to a perturbation would not necessarily imply better measure, especially when the different confidence measures under test might have different calibrations. Therefore, under the hypothesis that the discrepancy between Diod and D ood is correlated to the magnitude of the perturbation X, this benchmark measures the consistency of the OoD method under test.
Dind, P, X,,)). Although it would again be possible to measure the
8A
4 PROPOSED MODEL
In this section, we present two proposals for building confidence measure based the PARTICUL algorithm, as described in Xu-Darme et al. (2022). PARTICUL is intented to mine recurring patterns in the latent representation of set of images processed through CNN. In the context of this paper, patterns are learnt from the last convolutional layer of a classifier M over the training set Diod- in a plug-in fashion that does not require the classifier to be retrained. Let F be the restriction of classifier M up to its last convolutional layer, i.e., M = L o F, where L corresponds to the last pooling layer followed by one or several fully connected layers. Vx E Iz, F(x) #xx is a convolutional map of D-dimensional vectors.
The purpose of the PARTICUL algorithm is to learn p distinct 1 X 1 X D convolutional kernels K = ",.,,,..,, ](r) (or detectors), such that Vr E Xtrains each kernel k(()) strongly correlates with exactly one vector Fin, (x) in F(x) (Locality constraint). After the training of said kernels, PARTICUL uses the function H (6) (x) max (F),,)) +k())) returning the h,,
Vanilla PARTICUL
maximum correlation score between kernel ((0) and the convolutional map F(z). In Xu-Darme et al. (2022), the distribution H (i) (Dioa) is modeled random variable following normal distribution N(nl0 (()) and estimated (or calibruted) over Xerain- Then, the corresponding confidence measure is computed using the cumulative distribution function (CDF) associated with N (1((), ( However, such CDF does not have a closed-fome solution, which can slow down operations on tensors. Therefore, this work, we choose to model (+) (Diod) as a logistic distribution ((u("), o()),, where f(()) and (f)) are again estimated over Xtrain- and define the confidence associated with detector i for image as:
(())=
(2)
g0) (() ob)
1 1te
1 F
Finally, we average the confidence measures and define, Vx E I, Cy (x) = Σ (())..
p [=)
4

---

Under review as a conference paper at ICLR 2023
Classifier
M(x)
argmax
Class
Cyix)
C,(x)
C.x)-Cu
C.
Z
Class
Cclk)
Class-based PARTICUL overview. When processing a new sample, the confidence measure averages the contribution of the detectors corresponding to the predicted class only.
Figure I:
PARTICUL is well-suited for datasets with low intra-set variance i.e,, when images Xtroin are very similar which is the case for fine-grained recognition datasets (Welinder et al. (2010); Yang et al. (2015)). However, for more heterogeneous datasets (Krizhevsky (2009); Li et al. (2022)), becomes more difficult find recurring patterns that are present across the entire training Moreover, in order to learn more consistent patterns, might be beneficial to learn detectors that are specific to each of the N classes of Diod-
Class-based PARTICUL Here, we introduce a variant of PARTICUL that uses the labels of the training set Xtrain in order to learn p detectors per class. During the training and calibration phases, for each pair of images and class label (xr, c) E Xtrain x N, only the detectors of class e are modified. k!!'1 be the set of kernel detectors for class c. As in Xu-Darme et al. (2022), K2 is learned by minimizing the weighted combination of locality function Li and an unicity function Lu. L(K.) = ((( + AL. (K,, restricted to Xiraen,c = [x] c) E Xtrain). Importantly, since patterns may be similar across different classes (e.g., the wheels on car or bus), we do not treat images from other classes as negative samples when learning our bassd detectors. Therefore, although this approach may allow more specific detectors to be learnt than the vanilla-PARTICUL
More precisely, let K = TRL"
approach, effectively restricts the number of training samples per detector to the images of single class, which is potentially detrimental to the PARTICUL training process. The calibration of the detectors for class also uses a logistic distribution model, which is estimated training samples from class only. Then, as shown in Fig. for any image I, and with no access to the ground truth label of the confidence measure C (æ) is obtained by averaging the confidence scores from the detectors associated with the predicted class argmax (x)) of
1 c00 (x).
image x, i.e. C
As consequence, for an image I E Diod, the quality of our class-based confidence measure is also dependent on the ability of the classifier to assign the correct class to I, i.e., on the accuracy of M.
5 EXPERIMENTS
Setup We performed our evaluation using a ResNet50 He et al. (2016) classifier trained on various datasets, with different image sizes: the Caltech- Li al. (2022) dataset contains 9, 44 images from 101 different categories- that are resized to 224 x 224 during training (test accuracy 86.8%); the CUB-200 Welinder et al. (2010) dataset contains 11,788 images from 200 different bird species that are resized to 448 x 448 during training (test accuracy 83.6%); the Stanford Cars Yang et al. (2015) dataset contains 16, 185 images from 196 different car models that are resized to 448 x 448 (test accuracy 90.5%). Note that additional experiments were conducted on the CIFAR- 100 Krizhevsky (2009) dataset but are not detailed here (see the supplementary material) due to their similarity with the results obtained on Caltech-101. Both CUB-200 and Stanford Cars are fineggrained classification datasets and contain very homogeneous data from a single macro- macro-category. while Caltech- 101 contains more heterogeneous data from diverse categories. Each dataset is split between a training set Xtrain (also used for PARTICUL calibration) and a test set Xtest (used for evaluation). We follow the dataset providers' proposed splits. Using the trained classifiers for feature
5

---

Under review as a conference paper at ICLR 2023
extraction, we implemented our PARTICUL detectors using the two approaches described above: for vanilla PARTICUL, we trained 4 or 6 global detectors, ignoring image labels and using the loss functions (C, and L.) and learning parameters indicated in Xu-Darme et al. (2022); for class-based PARTICUL, we trained 4 or detectors per class, using the image labels of the training set and the same loss functions/learning parameters, but per-class basis. Note that although this approach leads to a high number of detectors, the learning process remains computationally efficient since the weights of the classifier are not modified and only the detectors of the expected class are updated during the back propaaaooon phase. As a comparison, we also implemented a confidence measure JN (D(x)) based on Hond et al. (2021) (INRD), as well as the measure O presented in Huang et al. (2021) (FSSD). We recall that FSSD is a OoD-specific method as it uses a validation OoD dataset D ood, vol in order to learn linear regressor with both positive and negative samples. In Huang et al. (2021), the authors choose Daad ~ Doods effectively training a binary classifier between Diod and Dood- For fair comparison between methods, experiments using different validation OoD datasets to perform FSSD calibration. Finally, since the output of the classifier can also constitute a valid confidence measure, we also compare our results with the maximum class probability CMCP (x) = max (softmar ( (x))) and the energy-based
furdd (z)
perform several
CM
(EB) method of Liu et al. (2020) B (x)
log( eMq(x)).
OoD-specific
OoD-agnostic vamilla PARTICUI.
Doud
Metrics
FSSD (Duod.
CT
CB
SC
MCP
FB
INRD
claxs-hwved PARTICUL
P 4
Dio.d=
Caltech
101
(heterogenenus)
AUROCT
(95.0)
28.6
87.2
|83.0|
46.8
72.5 - 4.8
71.7 1.4
72.7
0.8
69.7
+
1.7
CB
AUPRT
(88.5)
12.1
77.5
70.5
17.7
41.0
47.3
1.6
38.6
23
33.9
2.4
HPR801
(04.5)
94.5
22.1
34.1
86.3
49.9
76
57.3
18
49.5
+
11
54.6
2.2
AUROCT
178.0)
85.3
47.3
10.5
41.0
67.1
SC
AUPR
15.3
(49.2)
71.8
67.1
44.8
19.9
8.6 16.6
3.5
24.0 +
1.0
24.4
FPR801
96.4
(45.2)
26.9
32.5
68.1
88.9
95.5 1.5
58.1
0.8
00.5
4.2
Diod
8200
thomogenoous)
AUROCT
(94.7)
94.5
96.1
94..0
72.1
91.8 = 3.3
93.7
- 0.4
93.6 +
0.3
2.0
AUPRT
(98.3)
98.3
99.1
98.2
91.5
97.5 0.8
97.7 0.2
98.3 - 0.1
98.
0.6
EPR801
(7.0)
6.4
5.1
8.5
52.8
12.9
- 5.7
9.5
+ 1.3
9.4
1.6
12.8
+ 5.0
AUROCT
98.6
(99.3)
98.8
97.1
88.3
97.9
18
99.1
10.1
94.4
3.0
97.5
1.6
1
98.6
(98.9)
98.6
96.9
88.8
97.3
1.9
10.1 98.6
93.5
+ 4.6
96.0
2.0
FPR801
0.4
(0.5)
0.4
1.5
18.0
1.9
1 1.5
0.9
1: 0.1
7.9 1
8.5
4.9
1:
4.6
Diod
Stanford Cars (homogencous)
AUROCT
(99.9)
99.1
96.7
98.9
45.5
97.4 1.9
98.4
0.6
86.8 +
94.0
CT
AUPRT
(100)
99.8
/
99.4
99.7
83.1
99.5 0.4
99.7
+0.1
97.5 = 0.4
98.9 0.7
bPR801
(0.1)
1.9
2.9
1.8
0.7
21.0
AUROCT
100
(99.9)
97.6
99.6
51.8
90.6
0.5
99.8
10.2
95.4 1 U.8
97.0
CB
AUPRT
100
(99.9)
/
98.0
99.7
56.6
99.7
0.3
99.9 +0.1
96.7 + 0.7
97.9
0.7
FPR801
3.7
0.4
72.5
0.1
1
0.1
0.0
10.1
7.2 2.9 2.7 J: 0.9
Table 1: AUROC1, AUPR and FPRS0] scores (0-100 scale) for different pairs (Dood, Dood). SC denotes StanfordCars, CB denotes CUB200 and CT denotes Caltech101. The thick vertical line separates FSSD (OoD- specific) from all other agnostic methods. For FSSD scores, we indicate the validation dataset D Dood,vat used during the calibration, and scores in parenthesis correspond to the optimal case Dood,eat Dood. For OoD- agnostic approaches, we indicate bold the best performing measure for each experiment and each metric.
out-of distiibuuton In this section, we evaluate the ability of various confidence measures to distinguish the dataset Diod from another dataset Dood # Diod- As shown in Table and illustrated in Fig. 2, the ability to discriminate Diod from Dood widely differs, depending on the choice of measure and the datasets themselves. In particular, when used as reference distributions, the homogeneous datasets CUB200 and Stanford Cars are in general easier to discriminate than the more heterogenous dataset Caltech- across all measures except Cfardd This is consistent with the intuition that Caltech101 distribution covers a broader area of the latent space and is therefore harder to circumscribe. As expected, when using D Dood,val ~ DD Dood, Cfaad becomes a binary classifier with fully-supervised training and displays a high-level of discrimination. However, when using Dood, 7f Dood, the ability of this confidence measure to generalize to unseen datasets is more questionable. Indeed, for Caltech101 v. CUB200 and Caltech101 v. Stanford Cars, we obtain results which vary markedly depending on the choice of validation dataset. Surprisingly, the confidence measure CMCP shows good discriminative properties for most pairs (Diod. Dood). Without contradicting the conclusions of Hein al. (2019), this indicates that outside adversarial

---

Under review as a conference paper at ICLR 2023
Ukstributions
CU0220 HOC CUTVE
Cattehh01 (D6a)
14
StantordCars (OoD)
CUB200 (Ir))
Thus positive rate
Danstty
6/
62
Caltecha01
StanfordCars
60"
Confidence
False positive
Figure 2:
Example of distributions of confidence scores (left) and corresponding ROC curves (right) for a vanilla PARTICUL model (6 detectors) trained on CUB200 and tested against Caltech101 and Stanford Cars. Best viewed in color.
setting using the normalized output logits of classifier is often enough to distinguish OoD samples. Moreover, notice that, except for Diod =StanfordCars, the energy-basee method C&B does not exhibit better detection performance than the CMCP baseline. This experiment also reveals that in general the measure Cf is poor discriminator of OoD datasets. Both these results may be explained by our choice of datasets which differs from the experiments conducted in these papers, and also emphasize the sensitivity of OoD detection methods the experimental setting. Results of our MARTICUL-based confidence measure are contrasted and also depend on the choice of architecture (vanilla or class-based) with respect to the IoD dataset. For CUB200 and Stanford Cars as reference sets, not only does class-based PARTICUL architecture constitute good OoD detector (with AUROC scores greater than 90% 67% of the cases), but vanilla PARTICUL fully exploits the homogeneity of the loD dataset to train robust detectors, with AUROC scores greater than 90% in all cases, using fewer detectors compared to the class-based approach. For Caltech 101 as the reference set, class-based PARTICUL provides similar (CUB200) better (Stanford Cars) results than vanilla PARTICUL, but at the cost of greater number of detectors, and with an AUROC score lower than C CMCP. This is probably due the fact that Caltech 101 already contains images of birds and cars with similar size, making the distinction between distributions harder.
This first experiment confirms that our PARTICUL-based confidence measure is, in general, suited to perform OoD detection in a cross-dataset setting. For homogeneous training datasets, vanilla- PARTICUL shows detection capabilities on par or surpassing DoD-agnostic and ooo-speciiic (FSSD) approaches. For heterogeneous training datasets, our class-based PARTICUL approach improves upon vanilla-ARTICUUL by learning more specific detectors. Finally, it interesting to point out that our results are stable, with a standard deviation of the AUROC/AUPR//HS 10% in the vast majority of our experiments, and that the number of detectors (4 or 6) does not have a significant impact on the results.
scores of less than
Perturbation P
Description
Values
Sign
Blur
Gaussian blur with kernel 3 x 3, standard deviation À Gaussian noise with ratio
A= 0.0 (no blur), 0.I, 0.3, 0.5, 0.7, 1.0
2.0, 5.0, 10.0
Noise
A= 0.0 (no noise), 0.1, 0.2, 0.3, 0.4, 0.5
0.7, 1.0
Brightness
Blend black image with ratio
== 0.1, 0.3, 0.5, 0.6, 0.7, 0.8, 0.9,
1
1.0 (no change)
Rotation forth
Rotation with degree
x= (no rotation), 10, 20, 30, 40, 50.
70, 90, 110, 130, 150, 180
Rotation back
Rotation with degree À
A= 180, 210, 240. 270, 300, 320, 340,
350, 360 (no rotation)
Table 2: Description of the perturbations used in this paper. For each perturhation, we define the variable
parameter X. its range of values A and the expected sign of the correlation. "+" (resp. '-') indicates that the confidence measure is expected to increase (resp. decrease) with the intensity of the perturbation. For gaussian noise, we dynamically compute a standard deviation for each input æ as (xx) = x (max(x) min(x)) (similar to the method used in Smilkov et al. (2017)).
7

---

Under review as a conference paper at ICLR 2023
PARTICUL
class-based
PARTICOL
Arch.
Dind
Pert.
MCP
EB
INRD
p= 4
P 6
p=4
p= 6
CT
-1.00
-0.33
0991
0995
0.00
0,95
0.00
40.5
0.00
CB
Blur (-)
-0.90
0.61
1.00
0.95
= 0.00
0.95
4 0.00
0.95 0.00
1.00
0.00
SC
-1.00
0,96
.0.25
0.60
0.50
0.14
0.85
-0.86
0.03
CT
0.05
0.5
-1.00
-1.00
0.00
1.00000000000000000000000
-1.00
0.00
CB
Noise (-)
-0.24
0.12
-1.00
-0.69
0.00 +0.68
: 0.08
-0.78 0.10
-0.67
0.08
SC
-0.29
-0.29
-1.00
-0.77
0.03
-082
4.61 0.1
-0.93
0.12
0.98
1.00
-1.00
0.97
= 0.04
0.98 = 0.04
0.00
1.00
Resnet50
G
Brightness (+)
1.00
1.00
0.11
0,78
0.67 0.20
1.00
1.00
1,00
0.98 1 0.03
0.90 1 0.18
1.00 0.00
1.00 0.00
-0.30
-0.28
-0.41
0.18
.0.42
0.0
-0.1 = 0.08
.0.52 0.07
CB
Rotation forth
-0.95
-0.89
-0.42
-0.82
1:82:000
0.20
-0.94
0.01
4,94
= 0.01
-0.93 0.00
sc
-0.76
-0.76
-0,56
-0.71
0.06
+0.67 + 0.09
40,66 - 0.08
-0.89 0.08
0.52
0.65
0.20
0.51
0.09
0.44
0.60
0.03
CB
Rotation back (+)
0.95
0.90
0.32
0.77
= 0.32 0.94 = 0.01
0.98
0.03
0.98 0.03
SC
0.72
0.72
0.78
0.06
0.74 0.03
0.05
0.89
Blur (-)
-0.86
-1,00
1.00
-1.00
+::0::00 0.00
-1.00 +
0.00
-0.9
= 0.06
Noise (-)
-1.00
-1.00)
-1.00
-1.00
0.00
-1.00
1 0.00
-0.92
1: 0.06
-0.97
1 0.05
VGG19BN CB
Brightness (+)
0.98
1.00
0.98
1.00
0.00
1,00
1 0.00
1.00
1.00
1.00
1 0.00
Rotation forth (-)
-0.95
-0.94
-0.42
-0.95
4 0.00
-0.95
4 0.00
-0.55
- 0.00
-0.95
: 0.00
Rotation back (+)
0.95
0.93
0.94 0.01
0.92
0.92 0.01
0.92 0.01
Table 3: Spearman rank correlation coefficient between the intensity À of perturbation applied to dataset D and the average confidence measure [[Cx, D, P, A) computed over the dataset. For each perturbation P. the expected correlation sign is recalled in parenthesis. In red, we highlight experiments where the distributions are either uncorrelated (small correlation coefficient), inversely correlated (opposite coefficient sign) or unstable (high standard deviation for PARTICUL- measures). For CUB200, we also provide the results of the experiments performed using a VGG19 classifier (with batch normalization). Best viewed in color.
Sensitivity to dataset perturbation In this experiment, we study the behaviour of confidence measures using the various perturbations described in Table 2. Due to its instability during the previous experiment and its OoD-specific setting, Cfaadd is not considered here. As shown in Table 3, the four confidence measures under test exhibit very different behaviors depending on the nature of the perturbation, but also sometimes depending on the reference dataset. Indeed, although CM and C&B seem in general sensitive blur and brightness perturbations, they not correlated with gaussian noise, opening the door, if it were employed, to adversarial attacks (Szegedy et al. (2014); Hein et al. (2019)). Conversely, while Cf is strongly sensitive to noise, any perturbation reducing the amplitude of neuron activation values (blur, brightness) has the opposite effect of increasing this confidence measure. In general, both Cyf and C% correlate with each type perturbation. However, we notice that results on CUB200 (and Stanford Cars, in a lesser extent) for blur and brightness suffer the same issue as Crrrd (inverse correlation compared to the correct correlation) when using our baseline classifier (ResNet50). Finally, rotations seem harder to detect, with MCP, EB and PARTICUL-based measures achieving similar results, while Cffrrd is only weakly correlated the rotation angle. Interestingly, when using VGGI9BN (Simonyan & Zisserman (2015)) classifier trained on CUB200, we obtain better results for C CMCPIC (noise sensitivity) and our PARTICUL- based approach (blur, brightness), while confirming the drawbacks of Cxford (blur, brightness). This indicates that not only does the robustness of given measure depends on the type of perturbation and on the IoD dataset, but it also seems to rely on the architecture of the classifier.
A step towards more explainable confidence measure One of the key advantages of the PARTICUL-based confidence measure resides in the possibility for a human to visualize the detected patterns during the inference of the classifier in order to trace back the contribution of each detector. As in Xu-Darme al. (2022), we use the Smoothgrads (Smilkov al. (2017)) algorithm to highlight the most salient area of the image for each detector. As illustrated in Fig. 3a, images from Dind can be used to extract a visual reference of the pattern highlighted by each detector. When processing a new image through the classifier M, patterns can be compared to these visual references to better understand the confidence score. For example, some detectors trained on the StanfordCars dataset seem to be able to recognize similar patterns an image of motorcycle (row 3), which is to be expected from a visual point of view. Secondly, visualization can also help illustrate the behaviour of each detector w.E.t perturbation of the image, illustrated in Fig. 3b and Fig. 3c. In particular, it is interesting to notice that contrary to our initial intuition, our PARTICUL-based confidence measure does not steadily decrease for angles ranging from 0 180", but rather displays some invariance to
8

---

Under review as a conference paper at ICLR 2023
Cont: 24%
Cont: 0%
Cont: 0%
Cont: 4%
Cont: 35%
Cont: 65%
Conf: 59%
Conf: 95%
Cont: 77%
Cont: 72%
Canf: 37%
Conf: 14%
Cont: 8%
Cont: 15%
Conf: 25%
Cont: 13%
Cont: 10%
Cont: 11%
Conf: 58%
Cunf: 46%
Conf. 38%
Conf. 14%
Conf: 2%
Cont: 33%
Conf: 52%
Cont: 10%
Cont: 3%
Conf: 58%
Conf:
Conf:
Conf:
Conf: 9%
53%
35%
17%
Avg. 20%
Avg. 16%
(a) Pattern visualization and confidence scores of detectors (one per column) using vonilta PARTICUL StanfordCarrs applied to images from StanferCarrs (row 1). CUB200 (row 2) and Caltech101 first image can visual reference.
(b) Impact of increasing gaussian noise ratio on four detectors trained class-bused PARTICUL Caliech101 (motorbikes class. Averaging the confidence seures across all detectors creates ensemble effect rohustness.).
Average dataset confiddncee
0.4
Canf:
8% Conf. Gonf: 45%
0.2
Conf. 21% Conf: 23% Gonf: 19% Conf. 16% Conf: 11% Conf. 11%
angle
Cant.
54% Cont.
Conf. 53%
210
260
310
Evolution confidence score detectors
average confidence detector across the Stanford Cars datasct for various rotation angles. Rather than the expected V-shape curve, we observe local maxima for 180" rotation (see (c), 1).
PARTICUL.
fordCars (row 1), CUB200 (row 2) and Caltech101 (row 3) respectveeyy want. to rotation frow 1), changes in brightaess (row 2) and gaussian blur (row
Figure 3: Using pattern visualization to better understand our PARTICUL-based confidence measure.
horizontal symmetry (Fig.3d). Finally, shown in Fig. 3b, although each of our detectors might not be strongly sensitive to given perturbation, averaging scores across all detectors partially mitigates the variations of individual detectors and creates ensemble effect that improves the overall stability of the metric, without the cost of training several classifiers (Nguyen et al. (2020)).
CONCLUSION & FUTURE WORKS
In this paper, we have demonstrated how the detection recurring patterns, using the PARTICUL algorithm, be exploited develop confidence measures with the ability to discriminate between IoD and OoD inputs, while enabling a form of visualisation of the detected patterns. Our experiments showed that it offers mostly consistent results in the context of two different OoD detection bench- marks. In particular, we introduced a new OoD benchmark based on perturbations of the reference dataset with increasing magnitude which offers quantifiable evaluation of the discrepancy between IoD and OoD inputs. We also showed that the robustness of all tested metrics can depend on the architecture of the underlying classifier and the processed dataset. We believe that experiments with multiple architectures should be systematically conducted in future proposals for measuring Disrof--ittibbtion.. Moreover, since the datasets we used were not strictly exclusive (Caltech 101 contains images of both birds and cars), a proper ablation study (retraining classifier without these distributions actually improves.
classes) should be performed to verify whether the separability Finally, also wish extend our benchmark to more large scale dataset such as ImageNet Deng et al. (2009) and to study the possibility of applying our PARTICUL-basee confidence measure to other architectures such as vision transformers Kolesnikov al. (2021).

---

Under review as a conference paper at ICLR 2023
Reproducibility
All the code necessary reproduce experimental results is available the http::////...tt....uumisss.oo./.
following link:
Optimizer
Dataset
Architecture
Image size
Accuracy
# epochs
alg
learn. rate
decay/mom.
Caltech 101
86.8%
30
Adam
10
224
CIFAR 100
71.3%
30
Adam
10-3
Resnet50
CUB200 StanfordCars
83.6%
80
SGD
10
10 00.
448
90.5%
80
SGD
10-3
10-4
CUB200
VGG19BN
448
83.0%
100
SGD
10 10-3
10
10-9
Table 4: Baseline classifiers training parameters.
Table 4 summarizes the parameters used when training our baseline classifiers. Note that for CUB200 and Stanford Cars, perform data augmentation through random cropping of size 448 448 on the training images (for CIFAR100 and Caltech101, no data augmentation is used) and reduce the learning rate by factor 10 every 30 epochs.
For reproducibility purposes, we performed 3 random initializations of the PARTICUL detectors for both cross-datase and dataset perturbation OoD settings. We providing averaged scores along with corresponding unbiased standard deviation. In our tables, average score is on the left of the + symbol, standard deviation is the right.
When training our PARTICUL detectors (either the vanilla or class-based versions), we use the RMSprop optimizer, with learning rate of 10 4 and decay of 10-5, for epochs. We do use any data augmentation except resizing each image to the appropriate size. In all experiments, we try maximize the diversity of our detectors by setting the ratio of the Unicity constraint Xu-Darme et al. (2022)). The training process of the detector is in itself very fast: as an indication, when training Cmmilla-Particu detectors on the CUB200 dataset (images of size 448 448), we are able to process around 1000 images per min
(see
Quadro T2000 mobile.
For the implementation of FSSD, we used the code available on GitHub (tttss//gttub..commmeviii research/FSSDD (tt.///SSSSDDDDDeteeetnn) with the following modifications:
Allow the choice of a validation set Dood, vol H Dood for a fairer comparison betwen methods
Use the entire test sets during the evaluation rather than subsampling Dood match the size of Diod (would improve AUPR score in some cases).
Note that during training, we did perform hyperparameter tuning due its computational cost.
10

---

Under review as a conference paper at ICLR 2023
REFERENCES
Charles Corbiere, Nicolas Thome, Antoine Saporta, Tuan-Hung Vu, Matthicu Cord, and Patrick Perez. Confidence estimation via auxiliary models. IEEE Transactions Pattern Analysis and Machine Intelligence, pp. 1-1, 2021. doi: 10.1 1011091pamm2221.3385555 13005993.. 2
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large- scage-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 248-255. leee, 2009.
Li Deng. The mnist database of handwritten digit images for machine learning research. IEEE Signal Processing Magazine, 29(6):141-144 2012. 2
Farhadi, Ian Endres, Derek Hoiem, and David Forsyth. Describing Objects by their Attributes. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp.
2009. doi:
10.11099CVP..2009..00 5520079..
Yarin Gal and Zoubin Ghahramani. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In International Conference on Machine Learning, pp. 1050-1059,
PMLR, 2016. 2
Junwei Han, Xiwen Yao, Gong Cheng, Xiaoxu Feng, and Dong Xu. P-cnn: Part-based convolutional neural networks for fine-grained visual categorization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(2):579 2022. doi: 10.1 11110//TAM..201.993333
Kai Han, Jianyuan Guo, Chao Zhang, and Mingjian Zhu. Attribute-Aaan Attention Model for Fine-grained Representation Learning. Proceedings of the 26th ACM international conference on Multimedia, 2018. 3
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep Residual Learning for Image Recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770-778, 2016.
doi: 10. 10.1109/VVPR.2016.990 5
Matthias Hein, Maksym Andriushchenko, and Julian Bitterwolf. Why relu networks yield high- confidence predictions far away from the training data and how to mitigate the problem. 2019 JEEF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 41-50, 2019. 2, 6,8
Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-of-distribttion examples neural networks. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net, 2017. URL https:://// /aap::///pennee /pwpreveeeeeeeeeeeeeeeeeeeeeeeee net forum? -i-iiddkseee HKg4T11x1.
Dan Hendrycks, Mantas Mazeika, and Thomas Dietterich. Deep anomaly detection with outlier exposure. In International Conference on Learning Representations, 2019. URL https:/ /// openreview n forum? beprummid-Hyee Hdd 2
Darryl Hond, Hamid Asgari, Daniel Jeffery, and Mike Newman. An integrated process for verifying
deep learning classifiers using dataset dissimilarity measures. International Journal of Artificial Intelligence and Machine Learning, 11(2):1-21 jul 2021. doi: 10.4018/jaaiml.8895
2,
Haiwen Huang, Zhihan Li, Lulu Wang, Sishuo Chen, Bin Dong, and Xinyu Zhou. Feature space singularity for cutrof-distribution detection. ArXiv, abs/2011. 2021. 3,
Susmit Jha, Sunny Raj. Steven Fernandes, Sumit K Jha, Somesh Jha, Brian Jalaian, Gunjan Verma, and Ananthram Swami. Atribution-based confidence metric for deep neural net- works. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Gar- nett (eds.), Advances in Neural Information Processing Systems, volume 32, Curran Asso- ciates, Inc., 2019. URL Jcc1c6e8Ecc2227TEE84555beee
neurips cctpppper/21///11e.e. Papeaaeebbbbaaaa..pp... pdf.
Alexander Kolesnikov, Alexey Dosovitskiy, Dirk Weissenborn, Georg Heigold, Jakob Uszkoreit, Lucas Beyer, Matthias Minderer, Mostafa Dehghani, Neil Houlsby, Sylvain Gelly, Thomas Un- terthiner, and Xiaohua Zhai. An image is worth 16x16 words: Transformers for image recognition at scale. 2021.
11

---

Under review as a conference paper at ICLR 2023
Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, 2009. 2,
Kimin Lee, Honglak Lee, Kibok Lee, and Jinwoo Shin. Training confidence-calibrate classifiers for detecting out-of-distribution samples. In International Conference on Learning Representations, 2018a. 2
Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. A simple unified framework for detecting out-of-distribution samples and adversarial attacks. ArXiv, abs/1807. 0007.03888, 2018b.
Fei-Fei Li, Marco Andreeto, Mare' Aurelio Ranzato, and Pietro Perona. Caltech 101, 2022. URL htttps://data. caltech. edu/ records/cooeeeeeeeee 20086. 5
Hao Li, Xiaopeng Zhang, Qi Tian, and Hongkai Xiong. Attribute Mix: Semantic Data Augmentation for Fine Grained Recognition. In 2020 IEEE International Conference on Visual Communications and Image Processing (VCIP), pp. 243-246, 2020. doi: 10.11000CIPP491119000009001177
3
Shiyu Liang, Yixuan Li, and Srikant. Principled detection of distrriiis exampes networks. CoRR, abs/1700.00660, 2017. URL http::://aaxxii..o//aooos.///// http://b///1/00 02690. 2 arxia.
examples neural
Shiyu Liang, Yixuan Li, and R. Srikant. Enhancing the reliability of out-of-distribution image detection in neural networks. In International Conference Learning Representations, 2018. URL https:tp//// /oppnreeeeeeeeeeeeeeeeeeeeeerree www.weeweeweeeeeeeeeeeeeee.eee.e forum? -idiiddiiiii Hai--H1VGKIIi IxRZ. 2
Weitang Liu, Xiaoyun Wang, John Owens, and Yixuan Li. Energy-based out-of-distribution detection. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Ad- vances in Neural Information Processing Systems, volume 33, pp. 21464-21475. Curran As- sociates, Inc., 2020.
neurips cccccc//paaa httppppper/220... /ile/ 7aaeaaabbbbb00oppppppee pdf. 2,
Tien Thanh Nguyen, Anh Vu Luong, Manh Truong Dang, Alan Wee-Chung Liew, and John A. W. McCall. Ensemble selection based on classifier prediction confidence. Pattern Recognition, 100: 107104, 2020. 2,9 9
Yuxin Peng, Xiangteng He, and Junjie Zhao. Object-Part Attention Model for Fine-graine Image Classification. IEEE Transactions on Image Processing, 27(3):1487-1500 March 2018. ISSN 1057-7149, 1941-0042, doi: 10.1 109/TIP.201..27740. URL http: /ttttp:://aaeeeee /t////axiv// www.ooggabaao. /bbs/ 1704.01 01740. arviv: 1704.01740.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. CoRR, ftns/1409.155 2015.
Daniel Smilkov, Nikhil Thorat, Been Kim, Fernanda B. Viégas, and Martin Wattenberg. Smoothgrad: removing noise by adding noise. ArXiv, abs/1706.03825, 2017. 7,8 8
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In 2nd International Conference on Learning Representations, ICLR 2014, 2014. 1,8
Fahim Tajwar, Ananya Kumar, Sang Michael Xie, and Percy Liang. No true tteee-of-the-art? ood detection methods are inconsistent across datasets. ArXiv, abs/2109.055 2021. 2
P. Welinder, S. Branson, T. Mita, C. Wah, F. Schroff, S. Belongie, and P. Perona. Caltech-UCSD Birds 200. Technical Report ENS-TR-2010-00) 2010. 5.5
Romain Xu-Darme, Georges Quénot, Zakaria Chihani, and Marie-Christine Rousset. PARTICUL:
Part Identification with Confidence measure using Unsupervised Learning. Accepted at XAIE: 2nd Workshop on Explainable and Ethical Al - ICPR 2022, June 2022. https:// haapp:s//haaccaa...
Trchives-ouverie
/ea 0oteeaa-3703390 03703962. 3, 4, 5, 6, 8, 10
Jingkang Yang. Pengyun Wang, Dejian Zou, Zitang Zhou, Kunyuan Ding, WENXUAN PENG, Haoqi Wang, Guangyao Chen, Bo Li, Yiyou Sun, Xuefeng Du, Kaiyang Zhou, Wayne Zhang, Dan Hendrycks, Yixuan Li, and Ziwei Liu. OpenOOD: Benchmarking generalized of distribution detection. In sixth Conference Neural Information Processing Systems Datasets and Benchmarks Track, 2022. URL https:/ wpwpreveeweeeeee////p// ffot/ffoumm htt.//forommiddiooooee 1i= j4 j__tj______ skkt..
12

---

Under review as a conference paper at ICLR 2023
L. Yang, Ping Luo, Chen Change Loy, and Xiaoou Tang. A Large-Sctt -cale- Car Dataset for Fine- Grained Categorization and Verification. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3973-3981, 2015. 3,5
Xiangyun Zhao, Yi Yang, Feng Zhou, Xiao Tan, Yuchen Yuan, Yingze Bao, and Ying Wu. Rec- ognizing Part Attributes With Insufficient Data. In 2019 JEEE/CVF International Conference on Computer Vision (ICCV). pp. 350-360, Seoul, Korea (South), October 2019. IEEE. ISBN 978-1-72814-803-88 doi: 10. 10.1009/CCV22019.000044 URL https: htpp:://ieexxxooeeeeee. ieee.. org//document/9009920 9009111//
Heliang Zheng, Jianlong Fu, Tao Mei, and Jiebo Luo. Learning Multi-attention Convolutional Neural Network for Fine-Grained Image Recognition. In 2017 IEEE International Conference on Computer Vision (ICCV), pp. 5219-5227, Venice, October 2017. IEEE. ISBN 978-1-5386-
1032-9. doi: 10.1109/1CC.22017.5577 URLhttp htattt:://eeeexeeeee /eeeexpleeeeeeeeeeeeoe leee.org orgee.rr//occumeeee 8237819/. 3
13


Reviews:
[{"review_id": "lDNHQZi3rR", "reviewer": "Reviewer_w3oX", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a pattern identification-based OoD detection technique and further leverages a visualization-based approach to interpret the obtained results, providing more tangible results from the human’s perspective. The proposed techniques are based on a previous work PARTICUL, and the evaluation is performed to demonstrate the usefulness of the proposed methods.", "summary_of_the_review": "The paper proposes a sound and feasible method for OoD detection, the evaluation also demonstrates the potential of the proposed method. However, it still posts a few major concerns considering its current status:\n\n- Although the proposed techniques are overall feasible, the novelty is limited, which is mostly based on previous work. The insight of the proposed techniques is also not well discussed.\n\n- The contribution is also limited especially in terms of technical contribution to push the research of OoD, which are mostly based on two previous work for OoD detection, and post an explanation of the results.\n\n- As for the robustness analysis via perturbation, I think this could be interesting. However, the current status does not give too much insight into the impacts and implications of the perturbation, and how the OoD could be connected to robustness, e.g., what is the relation, what is the gap there.\n\nOverall, I do believe the problem this work intends to solve is important. However, it still needs further enhancement to clarify the novelty and contribution.", "strength_and_weaknesses": "Strength:\n\n- Important problem\n- Sound and feasible solution\n- Extensive evaluation to demonstrate the potential usefulness\n- Promise results\n\nWeakness:\n\n- Limited novelty, which is mostly an incremental extension of previous work\n- Limited contribution and insight, the authors mostly discuss how, but unclear why and what is the insight behind the techniques.\n- Interpretable results would be highly demanding, however, this paper again only leverages another existing method for visualization.\n- Not very clear about the implications of the perturbation magnitude and their relation to OoD.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "ZhTzf8rRTx", "reviewer": "Reviewer_2viJ", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "In this paper, the authors propose a new out-of-distribution (OoD) detection method. Specifically, the PARTICUL algorithm is used to find recurring patterns in the training dataset. Then, the degree to which each pattern is included in the data is measured and used as an OoD score. Also, thanks to the use of PARTICUL, the results of the proposed OoD detection method are visually interpretable. Finally, the authors conducted an experiment to detect 1) OoD data of a completely different class from the IoD dataset and 2) OoD data with a distributional shift.", "summary_of_the_review": "I believe the quality of the writing, the experimental settings, and the quantitative results should be improved.", "strength_and_weaknesses": "Strength\n1. The interpretability of the proposed method allows us to understand the cues of the data detected as anomalies.\n\n\nWeakness\n1. I believe that the writing quality of the manuscript should be improved.\n- The format of the paper is different from the ICLR format. It is necessary to fix the space between each paragraph and the space at the bottom of the page.\n- There is no explanation of how PARTICUL is trained. In particular, on page 4, there is no explanation of how L_l and L_u are calculated.\n- In Figure 3, each image seems to be unaligned, and in (b), the noise suddenly appears on both sides of the image.\n\n2. The OoD detection methods used as the baseline should be changed. Among those models [1,2,3,4,5] that are generally used as baselines in the OoD detection community, only [1] is used in this paper. If the authors want to use fNRD and FSSD rather than using these models as baselines, I think they should provide a clear reason for it.\n\n3. I have concerns about the applicability and practicality of the proposed method. First of all, the proposed model seems to be applicable only to those models using the CNN architecture. Therefore, unlike the existing approaches [1,2,3,4,5], it seems impossible to utilize the proposed method for other architecture such as Transformer or RNN. Also, I think training p detectors for each class requires a non-trivial amount of time.\n\n4. The improvement of the OoD detection performance is marginal. In Table 1, the proposed method does not consistently perform better than MCP, which is a highly simple method. Also, FPR95 is generally used as an evaluation metric rather than FPR80 in the community. I wonder if there is a particular reason for the authors to use the FPR80.\n\n[1] A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks, Hendrycks et al., ICLR 2017\n\n[2] Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks, Liang et al., ICLR 2018\n\n[3] A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks, Lee et al., NeurIPS 2018\n\n[4] Deep Anomaly Detection with Outlier Exposure, Hendrycks et al., ICLR 2019\n\n[5] Energy-based Out-of-distribution Detection, Liu et al., NeurIPS 2020\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "3DsA5jcjRc5", "reviewer": "Reviewer_DuyP", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors showed how the identification of recurrent patterns from the latent space of a CNN can be utilized for OoD detection problems. They utilize the existing PARTICUL algorithm to distinguish the anomaly patterns from the input and evaluate their approach in two modalities (cross-dataset OoD and perturbation OoD). ", "summary_of_the_review": "While the proposed approach is well described and the experiment results somewhat show the robustness and interoperability of the proposed model, its contribution and significance appear marginal because it is largely based on the previous work. Experimental validation could also be improved by adding other widely used benchmarks for OOD. \n\n", "strength_and_weaknesses": "[+] The problem statement and the proposed approach are clear and straightforward. The proposed method provides a graphical interpretation of OoD by mining patterns in the latent representations in CNN-based classifiers. And the key aspects of this work, the robustness and interpretability, have been investigated through experiments.\n\n[-] The major concern is the proposed approach's contribution and novelty. A considerable part of the proposed model appears to overlap with the previous model  (PARTICUL, Xu-Darme et al.(2022)). They modified the PARTICUL algorithm to be a more interpretable and robust confidence measure for OOD detection, but it does not show substantial improvement or novelty in the methodological aspects. \n\n[-] The paper mainly investigates the characteristics of their PARTICUL-based approach itself. It needs to be compared with the standard benchmark for OoD (e.g., Open-OoD). \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "01Qi86qFPt", "reviewer": "Reviewer_CPyj", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper extends a local-based confidence detection method to the image OOD detection task. They further examine the proposed method with variant kinds of perturbations, i.e. blurring, noise, brightness, and rotation. The model is tested on Caltech-101, CUB-200, and StanfordCars. ", "summary_of_the_review": "In general, I think the novelty of this paper is limited and still has some shortcomings in the experiments. Therefore, I do not recommend this paper generally.", "strength_and_weaknesses": "Strength:\n\nExtend the local-based confidence detection method to OOD detection task and further explore different perturbations which may contribute to the OOD detection.\n\nWeakness:\n\nCurrent work in Image OOD detection methods usually uses MINIST, ImageNet, and Cifar100 as the validation dataset, while the authors choose Caltech-101, CUB-200, and StanfordCars as the validation dataset. Is there any special reason for such a dataset choice? \nIn addition, the chosen datasets are focused on quite different object categories, (CUB -- birds, and StanfordCars -- cars), which can make the OOD images easier to be distinguished at the feature level. I suggest the authors also complete the experiments on the current popular OOD detection datasets so and compare the performance with proposed methods, i.e. ODIN, Maha, and FSSD.\n\nThe Table 3. Is not very clear. In the table, the authors show correlation coefficients between lambda and confidence measures, while do not provide the quantitative result for each test case. It is not clear the quantitative performance of each perturbation case on different datasets. And more analysis on the inversely correlated cases. For example, why do the blur cases and MCP have a positive correlation, while having an inverse correlation with fNRD?\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}]

Author rebuttal:

Decision: Reject