Leaving No OOD Instance Behind: Instance-Level OOD Fine-Tuning for Anomaly Segmentation
Authors: YUXUAN ZHANG
Zhenbo Shi
Han ye
Shuchang Wang
Zhidong Yu
Shaowei Wang
Wei Yang
Venue: NeurIPS 2025 poster ()
Keywords: Computer Vision
Semantic Segmentation
Out-of-distribution Detection
Outlier Exposure
Primary area: applications

Abstract: Out-of-distribution (OOD) fine-tuning has emerged as a promising approach for anomaly segmentation. Current OOD fine-tuning strategies typically employ global-level objectives, aiming to guide segmentation models to accurately predict a large number of anomaly pixels. However, these strategies often perform poorly on small anomalies. To address this issue, we propose an instance-level OOD fine-tuning framework, dubbed LNOIB (Leaving No OOD Instance Behind). We start by theoretically analyzing why global-level objectives fail to segment small anomalies. Building on this analysis, we introduce a simple yet effective instance-level objective. Moreover, we propose a feature separation objective to explicitly constrain the representations of anomalies, which are prone to be smoothed by their in-distribution (ID) surroundings. LNOIB integrates these objectives to enhance the segmentation of small anomalies and serves as a paradigm adaptable to existing OOD fine-tuning strategies, without introducing additional inference cost. Experimental results show that integrating LNOIB into various OOD fine-tuning strategies yields significant improvements, particularly in component-level results, highlighting its strength in comprehensive anomaly segmentation.

Leaving No OOD Instance Behind: Instance-Level OOD Fine-Tuning for Anomaly Segmentation
Yuxuan Zhang', Zhenbo Shil.2,, Han Ye', Shuchang Wang', Zhidong Yu! Shaowei Wang*. Wei Yang
School of Computer Science and Technology, University of Science and Technology of China Suzhou Institute for Advanced Research, University of Science and Technology of China
3 Hefei National Laboratory, University of Science and Technology of China IInttitute of Artificial Intelligence and Blockchain, Guangzhou University
Abstract
Out-of-distribution (OOD) fine-tuning has emerged as a promising approach for anomaly segmentation. Current OOD fine-tuning strategies typically employ global-level objectives, aiming to guide segmentation models to accurately predict large number of anomaly pixels. However, these strategies often perform poorly on small anomalies. To address this issue, we propose an instance-level OOD fine-tuning framework, dubbed LNOIB (Leaving No OOD Instance Behind). We start by theoretically analyzing why global-leved objectives fail to segment small anomalies. Building on this analysis, we introduce simple yet effective instance- level objective. Moreover, we propose feature separation objective to explicitly constrain the representations of anomalies, which are prone be smoothed by their in-distribution (ID) surroundings. LNOIB integrates these objectives to enhance the segmentation of small anomalies and serves as paradigm adaptable existing OOD fine-tuning strategies, without introducing additional inference cost. Experimental results show that integrating LNOIB into various OOD fine-tuning strategies yields significant improvements, particularly in component-level results, highlighting its strength in comprehensive anomaly segmentation.
Introduction
Semantic segmentation has achieved remarkable success in autonomous driving. However, traditional segmentation approaches adhere to closed-set training taxonomy [5] 55]. When deployed in
real-world scenarios, these segmentation networks struggle to predict instances of previously unseen categories (known as OOD instances or anomalies), inevitably leading to potential risks.
To identify OOD regions, great efforts have been made in anomaly segmentation (AS) 21 48
57 (55] One promising solution is OOD fine-tuning 4 47 41,, which uses mixed-content images (containing ID and OOD regions) enhance the generalization of segmentation models previously unseen instances. Current OOD fine-tuning strategies typically use global-level objectives, aiming to accurately predict as many anomaly pixels possible, rather than segmenting all anomalies. As result, these objectives lead to the neglect of small OOD instances. However, mispredicting small anomalies (e.g., a small cow Figure [ can pose sigmificant safety risks. Thus we argue that detecting small anomalies is also critical for ensuring the safety of segmentation networks.
In this paper, propose novel OOD fine-tuning framework called LNOIB ensure comprehen- sive detection of all OOD instances by the segmentation model. We start by theoretically analyzing
"Zhenbo Shi and Wei Yang corresponding authors. E-mail: (zbshi.qubit) @ussc.edu.cn
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

(a) input image
(b) OOD ground-truth
(c) M2A
(d) M2A++NOII (ours)
Figure I: Current OOD fine-tuning strategies for AS, such Mask2Anomaly (M2A) in (c), some- times neglect small anomalies (e.g., small cow in the green box). When incorporating LNOIB, M2A effectively segments both large and small anomalies (e.g.. both big and small cows in (d)).
why current global-level objectives for OOD fine-tuning often fail to detect small anomalies. For this reason, we introduce simple yet effective instance-level objective that equally optimizes each anomaly. Note that, beyond specific loss function, this objective serves as a versatile function that can be adapted to various existing global-level OOD fine-tuning losses (such as entropy- basedd loss []. energy-based loss 47), etc.). By integrating both global-level objective and our instance-level objective, we formulate an overall prediction baased objective, which effectively guides the model to detect small anomalies while maintaining high accuracy in identifying large anomalies.
Moreover, another reason for the neglect of small anomalies is that the features of OOI) instances often become smoothed by their ID) surroundings during convolution operations [54] 46). To tackle this issue, we propose a feature separation objective ensure that the prototype of each OOD instance diverges significantly from ID representations. This objective comprises an ID semantic loss and nearest neighbor loss: the former provides universal view prevent OOD features from aligning with ID semantic prototypes, while the latter ensures that each OOD prototype remains distinct from its nearest ID prototype neighbors.
LNOIB combines the prediction-baser objective with the feature separation objective to effectively segment all anomalies. Accordingly, LNOIB offers several advantages to OOD fine- mie-tuning: 1) Effectiveness: The proposed objectives in LNOIB significantly enhance the overall performance, particularly in small anomalies. 2) Efficiency: LNOIB incurs additional computational cost during inference, ensuring a consistent inference speed. 3) Versatility: LNOIB can be seamlessly adapted to existing OOD fineituning strategies, such as Entropy Maximization (EM) 44, PEBAL [47] and M2A [41]. Note that LNOIB acts as paradigm that can be integrated into existing OOD fine-tuning strategies, rather than being limited specific loss function. Experimental results show that extended by LNOIB, current OOD fine-tuning strategies achieve better performance across various benchmarks, with notable improvements in component-led and instance-level metrics.
We briefly summarize our contributions as follows:
We propose LNOIB, a novel OOD fine-tuning framework for AS, which adapts existing strategies that merely use global- objectives to improve the segmentation results of small anomalies.
We introduce prediction-bas based objective and feature separation objective within LNOIB to guide the segmentation models in focusing on small OOD instances.
Experimental results show that integrating LNOIB significantly enhances the performance of existing OOD fine-tuning strategies (EM, PEBAL, and M2A), particularly in component-level and instance-level metrics.
2 Related Work
Anomaly Segmentation AS aims to segment instances whose categories are not present in the training dataset. Existing approaches can be broadly categorized into discriminative methods 21
18]. The former primarily utilizes the predictions of semantic segmentation to estimate uncertainty, while the latter typically employs extra generative networks to model the distribution of ID samples. Both methods produce a wixel-wise anomaly score map as the result of AS. Accordingly, the most commonly used pixel-level metrics measure the accuracy of pixel predictions. However, this focus can lead to the neglect of small OOD instances, as they have less impact on the overall performance. In this paper, we mainly compare component-leved and instance-level metrics introduced in [3 37 to evaluate LNOIB.
34 24 17 58 and generative methods [16 48
10
27
29

---

aegrienaatinn may
Teature
Input image
encoder
decoder
anomaly score
(e)
(b)
Lloob
anomaly scare
GT
Lpped
ID callgory protutypes
ODD instance prototypes
back propagation for OOD fine-tuning
pixel-wise
mntancc-level anomaly soore
pixel-wise distance iglota/insttane))
cosine similariyy
ID calegoty prolotypes
aFOD instances
Figure 2: Overview of LNOIB fine-tuning framework for AS. (a) illustrates the prediction- based ob- jective containing commonly-used global-level objective glob and novel instance-level objective Lins; (b) depicts the feature separation objective, including an ID semantic loss Lsem (with some connections omitted for clarity) and nearest neighbor loss Lnear-
OOD Fine-Tuning OOD fine-tuning is optional yet effective strategy in discriminative methods for AS. The main goal is to use mixed-content dataset (containing both ID and OOD regions in each image) to simulate anomalies, thereby exposing the models to outliers and enabling them to learn the differences between ID and OOD patterns [22 40, 38). Common approaches for generating these mixed-content datasets follow direct cut-and- and-paste strategy [47 311. Moreover, suitable fine-tuning objective is required to guide the model in distinguishing between ID and OOD data. Existing literature often employs softmax entropy 44, energy 47), logit [36] to design fine-tuning objectives in pixel-wise manner. However, these objectives tend to focus on correctly predicting more pixels globally, falling short of detecting small anomalies. In this paper, we propose novel OOD fine-tuning framework, termed LNOIB, to segment both large and small anomalies.
Prototype Learning Prototypes can be regarded as the mean representations of semantic category features few-shot segmentation [50 25 26
60 3 Existing approaches typically utilize category-
wise prototypes to align pixel-wise features and produce dense predictions [50] 26)] In contrast
to previous studies, this work investigates whether prototype-based representations can benefit the OOD fine-tuning process. Specifically, construct ID prototypes for each ID category and OOD prototypes for each OOD instance, and encourage clear separation between OOD and ID prototypes. The motivation for adopting instance-wise OOD prototypes, rather than merely enforcing the separation between ID category prototypes and all pixel-wise OOD representations, is also supported our instance-level theoretical analysis perform well small anomalies.
Methodology
We begin with brief review of AS. Then we theoretically analyze the limitations of existing global- level fine-tuning objectives and introduce versatile instance-level objective. Next, we propose feature separation objective to enhance the detection of small anomalies. Finally, we present the overall OOD fine-tuning framework of LNOIB.
3.1 Task Preview
Given an input image x E R segmentation model is utilized extract the latent features F E RCX of through the encoder, where C is the feature dimension. The decoder of the model then processes these features to produce the segmentation results fxx E KRKHxx where K is the total number of ID categories.

---

To segment anomalies, AS yields a pixel-wise anomaly score map s(z) E [0, 1) based on f()). For example, the pioneering work [2]] uses the maximum softmax probability of f(z) as the confidence score to produce <(x). However, segmentation models often exhibit overconfidence in their predictions. To address this, OOD fine-tuning is widely adopted way to enhance the model, enabling it to separate IID and OOD patterns to refine AS performance. Despite this, current OOD fine-tuning strategies struggle to capture small anomalies. We theoretically analyze the reasons for this limitation below.
3.2 Global vs. Instance-Level Objective
Before introducing our approach, we first analyze why existing OOD fine-tuning strategies neglect small anomalies. Given a mixed-content image I and its ground-truth binary mask E {0, 1 (indicating normality or anomaly), previous OOD fine-tuning strategies typically employ global- level contrastive losses to separate ID and OOD regions, aiming to maximize the accuracy of pixel predictions across the entire image. These global-level objectives can be uniformly formulated as:
1
Lglob
Σ x:Elin
Σ ZyEnRot
Lour (s(x))
Nin
Nout
Lip
Lowut
where Qin and Sout denote the image lattices of ID and OOD regions (based on the binary mask y), respectively. i and j are the spatial indices. Nin |Stnn| and Nout |Rourl separately are the total number pixels in Qin and Pout- Additionally, lin and lout signify the pixel-wise loss for ID and OOD pixels, respectively. Note that, Lin and lout can be any metric that measures the similarity between the pixel-wirg -wiee- anomaly score and its ground-truth. including entropy-buse loss [4. energy-based loss [47], and logit-based based loss [41]. For convenience, denote the first and second terms of Eq. (I as Lin and Lout, respectively.
As to the OOD region Douts it typically contains several independent OOD instances O {01, 02, ON where is the number of anomalies. Accordingly, we have Dout Qoy Zog
U Ωox. Below, we present Lemma
Lemma 1. (Weighted Loss Decomposition). The total loss Liout decomposes into instance-speciffc components:
Lout
1ooz I- Beynston Houe [s(xz), wk Lo
noout
OxEO
Long
where wik
A
[Rayl Most
is the normalized weight of instance og and Loy is the mean loss over Day-
The proof of Lemma provided in Appendix A Based on this decomposition, the overall loss Lout can be viewed as weighted sum of anomaly-wise losses Log where the weights correspond the relative size of each anomaly region. This formulation enables further analysis of the influence of the dominant anomaly on the total loss. That is, when single OOD instance occupies a large portion of the anomaly region, the following theorem provides a bound on Lout in terms of the loss contributed by that dominant OOD instance:
Theorem 1. (Dominant Instance Effect). there exists & {1...., N), such thut:
|2,1 2 - 2)utt for some E (0,
(3)
then the loss Lout is bounded by:
e) S Lout S (1 e)oo + max Loz
ket
The proof of Theorem 1 given in Appendix A
Based on this, derive the following corollary:
Corollary 1. (Asymptotic Dominance). If
Doy 1 (i.e., e - 0), then Lous - Lo. in probability.
Dand
Proof. From Theorem I as 0, both bounds converge to Lo.

---

According to Corollary [ we observe that small anomalies contribute minimally to Lous, leading to limited optimization during OOD fine-tuning. This likely explains why small anomalies are often ignored in the commonly used global-level objectives. To address this issue, we propose simple yet effective instance-level objective that focuses on each separated OOD instance, formulated as:
Lins Lin
Σ Σ OR 2y
lour (s(r) Roz
1
(5)
where Tin equal that of Eq. This instance-level objective addresses the above drawback by encouraging segmentation models to equally focus on each anomaly. However, merely employing Lime insufficient because ensuring the segmentation quality of large anomalies also important. Specifically, Line draws attention to all potential anomalies while reducing the penalty on large ones compared to Lygob. Hence, we incorporate the commonly used global-Ieve loss to Lins, and formulate the total prediction-based loss as:
Lyredd googglo + (1 a)) Lins
where a balanced factor that controls the trade-off between global-level quality and instance-level completeness. Note that, Lpred is not limited to one specific loss; instead, it acts as a paradigm for existing global-level objectives that calculate the similarity between s(z,) and yy- For example, if we select the entropy-based function [4] for lin and lout to calculate Lgtob. our proposed Lins will also use the same entropy-based function. This paradigm can extended other global-level objectives as well. In this paper, we use the global-level objectives in EM [4] PEBAL [47]. and M2A [4] to evaluate the versatility of our proposed instance-level objective. For detailed formulations, please refer to Appendix B
3.3 Features Separation Objective
In addition using global-level objectives for OOD fine-tuning, another reason for inadequate performance on small anomalies is that the features tend to be smoothed out by their ID surroundings under convolution operations [54] 46). Therefore, explicitly constrain the anomaly features to make them more distinguishable to address this issue.
Given image and its latent feature F, we first calculate the prototype Pos, for each OOD instance og in a widely used masked average pooling (50) manner:
1
Σ F3 xyENa
Pos
Now
where F, E C represents the pixel-wise representation of pixel τy. Accordingly, the prototype set P = (Por .o...... Pox for all anomalies be easily obtained. The rationale for constructing prototypes for each OOD instance, rather than merely enforcing the separation between ID category prototypes and all pixel-wise representations, further elaborated in Appendix [..4]
As our goal is to separate OOD prototypes from ID representations, one intuitive approach is to ensure that each OOD prototype far from each prototype of the ID category (2). The prototype for the c-th ID category can be established based on the segmentation model by:
En, F5. ([a((,))) r] E., ((f(z)) > -]
qe =
(8)
where fa(x,) represents the probability of predicting pixel x; as category c, is threshold for feature filtering, and [.] is an indicator function. Inspired by [13, we set higher = 0.7 to ensure the quality of each ID prototype. Accordingly, the prototype set for all ID categories in x can be obtained as Q (41, (2, .... ).
Therefore, we formulate the ID semantic loss as:
1
Leem
Σ
Σ
cosSim qel
(9)
N K
Poy, EP E
5

---

where cosSim[:: is the normalized cosine similarity (ranging [0, 1)) between the prototype of each OOD instance and that of each ID category. In this way, the prototypes of OOD instances are separated from those of ID categories.
However, this strategy raises another issue: OOD prototype may be close to one specific ID proto- type while being far from others. We present Theorem provide a foundation for understanding this phenomenon.
Theorem 2. (Finite- (Finite-Categon Category Mean Similarity Bound). Let Q {qc) be set of K ID class
prototypes, and Pt, be an ID instance prototype from class t. Assume:
1. Intra-class Alignment: cosSim(pis, 9:) where E [0, 1] is small (e.g., - 0*).
Inter-class Separability: cosSim(pi.. 9e) S for all + e, where E [0, 1] is close to (e.g.
5 + 0+).
Then, the mean cosine similarity SK between Pe, and all ID prototypes bounded by:
1 1- E + (K - 1)
SK = K
cosSim(pio 1o) S K
(10)
For large (e.g., K 1), this simplifies to:
SK 8 1-e-d K 8
(11)
The proof of Theorem is provided in Appendix A] Theorem indicates that using Leem cannot determine whether the prototype of an OOD instance falls into specific ID category, especially when there are numerous ID categories (large K). To address this, we advocate for separating OOD prototypes from their nearest ID prototypes. Hence we calculate nearest neighbor loss Lnear
Lpear
Σ Poç PP
Σ
cosSimlpor 9el
(12)
N
where Qo represents the top-M nearest ID prototype neighbors of the OOD instance Og- Combining Leem and Lnear, we formulate the overall instance-level feature separation loss as:
L = BLeem + (1 (L- Lnear
(13)
where 3 is balanced factor to adjust Leem and Lnear-
3.4 OOD Fine-tuning and Inference
OOD fine-tuning: Given a segmentation model trained on the close-set taxonomy, we propose the LNOIB framework for OOD fine-tuning to encourage the model to segment all OOD instances, as shown in Figure
The overall objective LNOIB formulated by:
LLNOTB 7/1 Lpred + 722 Lf
(14)
where 71 and 722 are balanced factors. LLNOIR can be technically adapted to global-level objectives in existing OOD fine-tuning strategies. Specifically, Lpred acts as an instance-level extension of the global-level objectives, while L can be casily computed using the latent features of segmentation models. As to the weights " and 7/2 please refer to Appendix E for further analysis.
After OOD fine-tuning with LLNOIB, we use the segmentation model to generate anomaly scores. Different OOD fine-tunie tne-tuning strategies may provide various methods yield anomaly scores. To ensure the versatility of LNOIB, we retain the original method used in each strategy. For example, if LNOIB based on PEBAL 47 the anomaly score obtained by calculating energy map as described in 47, without any modification. Consequently, the inference process depends on the specific strategy used and does not introduce extra computations, demonstrating the versatility and efficiency of LNOIB.
Inference:
free

---

Table 1: Component-level results multiple datasets. By incorporating LNOIB, existing OOD fine-tuning methods achieve higher results. The best results are in bold and the second best results underlined. means higher values are better.
Approsc
SWIYC-RA
10V
F78
SMIYC-BD
1PV
F1"
<21
10V7
F11
6311
FST.2
10V
111
1074"
FTT
Syalhae (CVPR 21
26.93
19:01
0.57
35.35
36.32
35.27
19.34
27.93
25.67
22.35
14.06
10.85
35
27:57
20.35
FomtEaeler (LAI 23
15
21.16
17.06
5.72
40295
42,07
37.20
13:43
5.87
11.82
10.00
9.56
3.40
21 1.300
23.04
20.45
RhA (CEV'E))
56.26
41.35
43/04
47:44
56.16
50:43
37.03
30.95
26.54
27.4
18.06
38.27
37.15
35.70
43.37
CSL (AAAF3)
4510
47:30
41.35
41.66
05:98
23.55
18:90
12.64
6.70
g07M (BOCY 26
55,10
58.15
47.44
$2,89
7021
64.85
34.52
$7.15
28.21
19.96
36.27
18.31
41.65
44.14
45.38
(BCCD 24 107
46.5
24:22
19.83
42.65
$7.49
50.82
23.65
20:05
24.71
25.25
22.54
20.31
20.74
38.90
41.18
40115
78758
45.79
15:11
11.35
21
21.53
21.5
11
3051
43:90
45.25
32.20
22.29
29.56
23.00
27.30
(=0.35)
PERAL (ECEN"22)
41442
11:07
18.00
2131
9.66
1775
21.75
22.33
17.1
12.85
1295
5.51
31.51
16.44
2187
(LNOIBouus)
(2340)
52.17
(50.13)
39.55
(40.4))
33.45
(10.30)
32,85
(4006)
12.50
(30 163
13:46
31,81
25
(94.19)
24.25
(20119)
29.44
(30.30)
27.36
130.33
25.74
17.73
(30
57.68
(40.15)
30
(=0.22)
32.65
MIA (AM4)224 24)
45:36
50.49
6402
35:34
22.30
26.51
19.4
23.76
$7.43
40.90
44.57
(LNOIBour))
(-0.00
(40,35)
(-007)
(44.15)
640
(40
(40.25)
(-0006)
4 Experiments
4.1 Experimental Setup
Datasets: As the ID dataset, we adopt the Cityscapes dataset 9 for pre-training, which includes 2975 training and 500 validation images, containing 19 different urban scene categories. For OOD datasets, we evaluate our approach on various AS benchmarks. The Fishyscapes benchmark M includes two datasets: Fishyscapes Static (FS Static) and Fishyscapes Lost & Found (FS I F&F.. The former contains 30 validation images from blending Pascal [12. and the latter is based on Lost and Found dataset [39]. with 100 validation images. SMIYC benchmark 3 consists of two separate datasets: RoadAnomaly (SMIYC-RA) and RoadObstacle (SMIYC-RO), which contain 10 and 30 validation images with road anomalies and obstacles, respectively. Additionally, the Road Anomaly dataset (30), which served as a precursor to SMIYC, includes 60 images with anomalies located in or near the road for validation.
As our target to segment all anomalies regardless of their sizes, we mainly focus on the component-level metrics to evaluate LNOIB, including the component- wisee intersection over union (sIoU), the positive predictive value (PPV), and the averaged component-wis Fl score (FI*). Please refer to Appendix C for detailed information. These metrics reflect the extent to which each disjoint component is covered, ensuring that smaller components are weighted equally with larger ones. Furthermore, we also consider the commonly used pixel-level metrics, i.e., Area under the Precision-Recal Curve (AuPRC) and False Positive Rate at true positive rate of 95% (FPRas), to guarantee the overall pixel-level quality. In addition, inspired by (37], we also adopt instance-level metrics iAP and iAP50 to further validate the effectiveness of our method. For detailed definitions and corresponding results, please refer to Appendix F
Evaluation Metries:
Implementation Details: To test the versatility of our approach, we employ LNOIB to EM PEBAL 47). and M2A [4Л) for the three leading AS solutions utilizing global-level OOD fine- tuning strategies. For fair comparisons, we adopt the same configurations as each respective approach during the first stage, where closed-set segmentation model is trained on the Cityscapes [9]. During the OOD fine-tuning stage, our LNOIB objectives are built on the global-level losses used in EM, PEBAL, and M2A, respectively, with parameters set as = 0.5, 3 0.5, M 1, T 0.7, and
71 = 72 = 1 empirically. For detailed formulations of LNOIB objective, please refer to Appendix B We select features from stages 2, 3, and 4 of each backbone, upsample them to 1/4 of the image size, and incorporate them to calculate L feaat- For consistency, we fine-tune the whole segmentation model for each method using their corresponding configurations. We adopt AnomalyMix [47] to sample 297 images from COCO (28) and mix them into Cityscapes to generate outlier images and identify each OOD instance for calculating Lins and Lfeat- During inference, we follow the methods to yield anomaly scores in EM, PEBAL, and M2A, respectively.
4.2 Main Results
Component-level performance: We first evaluate the component-level performance of LNOIB on multiple datasets. The results for sloU, PPV, and F1 are shown Ta As seen, OOD fine-tuning strategies like EM, PEBAL, and M2A show significant improvements when integrated with LNOIB. Notably, M2A, when extended with LNOIB, outperforms other methods such as SynBoost [TO) Det [18). CSL [58), RbA 36). RPL (31] RWPM (57) and PixOOD 49 Please note that
7

---

Table 2: Pixel-level performances SMIYC-RA, SMIYC-RO, FS Static, FS L&F, and Road Anomaly. By incorporating LNOIB, existing OOD fine-tuning strategies achieve improvements in most cases. | means lower values better.
Approach
SMIYC-RA
SMIYC-RO
Static
FSLAF
Road Anomaly
AuPRC 1
FiPR::
AuPRC'
FPRu:
AuPRC'
FPR
AuPRC' 1
FPRas.
AuPRC
FFR
SynBoost (CVPRR221)
58.89
8.47
59.72
Hlw (UA1)23)
52.61
61.43
76.04
1:38
52.61
3.87
76.35
15.24
RbA (ICCV"23)
56
86.13
15.94
87.85
3.33
83.26
4.22
60.96
10.63
78.15
11.83
RPL (ICCV'23)
72.60
12.65
75.53
4,19
87.27
49.92
16.78
63.96
26.18
CSL (AAAA'24)
RWPM (BCCV 24)
577
$8.30
11.87
91.68
0.40
81.69
4,51
74.45
4.83
76.12
13.67
PiOOD (20CV
[59
58.83
34.64
84.96
0.93
85.37
4.77
84.53
2.27
75.79
13.38
(ICCV
83.93
17.13
84.81
5.40
70.86
21.47
(NNOIB
86.26
15.28
84.47
2.39
86,95
8.89
81.68
5.18
72,42
19.74
(10.47)
(100.11)
(2:0.35)
(100.06)
(=:0.23)
(400.10)
(=0.24)
(100007)
(200688
(1:0.17)
PEBAL
53.72
32.18
27.24
19.58
$2.73
6.81
59.83
6.49
62.37
28.29
-LNOOB((urs)
61.46
22.03
31.30
18.44
83.81
4.47
64.50
3.82
65,35
22.77
(+0.444
(+0.18)
(+0.00)
(+0.144
(+0.52)
(+0.0)
(-0.40)
(+0.10)
MEA (PAMI 24) 2N
S7.83
15.09
91.32
0:43
71.36
10:28
89.91
1.8
75.70
16.31
(NOOB(ouus)
92.20
11:34
92.58
0.33
75.19
8.27
90.54
1,68
79.38
13.29
(20.07)
(20.42)
(+0.05)
Image
Ground- Truth
Former
M2A
LNOIB
Figure 3: Qualitative results on Cityscapes. We compare the close-set performance of Mask2Former, Mask2Former fine-tuned with M2A, and Mask2Former fine-tuned with LNOIB (based on M2A), showing minimal close-set performance drop.
LNOIB is compatible with existing OOD fine-tuning approaches. Therefore, we select EM, PEBAL, and M2A the baseline methods for integration. For more precise assessment of the performance gains brought by our instance-level extension, we report our own reproduced results of these baselines exclude the influence of setup discrepancies (such hardware discrepancy and random seed discrepancy, as we report the mean of 3 runs). These results highlight that LNOIB improves AS by anomalies. To further isolate the impact large anomalies, need to analyze pixel-level metrics. If pixel-level improvement is less pronounced, it would indicate that LNOIB primarily enhances small anomaly segmentation.
comprehensively capturing
Instance-level performance: Integrating LNOIB into existing OOD fine-tuning strategies signifi- cantly improves instance-level performance. For more details, please refer to Appendix F. which further demonstrates that LNOIB enhances segmentation across all anomalies.
Pixel-level performance: LNOIB is primarily designed to enhance component-level and instance- level results but also delivers promising results in pixel-level metrics. Specifically, we evaluate AuPRC and FPRg5 metrics across multiple datasets, summarized in Table The results show that incorporating LNOIB improves AuPRC and FPRg5 for most OOD fine-tuning approaches, including EM, PEBAL, and M2A. While these improvements instance levels, pixel-level gains appear relatively modest. This subtle improvement suggests that the enhancements are primarily driven by better handling of small anomalies. Furthermore, the competitive pixel-level results indicate that OOD fine-tuning with LNOIB effectively preserves the high quality of large OOD instances. This is because large anomalies contribute substantially to pixel-level scores, any degradation in their segmentation would have led to a noticeable drop in overall performance.
more pronounced at the component and
Close-set segmentation: OOD fine-tuning may degrade the performance on ID categories. To investigate this, we evaluate the closed-set performance of segmentation models Cityscapes after
8

---

Image
M2A (Log)
Lig Lins
Lgglo+L.jaa
LNOIB
Figure 4: Qualitative results of applying LNOIB M2A. Instances that do not belong to the Cityscapes categories are regarded as anomalies. Vanilla Lyghth in M2A falls short in predicting small anomalies (see green boxes). With the combination of Lins and L Lifeat in LNOIB, it yields competitive performances on small OOD instances.
Table 3: Ablation results each component in prediction objective and feature separation objective SMIYC-RA. Incorporating Lina, Leens and Leear contributes to further improvements.
(b) Feature Separation Objective
(a) Prediction-based Objective
Approach
doU?
PPV
PIY
AuPRC F
FPR
Aperoach
Torer
slll WPV
45.50
20.13
FI"
20.25
AmPFCT
53.05
FP
17.13
48.50
17.11
EM
48.71
42.18
41.10
35,62
29.16
27.21
75.74
$1.57
16.88
18.87
17.02
53.94
43.61
38.77
88.53
16.70
40.42
31.07
18.00
51.73
32.18
40142
31.07
15.00
53.72
3LTS
PEBAL
50.60
31.93
26.69
36.95
27.44
14.37
58.98
41.16
27.59
44.70
PEBAL
40.78
33.03
25.85
30.56
14.47
18.85
46,00
55.01
$7.57
30.78
44.36
34.76
22:13
56.41
26,89
M2A
47.72
46.70
6
62.82
53.58
54.86
$8.07
15:47
42.73
applying OOD fine-tuning with LNOIB. Interestingly, LNOIB maintains competitive mloU scores
(within 1% drop compared vanilla segmentation models, please see Appendix for details). For clearer understanding, qualitative results are also presented in Figure 3 showing that OOD fine-tuning with LNOIB preserves strong ID performance on Cityscapes, with results that remain comparable to those of the original Mask2Former.
4.3 Ablation Study
We investigate the impact of Lpred and Lfeat in LNOIB, To achieve this, we conduct thorough ablation experiments to determine the optimal settings on EM, PEBAL, and M2A using the SMIYC- RA dataset, which includes OOD instances of various sizes, from large to small. Moreover, to evaluate the generality of these components, also conduct extensive ablation study on multiple datasets. Please refer to Appendix E for more details.
We explore the influence of the commonly used global-level objective Lyghs and our proposed instance-level objective Lins for OOD fine-tuning on overall performances. Specifically, we set = 1, 0, and 0.5 in Eq. (6). which correspond to solely adopting the global-level objective Lyiob (equivalent to the vanilla OOD fine-tuning strategy), merely employing the instance-leve objective Lines and using hybrid objective of Lgiob and Line. respectively. We remove the irrelevant feature separation objective here, and the results are presented in Table 3a As shown in the table, combining Liglob and Lins achieves the best performance across multiple metrics. We also observe that solely using Lins even results in decreases both metrics, due its insufficient attention large anomalies compared to Lgtob- Consequently, the overall prediction- objective that incorporates both Lgtob and Lins effectively addresses both large and small anomalies, demonstrating promising performances.
Global vs. Instance-Level Objective:
Furthermore, we explore the optimal factor a in Eq. (6) with respect to the performance. The results and analyses provided in Appendix

---

Features Separation Objective: We explore the impact of feature separation objective for OOD fine- tuning. We directly incorporate Ssem and Leea with the commonly used gtob in each respective OOD fine-tuning strategy, and the results are shown in Table 3b. The table shows that using T.sem alone even decreases the overall perfomance. Solely adopting T.sem may cause the OOD prototype to align with the prototype of a specific IID category, as presented in Theorem 2. On the other hand, solely using Lnear is also insufficient because it cannot ensure that the OOD prototype stays separated from all ID prototypes. especially when several ID categories share similar representations. Therefore, Leem and Lnear are complementary, and their combination achieves the best results.
Moreover, we aim to identify the optimal balance factor B. The results and analyses are provided in Appendix E Additionally, we investigate the impact of the threshold and the number of nearest neighbors M for feat. For more details, please also refer to Appendix
4.4 Qualitative Results
We further explore the qualitative results of LNOIB to show the effectiveness of each proposed objective. Figure 4 iillustrates how each objective refines the performance of M2A. Specifically, we compare the vanilla M2A (solely using Igglob) with M2A+Tins (Epreat). M2A+I. and M2A with LNOIB (using the OOD fine-tuning objective in Eq. (14) for OOD fine-tuning, As depicted in the figure, the vanilla M2A falls short capturing small anomalies. With the addition of Lins and feat, LNOIB achieves superior segmentation performances small cow, traffic cones, and cup in the first, second, and third rows, respectively), while maintaining competitive results on larger anomalies. When using the overall objective in LNOIB, M2A achieves the best performance.
several small OOD instances (e.g..
5 Conclusion
In this paper, we proposed a novel instance-level OOD fine-tuning framework for AS, dubbed LNOIB, which is designed to effectively segment OOD instances regardless of their sizes. We provided theoretical analysis explaining why current OOD fine-tuning strategies struggle to detect small anomalies. Building on this insight, we introduced simple yet effective instance-level objective to target small anomalies. Furthermore, we proposed a feature separation objective to further enhance the segmentation of small anomalies. Note that, beyond single objective design, LNOIB serves as versatile paradigm that can be seamlessly integrated with existing OOD fine-tuning strategies, without introducing extra cost during inference. Extensive experimental results show that incorporating LNOIB into existing OOD fine-tuning strategies yields superior performance.
Limitation and Future Work
LNOIB serves as a versatile mechanism for existing OOD fine-tuning strategies, demonstrating strong performance across EMM. PEBAL, and M2A with a unified set of hyperparameters. However, for other approaches, particularly those whose global-level objective values are significantly larger or smaller than the feature separation term, hyperparameter tuning may be necessary to achieve optimal results. Although LNOIB effectively refines small anomalies in most cases, the quality is not always perfect, as illustrated in the first row of Figure 4 Therefore, there remains a potential risk when deploying it in safety-critical applications. Moreover, since LNOIB serves as an extension of existing OOD fine- tuning approaches, our current evaluation is conducted on validation images. more challenging test sets are expected to provide more comprehensive
Further experiments assessment of its generalization capability. Finally, OOD fine-tuning itself remains controversial, as only mimics limited OOD scenarios and fails to capture the full diversity of real-world OOD patterns. Therefore, we believe it is worthwhile to explore alternative approaches for addressing this issue.
For future work, we plan to extend our instance-level framework to other segmentation scenarios to investigate whether this mechanism can improve the segmentation of small components across different tasks [6] 45 59 53 Specifically, although there are several hyperparameters in LNOIB, they also provide different selections to suit the corresponding tasks. Moreover, extending this framework to video tasks [23
15 51
may also be promising direction for future research.
10

---

Acknowledgments
This work was supported by the National Natural Science Foundation of China (No. 62172385), the Natural Science Foundation of Jiangsu Province (BK20241819) the Innovation Program for Quantum Open Fund, Natural Science Foundation
Science and Technology (No. 20212D030290)), of Jiangsu Province (BK20240463), Fundamental Research Funds for the Central Universities, and the Anhui Provincial Department Science and Technology under Grant 202103a05020009.
References
[1] Hermann Blum, Paul-Edouard Sarlin, Juan Nieto, Roland Siegwart, and Cesar Cadena. The fishyscapes benchmark: Measuring blind spots in semantic segmentation. International Journal of Computer Vision, (1(11)::11931133 2021.
[2] Jun Cen, Peng Yun, Junhao Cai, Michael Yu Wang. and Ming Liu. Deep metric learning for open world semantic segmentation. 2021 ieee. In CVF International Conference on Computer Vision (ICCV), pages 15313-15322, 2021.
[3] Robin Chan, Krzysztof Lis, Svenja Uhlemeyer, Hermann Blum, Sina Honari, Roland Siegwart, Pascal Fua, Mathieu Salzmann, and Matthias Rottmann. Segmentmeifyoucan: A benchmark 2021.
for anomaly segmentation. arXiv preprint 104.
[4] Robin Chan, Matthias Rottmann, and Hanno Gottschalk. Entropy maximization and meta classification for out-of-distribution detection in semantic segmentation. In Proceedings of the ieeelcvf international conference on computer vision, pages 5128-5137, 2021.
[5] Liang-Chies Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan Yuille. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE transactions on pattern analysis and machine intelligence, 4044))834-848, 2017.
[6] Liang-Chieh Chen, Yukun Zhu, George Papandreou, Florian Schroff, and Hartwig Adam. Encoder-decoder with atrous separable convolution for semantic image segmentation. In Proceedings of the European conference on computer vision (ECCV), pages 801-818, 2018.
[7] Tao Chen, Chenhui Wang, Zhihao Chen, Yiming Lei, and Hongming Shan. Hidiff: Hybrid diffusion framework for medical image segmentation. IEEE Transactions on Medical Imaging, 2024.
[8] Bowen Cheng, Ishan Misra, Alexander G Schwing, Alexander Kirillov, and Rohit Girdhar. Masked-attention mask transformer for universal image segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1290-1299, 2022.
[9] Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke. Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene understanding. In Proceedings of the IEEE conference computer vision and pattern recognition, pages 3213-3223, 2016.
[10] Giancarlo Di Biase, Hermann Blum, Roland Siegwart, and Cesar Cadena. Pixel-wise anomaly detection in complex driving scenes. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16918-16927, 2021.
(11) Henghui Ding, Kaining Ying, Chang Liu, Shuting He, Xudong Jiang, Yu-Gang Jiang, Philip HS Torr, and Song Bai. Mosev2: A more challenging dataset for video object segmentation in complex scenes. arXiv preprint arXiv:2508 056300 2025.
[12] Mark Everingham, Luc Van Gool, Christopher KI Williams, John Winn, and Andrew Zisserman. The pascal visual object classes (voc) challenge. International journal of computer vision, 88:303-338, 2010.
[13] Qi Fan, Wenjie Pei, Yu-Wing Tai, and Chi-Keung Tang. Self-support few-shot semantic segmentation. In European Conference Computer Vision, pages 701-719. Springer, 2022.

---

[14] Chengjie Ge, Xueyang Fu, Peng He, Kunyu Wang, Chengzhi Cao, and Zheng-Jun Zha. Neu- romorphic event signal-driven network for video de-raining. In Proceedings of the AAAI Conference Artificial Intelligence, volume 38, pages 1878-1886, 2024.
[15] Chengjie Ge, Xueyang Fu, Peng He, Kunyu Wang. Chengzhi Cao, and Zheng-Jun Zha. Event- mamba: Enhancing spatio-temporal locality with state space models for event- video reconstruction. Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 11124-1 2025.
[16] Matej Greié, Petra Bevandić, and Siniša Segvić. Dense open-set recognition with synthetic outliers generated by real nvp. arXiv preprint arXiv:2011.11094 2020.
[17] Matej Greié, Petra Bevandić, and Siniša Segvić. Densehybrid: Hybrid anomaly detection for dense open-set recognition. In European Conference on Computer Vision, pages 500-517. Springer, 2022.
[18] Denis Gudovskiy, Tomoyuki Okuno, and Yohei Nakata. Concurrent misclassification and out-of-distribution detection for semantic segmentation via energy-based normalizing flow. In Uncertainty in Artificial Intelligence, pages 745-755, PMLR, 2023.
[19] Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask r-cnn. In Proceedings of the IEEE international conference on computer vision, pages 2961-2969, 2017.
[20] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.
[21] Dan Hendrycks and Kevin Gimpel. baseline for detecting misclassified and -uittr-distribution examples in neural networks. In ICLR, 2017.
[22] Dan Hendrycks, Mantas Mazeika, and Thomas Dietterich. Deep anomaly detection with outlier exposure. arXiv preprint arXiv: 04606, 2018.
[23] Lingyi Hong, Zhongying Liu, Wenchao Chen, Chenzhi Tan, Yuang Feng, Xinyu Zhou, Pinxue Guo, Jinglun Li, Zhaoyu Chen, Shuyong Gao, et al. Lvos: A benchmark for large-scale long-term video object segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025.
[24] Sanghun Jung, Jungsoo Lee, Daehoon Gwak, Sungha Choi, and Jaegul Choo. Standardized max logits: A simple yet effective approach for identifying unexpected road obstacles in urban-scene segmentation. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pages 15425-15434, 2021.
[25] Chunbo Lang, Gong Cheng, Binfei Tu, and Junwei Han. Learning what not to segment: A new perspective on few-shot segmentation. In Proceedings of the IEEE/CVF conference on computer vision and patlern recognition, pages 8057-8067, 2022.
[26] Gen Li, Varun Jampani, Laura Sevilla-Lara, Deqing Sun, Jonghyun Kim, and Joongkyu Kim. Adaptive prototype learning and allocation for few-shot segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2334-8343, 2021.
[27] Chen Liang, Wenguan Wang, Jiaxu Miao, and Yi Yang. Gmmseg: Gaussian mixture based generative semantic segmentation models. Advances in Neural Information Processing Systems, 35:31360-311 -1375, 2022.
[28] Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zimick. Microsoft coco: Common objects in context. In Computer Vision-ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, pages 740-755. Springer, 2014.
[29] Krzysztof Lis, Sina Honari, Pascal Fua, and Mathieu Salzmann. Detecting road obstacles by erasing them. IEEE transactions on pattern analysis and machine intelligence, 2023.
12

---

[30] Krzysztof Lis, Krishna Nakka, Pascal Fua, and Mathieu Salzmann. Detecting the unexpected via image resynthesis. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2152-2161, 2019.
[31] Yuyuan Liu, Choubo Ding, Yu Tian, Guansong Pang, Vasileios Belagiannis, Ian Reid, and Gustavo Carneiro. Residual pattern learning for pixel-wise out-of-distribution detection in semantic segmentation. In Proceedings of the IEEE/CVF International Conference Computer Vision, pages 1151-1161, 2023.
[32] Ziyin Liu, Zhikang Wang, Paul Pu Liang, Russ R Salakhutdinov, Louis-Philipee Morency, and Masahito Ueda. Deep gamblers: Learning to abstain with portfolio theory. Advances in Neural Information Processing Systems, 32, 2019.
[33] Juhong Min, Dahyun Kang, and Minsu Cho. Hypercorrelation squeeze for few-shot segmen- tation. In Proceedings of the IEEE/CVF international conference computer vision, pages 6941-6952, 2021.
[34] Jishnu Mukhoti and Yarin Gal. Evaluating bayesian deep learning methods for semantic segmentation. arXiv preprint arXiv:1811.12709 2018.
[35] Nazir Nayal, Youssef Shoeb, and Fatma Güney, A likelihood ratio-based approach to segmenting unknown objects. arXiv preprint arXiv:2409.06424 2024.
[36] Nazir Nayal, Misra Yavuz, Joao F Henriques, and Fatma Güney, Rba: Segmenting unknown regions rejected by all. In Proceedings of the IEEE/CVF International Conference Computer Vision, pages 71 1122 2023.
[37] Alexey Nekrasov, Alexander Hermans, Lars Kuhnert, and Bastian Leibe. Ugains: Uncertainty guided anomaly instance segmentation. In DAGM German Conference on Pattern Recognition, pages 50-66. Springer, 2023.
[38] Sangha Park, Jisoo Mok, Dahuin Jung, Saehyung Lee, and Sungroh Yoon. On the powerfulness of textual outlier exposure for visual ood detection. Advances in Neural Information Processing Systems, 36, 2024.
[39] Peter Pinggera, Sebastian Ramos, Stefan Gehrig, Uwe Franke, Carsten Rother, and Rudolf Mester. Lost and found: detecting small road hazards for self-driving vehicles. In 2016 IEFF/RS/ International Conference on Intelligent Robots and Systems (IROS), pages 1099-1 106. IEEE, 2016.
[40] Chen Qiu, Aodong Li, Marius Kloft, Maja Rudolph, and Stephan Mandt. Latent outlier exposure for anomaly detection with contaminated data. In International conference on machine learning, pages 18153-18167. PMLR, 2022.
[41] Shyam Nandan Rai, Fabio Cermelli, Barbara Caputo, and Carlo Masone. Mask2anomaly: Mask transformer for universal open-set segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024.
[42] Shyam Nandan Rai, Fabio Cermelli, Dario Fontanel, Carlo Masone, and Barbara Caputo. Un- masking anomalies in road-scene segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4037-4046, 2023.
[43] Matthias Rottmann, Pascal Colling, Thomas Paul Hack, Robin Chan, Fabian Hüger, Peter Schlicht, and Hanno Gottschalk. Prediction error meta classification in semantic segmentation: Detection via aggregated dispersion measures of softmax probabilities. In 2020 International Joint Conference on Neural Networks (IJCNN), pages 1-9. IEEE, 2020.
[44] Hengcan Shi, Son Duy Dao, and Jianfei Cai. Limformer: Large language model for open- vocabulary semantic segmentation. International Journal of Computer Vision, 133(2):742-759 2025.
[45] Jiaqi Tang, Hao Lu, Xiaogang Xu, Ruizheng Wu, Sixing Hu, Tong Zhang, Tsz Wa Cheng, Ming Ge, Ying-Cong Chen, and Fugee Tsung. An incremental unified framework for small defect inspection. In European conference on computer vision, pages 307-324. Springer, 2024.
13

---

[46] Shiyi Tang, Yini Fang, and Shu Zhang. Hic-yolov5: Improved yolov5 for small object detection. arXiv preprint arXiv:2309.16.995 2023.
[47] Yu Tian, Yuyuan Liu, Guansong Pang, Fengbei Liu, Yuanhong Chen, and Gustavo Carneiro. Pixel-wise energy-biased abstention learning for anomaly segmentation on complex urban driving scenes. In European Conference on Computer Vision, pages 246-263. Springer, 2022.
[48] Tomas Vojir, Tomáš Sipka, Rahaf Aljundi, Nikolay Chumerin, Daniel Olmeda Reino, and Jiri Matas. Road anomaly detection by partial image reconstruction with segmentation coupling. In Proceedings of the IEEE/CVF international conference on computer vision, pages 1565 36651-15660, 2021.
[49] Tomá' Vojft, Jan Sochman, and Jiří Matas. Pixood: Pixel-level out-of-disrribution detection. In European Conference on Computer Vision, pages 93-109. Springer, 2024.
[50] Kaixin Wang, Jun Hao Liew, Yingtian Zou, Daquan Zhou, and Jiashi Feng. Panet: Few-shot image semantic segmentation with prototype alignment. In proceedings of the IEEE/CVF international conference on computer vision, pages 9197-9206, 2019.
[51] Xudong Wang, Ishan Misra, Ziyun Zeng, Rohit Girdhar, and Trevor Darrell. Videocutler: Surprisingly simple unsupervised video instance segmentation. In Proceedings of the IEEE/CVF 2024.
Conference Computer Vision and Pattern Recognition, pages
[52] Bin Xie, Jiale Cao, Jin Xie, Fahad Shahbaz Khan, and Yanwei Pang. Sed: A simple encoder- decoder for open-vocabulary semantic segmentation. In Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition, pages 3426-3436 2024.
[53] Shibiao Xu, Shuchen Zheng, Wenhao Xu, Rongtao Xu, Changwei Wang. Jiguang Zhang, Xiaoqiang Teng, Ao Li, and Li Guo. Hcf-net: Hierarchical context fusion network for infrared small object detection. In 2024 IEEE International Conference on Multimedia and Expo (ICME), pages 1-6. IEEE, 2024.
[54] Xue Yang, Junchi Yan, Wenlong Liao, Xiaokang Yang. Jin Tang, and Tao He. Scrdet++: Detecting small, cluttered and rotated objects via instance-level feature denoising and rotation loss smoothing. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(2):2384 2399, 2022.
[55] Chengyang Ye, Yunzhi Zhuge, and Pingping Zhang. Towards open- vocabulary remote sensing image semantic segmentation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 9436-9444, 2025.
[56] Sergey Zagoruyko and Nikos Komodakis. 2016.
Wide residual networks arXiv preprint
[57] Zelong Zeng and Kaname Tomite. Random walk on pixel manifolds for anomaly segmentation of complex driving scenes. In European Conference on Computer Vision, pages 306-323. Springer, 2024.
[58] Hao Zhang. Fang Li, Lu Qi, Ming-Hsuan Yang, and Narendra Ahuja. Csl: Class-agnostic dtructure-constrainee learning for segmentation including the unseen. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 7078-7086, 2024.
[59] Yuxuan Zhang and Wei Yang. Bsolo: Boundary-aware one-stage instance segmentation solo. In ICASSP 2022-2022 IEEE International Conference Acoustics, Speech and Signal Processing (ICASSP), pages 2594-2224020 2598. IEEE, 2022.
[60] Yuxuan Zhang, Wei Yang, and Shaowei Wang. Fgnet: Towards filling the intra-class and inter-class gaps for few-shot segmentation. In IJCAI, pages 1749-1758, 2023.
[61] Zilong Zhang, Chang Niu, Zhibin Zhao, Xingwu Zhang, and Xuefeng Chen. Small object few-shot segmentation for vision-based industrial inspection. IEEE Transactions on Industrial Informatics, 2025.
[62] Wenjie Zhao, Jia Li, Xin Dong, Yu Xiang, and Yunhui Guo. Segment every cattof-distribution object. Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recogni- tion, pages 3910-3920, 2024.
14

---

NeurIPS Paper Checklist
The checklist is designed encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes]. [No] or [NA].
[NA] means either that the question is Not Applicable for that particular paper or the
relevant information is Not Available.
Please provide short (1-2 sentence) justifiictin right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation.
While "[Yes] " is generally preferable to "[No]" ", is perfectly acceptable to answer "[No] "provided proper justification is given (c.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). "[No] or "[NA] is not grounds for rejection. While the questions are phrased in binary way, we acknowledge that the true answer often more nuanced, so please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
general, answering
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Check- list".
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the
paper's contributions and scope?
Answer: [Yes]
Justification: We clearly state the problem that we want to solve and our contribution in both abstract and introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should elearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
15

---

Justification:
We have described the limitations in the main paper.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tee system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [Yes]
Justification: We have given the proof of each lemma, theorem, and corollary in the main paper and supplementary material.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: We have described the details that allow us to reproduce the results in Section and supplementary material.
16

---

As LNOIB is versatile mechanisms for current OOD fine-tuning approaches, we give the pseudo code of how to incorporate LNOIB into EM. Moreover, all of our source code will be publicly available upon publication.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We have provided the code in the code appendix. More details will be available at: htttps://githu.
wwt womm/yyuuun337/LN017
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (nttps://nips.cr cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https for more details.
//iips.c//pull wttp:/ccppppccccccctecc.ccc.ccct http://ubiicgguudeeeeeeeeee ottpuudeee/ddeeubbissiiiiooooo..
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
17

---

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification: We have describe the details and instructions in Section and supplementary material.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: Each experiment conducted by 3 times and we report the average score. Moreover, we report the standard deviation of our main results in Tables I aad
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for caleulating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative eтror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
18

---

Justification:
Related information is provided in the supplementary material.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification:
The research is conducted with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: As the potential negative societal impact, while method shows signifi- cant improvements over prior approaches in autonomous driving scenarios, it still cannot guarantee 100% accuracy. Therefore, there remains potential risk when deploying it in safety-critical applications. This claim demonstrated in the Limitation section.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologics that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
19

---

11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
This paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
We have properly cited all the used open-sourced datasets.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperswi thcode. cww.cooccaaooooo datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re- re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/moodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
20

---

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
Justification: This paper does not involve crowdsourcing nor research with human subjects
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing. editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
Justification: The core method development in this research does not involve LLMs as any important, original, or non-standard components. This paper only employed LLM for editing purpose.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non standard components.
Please refer to LLM policy for what should or should not be described.
21

---

Supplementary Material
A
More Proofs
A.1 Proof of Lemma
Lemma 1. (Weighted Loss Decomposition). The total loss Lou decomposes into instance-specific
components:
N
Lout
Σ
1aozl
Boy Hone (s(1g),
Wk Loz
(15)
12aut
oxEO
Lon
where wk
A
[0 Mowe
is the normalized weight of instance on and Log is the mean loss over Nox
Proof. As Σ ===
- /owtl < +000 and Roy S2g A... Noo = 0. the expectation over Dout
splits into disjoint domains:
Lout
Σ
low (s())) Vyl
(16)
poout
Σ
lowe [s(x)))
(17)
Mouet
ozEO My Eok
Σ
Noxl Eeenstan Howt (((z)))))1]
(18)
1/oue
ONEO
A.2 Proof of Theorem
Theorem 1. (Dominant Instance Effeett). If there exists & E (1,..,, N}, such that:
[2 2 (1 - 2out for some E (0,
(19)
then the loss Lout is bounded by:
(1 eo, S Toout (1--e) Lo + max Lop
(20)
ket
Proof. From Lemma and the weight condition, we have the lower bound:
Lout weLo, +
Σ
wn waro 2 weLo. 2 (1 bo,
(21)
As to the upper bound:
Lout S welbo
Σ
Wk
max Lon (1 E) + emxx Loy
(22)
kit
ktt
light
22

---

A.3 Proof of Theorem 2
Theorem 2. (Finite- (Finite-Category Mean Similarity Bound). Let Q = fae be set of K ID class
prototypes, and Pt, be an ID instance protorype from class t. Assume:
1. Intra-class Alignment: cosSim(pie: qt) 6, where E [0, small (e.g., E + 0*
Inter-class Separability: cosSim(pi, 9e) S for all + e, where E [0,1] is close to (e.g..
5 > 0+).
Then, the mean cosine similarity SK between pi, and all ID prototypes bounded by:
K
1 1 + (K - 1)0 cosSim(pie 96) S
SK = K
(23)
K
For large (e.g, > 1), this simplifies to:
SK
8 6 k 8
(24)
K
Proof. The sum of Sx can be decomposed as:
SK
1
cosSim(pi
Σ
cosSim 9c)
K
1
1 - + Σ
cosSim(pi.
K
S K (1 E + (K 116)
(25)
More explanations: In well-trained segmentation model, the intra-class feature variation tends to be minimal. Consequently, the prototype of an ID instance is typically close to the center of its corresponding category. However, this raises concern: relying solely on IID semantic loss may cause an OOD instance to be mistakenly classified as belonging to one of the ID categories, especially when the number of ID categories large. This occurs because the semantic loss for ID categories can also be near zero under such circumstances, which gives a false sense that the prototype of an OOD instance is far from all the ID semantic prototypes. Thus we also need the nearest neighbor loss to deal with this issue, as demonstrated in the main paper.
A.4 Instance-Level OOD Prototypes
In the main paper, propose construct OOD prototype for each OOD instance, rather than merely enforcing the separation between ID category prototypes and all pixel-wise OOD representations. This design choice motivated by the limitations global-level feature separation objective, which tends to overlook small or localized anomalies.
We provide theoretical analysis of the feature separation objective Leem when globally separating each OOD pixel-wise representation from ID semantic prototypes (Averaging all OOD pixels suffers from the same issue). This analysis shows that, without instance-level OOD prototypes, small anomalies are likely to be ignored in the feature separation objective.
Firstly, we formulate the global-level ID semantic objective I Lygsem as:
Lysem
Σ
Σ
cosSim[Fa, gel
(26)
K K|Soud
E. a.
23

---

The objective of Lgsem is to enforce clear separation between ID category prototypes and pixel-wise OOD representations.
Lemma 2. The total objective of Ligsem decomposes into instance-specn components:
Tggee
MosSim[Faa. qcl
(27)
K|Dout
qeEQ Eyeul
1
Σ
E By Roz
cosSSm qel
(28)
Mooutl
K
oxE0
BeEQ
bysem,nk
Tk Lgsemo
(29)
where over Dow.
is the normalized weight of instance, and Lgsem,or is the mean ID semantic loss [ooee
[Rel_
Proof.
T Igsem
Σ
Σ
cosSim F Fay. 9e]
(30)
K|Douel
9-EQ s Enout
Σ
JosSim(Fa, qel
(31)
K|Soutl
4.EQ oN E0 E,E0k
Σ
Σ
10 B2 cosSim[Faa. 4cl
(32)
K|Soue
q. q.EQ oNE0
Based on the definition Lysemoo we have:
Lgsem
Payl Lgsem.oo
(33)
Paane
ONEO
Theorem 3. If there exists t (1,2, ..., N), such that:
[2,1 2 (1 - Rouel for some E 1),
(34)
then Lisem is bounded by:
(1 Lgeem,or S Lyeem S (1 e) Lgsemoo
max Losem.or
(35)
Right
Proof.
The proof follows a reasoning analogous to that of Theorem [0a]
Corollary 2. If
1, then Lagser Lgsem,o, in probability.
[Rout]
Proof. This proof mirrors the approach taken in Corollary
Based on the theoretical analysis above, globally separating each pixel-wise OOD feature from ID prototypes also overlooks the optimization for small anomalies, thus our semantic objective Lsem in Eq. (9) is more reasonable, which equally optimizes each anomaly. Moreover, the theoretical analysis of using T.near is the same. Therefore, the feature separation objective also benefits from our instance-leve framework to refine the optimization small anomalies.

---

B
How LNOIB Adapts to Existing OOD Fine-Tuning Strategies
To provide clearer understanding of how to apply LNOIB to current OOD fine-tuning strategies that employ global-level objectives, we present a detailed formulation of the integration of EM 4 with LNOIB, 47 with LNOIB, and M2A 41 with LNOIB, respectively.
B.1 Global vs. Instance-Level Objective
As shown in the main paper, the commonly used global-level objective Lgtob and our proposed instance-level objective Lins can be formulated as follows:
gla
1
Σ
lin(s(22),
Σ
lous [8(1)).
(36)
Nin
Nout
my ayEloou
Lin
Lous
Lins
Lin
Σ Σ ok ED x Elay
1 Nop
Loue (ons((
(37)
N
where Din and Pout represent the image lattices of ID and OOD regions (based on the ground-truth binary mask y). respectively. Ni [2tml and Nout |&oou separately are the total number of pixels in Qin and Dout- O denotes the set of OOD instances and S2,, is the image lattice of the OOD instance og- These notations are also introduced in the main paper.
The proposed instance-level objective Line does not refer to a specific loss function. Instead, acts as versatile function that can be extended the existing global-level objective Lgtab for OOD fine-tuning, Specifically, the concrete formulation of Lins is determined by 4in and lout used in the corresponding Ligloh- Note that, the formulation of 4in and lout in Eq. (1) is for high-level understanding by comparing the similarity between the predicted anomaly score and the corresponding tround-truth. For each specific implementation, the input parameters ln and loat may vary. For example, EM directly adopts the predicted softmax distribution fzz), which acts as an indicator of anomaly score, to calculate the loss. Consequently, we omit the input parameter lists In and lout below for clarity.
B.1.1 LNOIB on Entropy Maximization
Given mixed-content image 33x (containing both ID and OOD regions), we first employ the well-trained semantic segmentation model DeepLabv3+ to generate a softmax prediction f(x) E R x for close-set categories, where K is the number of ID categories. Then, according JEM and JEM are formulated as:
to the close-set mask close E 2
K log(fe(x;)) 1/c yflose]
JEM in
c=1
(38)
JEM
Σ (og(f((x,))
out K
where [[]]] is an indicator function. Note that In is a typical cross-entropy loss used close-set semantic segmentation, while 1EM is an upper bound of the softmax entropy according Jensen's inequality (for more details, please refer to 4)). The goal for minimizing EEM to make each pixel predict a higher score for the corresponding ID category, while the target for minimizing JEM is to let the predicted results follow a uniform distribution. Both of these losses measure the similarity between the indicator of anomaly scores and the corresponding binary ground-truth mask, resulting in well-trained boundary of ID and OOD regions after OOD fine-tuning.
25

---

Accordingly, the vanilla global-level objective L FM is calculated by: glob
T. glob
JEM
JEM
(39)
Nin
in
Nout
2g Elauee
To adapt the instance-level objective to L first divide Rout into several separate OOD instances: Pout = 12, L R2z U... U fox as demonstrated in the main paper. Building on the global-level objective L EM we then formulate the instance-level objective LEM for EM as:
glob
F.M L ins
Σ
F.M
E
Σ
1
(40)
Nin
in
N
Doz
OL a:ERo
In this manner, we adapt the instance-level objective to E.M, followed by the calculation of I. L.
pred = a L.EM glob + (1 - (x) Line
EM
(41)
where a is balanced factor as demonstrated in the main paper.
B.1.2 LNOIB on PEBAL
Unlike EM, PEBAL employs K + class semantic segmentation model, where the class K + is designated for the anomaly score. the case of mixed-content image, if pixel belongs to an inlier class, its ground-truth is y/ {1, ..., K), which corresponds to typical one-hot encoded vector. On the other hand, if a pixel is an outlier, we assign a uniform value across all the closed-set categories (similar to EM). This is because need to calculate the close-set predicted distribution that also reflects the anomaly score for optimization.
Therefore, 1PEBAL and (PEBAL can be calculated as: out
in
Σ (fc(z))) 1/c yflose] fK+1(2;)
IPEBAL - log in
(42)
E (f((x))) [K+1(18)) K
PEBAL log
where a is an energy-biased pixel-wise adaptive factor to penalize an outlier pixel if is predicted as a close-set category. On the other hand, encourages inlier pixels to make a close-see set prediction. This loss is known as Gambler loss, and a is obtained by calculating the pixel-wise free energy. Please refer 47 and [32 for more details about abstention learning. Minimizing 1, IPERAL. and out BPEBAL also aims to bring the prediction of the anomaly score closer to the corresponding binary ground truth. Note that,
are also two auxiliary terms I.EBAL and IPEBAL We omit the formulation out
<m of them here for convenience, as these two terms contribute little to the overall performance.
Accordingly, the global-level objective for PEBAL calculated as:
L PEBAL
Nin
Σ
in
PEBAL
+
Nout
Σ
out
PEBAL
(43)
(PEBAL our instance-level objective for PEBAL Based on the formulation of PEBAL and I calculated by:
in
PEBAL
Σ
PEBAL
1
3
PEBAL
(44)
ins
Ni
in
N
1oyz
o6 E0 E.f
26

---

Combining L gl. and PEBAL ins
we obtain the overall prediction peade objective L PEBAL pred
by:
glob
I PEBAL a
pred
globb
glob
PEBAL
-ax) I
PEBAL
(45)
B.1.3 LNOIB on Mask2Anomaly
M2A is based on the segmentation model Mask2Former [8, which decouples the prediction masks and categories. Despite this difference, it ultimately yields pixel-wise prediction f(x) IR xxxxx as well. Note that, f() is the logit here, rather than softmax prediction.
According to 41
422 12A and out are calculated by:
in
M2A
K (max fa(cr))) a=l
in
2
(46)
M2A
max(0, y + max fe(x:)))
out
where is hyperparameter that decides the minimum distance between ID and OOD classes. Minimizing 1M2 and AM2A aims to create a gap between ID and OOD logits, which also makes the anomaly score closer the binary ground-truth (normality or anomaly), as demonstrated in Eq. (36).
Please refer to [11 for detailed information.
Then the global-level objective for M2A is calculated as:
L M2A
M2A
1
M2A
(47)
Nin
in
Nout
out ayERRut
2:E0...
Building 1M24 and out our instance-level objective adapted to M2A can be formulated as:
L M2A
Σ
M2A in
1 E
out
(48)
Poy
žy Elog
Then the prediction-based objective for M2A can be formulated as:
L pred = glob (1 a) M2A L
(49)
B.1.4 Summary
Above, we demonstrate how the instance-level objective in LNOIB adapts to current OOD fine-tuning strategies, including EM, PEBAL, and M2A. Lins is not specific loss function; instead, Lins can be built upon each lin and Lout and thus can be technically adapted to existing OOD fine-tuning strategies that employ global-level objectives, showcasing the versatility of LNOIB.
B.2 Feature Separation Objective
This objective can also be easily integrated into current OOD fine-tuning strategies. Since each OOD fine-tuning strategy is based on semantic segmentation model, we can leverage the feature map from the backbone to generate prototypes and calculate the feature separation objectives. Specifically, EM and PEBAL use DeepLabv3+ [6] with the WideResNet38 (56) backbone, while M2A employs Mask2Former [ with global masked attention [41] with the ResNet50 2 backbone. Accordingly,
we use the feature maps from stages 2, 3, and each backbone, resize them to 1/4 of the original input image size, and add these features together for prototype establishment.
B.3 Configurations of OOD Fine- Fine-Tuning
We apply LNOIB to current OOD fine-tuning strategies, including EM, PEBAL, and M2A. Apart from altering the overall objective, we follow the configurations used in each respective strategy without
27

---

bells and whistles, including preprocessing methods, training epochs, learning rates, optimizers, batch sizes, random seeds, and other settings. Our aim to present the versatility of LNOIB and validate whether LNOIB can enhance existing OOD fine-tuning strategies, although we acknowledge that further adjusting these settings could potentially lead to higher performance for each respective approach. For the detailed configurations, please refer to the official github repos of EMO PEBAL and M For each experiment, we conduct times and report the average performance to guarantee fair comparison.
As to the hardware, we use a server running Ubuntu 22.04. equipped with 4 RTX 3090Ti GPUs, each with 24 GB of memory, as well as another server with 2 NVIDIA A100 GPUs, each with 80 GB of memory. We adopt the Pytorch framework conduct the training and evaluation process.
C Component-Level Metries
The component-level evaluation metrics are introduced in 4 which are designed to focus on detecting anomalies, regardless of their sizes. These metrics essential because pixel-level metries may not adequately penalize a model for missing small anomalies, even though such anomalies might be critical detect. Evaluating this metric is crucial for LNOIB, our target is to capture all anomalies using such an OOD fine-tuning strategy. For comprehensive component-level assessment of detected anomalies, we need to consider component- wise true positives (TP), false negatives (FN), and false positives (FP). These quantities are measured by treating anomalies as the positive class. From these measurements, we are able to use three metrics, i.c., sloU, PPV, and F1*, to evaluate component- sesponeett-iseeeeee wise segmentation performance of anomalies. Below, we detail the computation of these metrics, using O {/1, 22, .... (T) to represent the set of ground-trute components and C' {c1, C2, ...,, CN }o to denote the set of predicted components.
sloU employed in SMIYC 3 is a modified version of the component intersection over union proposed in [4). It mainly considers the ground-truth components in the computation of TP and FN, The sloU score for a ground-truth component ly can be formulated as:
14 ConC C(tl
sloU())
((.)))()))()))
C(u)
U
Ck
(50)
c+EC,eente00
where A(t.) is term that excludes from the union of those pixels that correctly intersect with other ground-truth components different from 4. C(t.) represents the set of predicted components that intersect with &e.
Accordingly, given a threshold 77 E [0, 1], a target l E O is considered as a TP if sloU (t) > n. otherwise an FN.
PPV measures whether a predicted component Cj C' belongs to FP, and it formulated as:
(PV(c)) |cx n O(w)] [ck|
O(c))
U
(51)
la LE00nn 7M1
where O(cx) represents the of ground-truth components that intersect with the predicted compo- nent A predicted component C is an FP if PPV (c+) S n.
F1 is calculated based on the results of sloU and PPV by:
P()())
F + (n) 2TP(n) FN()) FP(n) E [0,
(52)
where TP, FN, and FP are determined by the value of 17.
Phups://github.c Phtpss//gituu..co/mobbnn http://www.ooom/ooonnn/a//tttttr
hhups://githubbbu "hanp://gituub..mmtiannnn cowpaa/aaicccca00///////.....
"aapss//gihuub..cm//hyya/////// http:///hhhyaaoooaaaaaaaaoaooom Mnnaa Unmasking Anomalies in- Soad-SScen-SSaee Scene-Segmentatioon
Unmasking-Anommlies..
28

---

Table 4: Close-set set semantic segmentation results on Cityscapes, which are presented in three groups of experiments. In each group, the first line shows the performance of the vanilla semantic segmentation model. The second line presents the results after OOD fine-tuning with current strategies (EM, PEBAL., and M2A) based on the segmentation model. The third line represents the performance after OOD fine-tuning with LNOIB based on each OOD fine-tuning strategy mentioned in the previous line.
Approach
mloU f
DeepLabv3+ (WideResNet38)
77.85
+EM
76.71
+LNOIB (based on EM)
77.33
DeepLabre abv3+ (WideResNo(38)
77.85
+PEBAL
77.21
+LNOIB (based on PEBAL)
77.14
Mask2Former (ResNee50)
80.14
+ M2A
79.39
+1 LNOIB (based on M2A)
79.18
These component-level metrics assess component locations independently of their sizes, ensuring that larger components do not dominate the metrics. Following SMIYC, we set v) = 0.5 for evaluation.
Besides SMIYC benchmark, we also employ the component-level metrics to explore the coverage of small anomalies on Road Anomaly dataset and the validation sets of FS Static and FS L&F (as the test set of Fishyscapes benchmark does not provide an API to evaluate such component-level metries). Accordingly, in Table we reproduce the component-level results of previous approaches on SMIYC RA, SMIYC RO, Road Anomaly, and the validation sets of FS Static and FS L&F, respectively. This enables a fair evaluation of how our instance-level extension improves the performance of each baseline. If we were to directly adopt the benchmark results reported in the original papers, the comparison would be less accurate due to minor discrepancies between their setups (such as hardware discrepancy and random seed discrepancy, as we report the mean of 3 runs) and our reproduced baselines. Although these differences are relatively small, using our own reproduced results allows for more precise assessment of the performance gains brought by our instance- extension. Based on the experimental results, we find that the combination of LNOIB and current JOD-fine-tunin strategies significantly enhances the component-leve' metrics, and the incorporation of M2A [42 and LNOIB achieves the top-performing sloU, PPV, and F1* results in most cases.
D Close-set Segmentation Performance
OOD fine-tuning can potentially reduce the performance of the close-set semantic segmentation. To investigate this effect, we evaluate the mean Intersection over Union (mloU) on the Cityscapes dataset. We conduct three groups of experiments: in each group, we compare the results of the vanilla segmentation model, the model fine-tuned with current global-level objectives, and the model fine-tuned with LNOIB (built on each global-level objective). The results of this comparison are presented in Table 4 We observe that adopting LNOIB results in only 0.52% to 0.96% mloU drop compared to the vanilla segmentation model. In most cases, the mloU results are slightly higher than those obtained through existing OOD fin-ttning with global-Ievel objectives.
E
More Ablation Results
E.1
Fine-grained Balance of Lpred
In the main paper, we analyze the effects of combining Lgtob and Lins- and observe that their joint use yields superior performance. In this section, we further investigate the impact of the weighting factor with respect to the performance. The results are depicted in Figure (a))re (a)-(c). Interestingly, a 0.5 achieves the top-performing results in most cases. We observe that when a S 0.4, there a significant decline in pixel-level performance (AuPRC in red). Conversely, when 0.6, the improvements in
29

---

Gicbbll Instance Objective
Instancelevel Céjective PEBAL
Global Instance Objective
Scores
Scores
(d) Feature Separation Objective on EM
Feature Separation Objective on PEBAL
I12 Feature Seporation Objective on MZA
Scoreh
scorcs
Figure 5: Results for varying balanced factors, i.e., in Lored (first row) and in feat (second row). The settings of a 0.5 and 3 = 0.5 achieve competitive performances, although they are not the best across all metrics for all OOD fine-tuning strategies.
component-level metrics (sloU, PPV, and F1*) diminish. Therefore, compromise, = 0.5 offers competitive results across all OOD fine-tuning strategies, including EM, PEBAL, and M2A.
E.2 Fine-grained Balance of feat
In the main paper, explore the effects of incorporating Lsem and Lnear, and find that their joint use yields superior performance. Here, we aim to further identify the optimal balance factor 3, as illustrated in Figure (ig (d)-(f). Our findings indicate that 0.5 delivers competitive performance across EM, PEBAL, and M2A. For simplicity and versatility, we adopt 3 = 0.5 in the final results, despite the real optimum potentially varying for each OOD fine-tuning strategy. To achieve the optimal performance for specific approaches, we can further explore fine-grained intervals within (0.40.66]
E.3 Balance of LNOIB Objective
In our main paper, we formulate the overall LNOIB objective as:
LLNOIB Lyred yeLt
(53)
In LENOIB, we simply set 71 72 1. This decision is based on the following considerations.
Firstly, the ranges of Iigiob and Tine in Ixpred the same, both being relevant to the definitions of 4. and lout. Moreover, the ranges of Leem and Lnear in L are also identical, both falling between O and 1. However, the ranges of Lpred and L feat are completely different, making challenging to set a uniform balanced factor.
Although setting proper balance factors might further improve the overall performance, the search space will be significantly larger compared to Lpred and L feat- Furthermore, since the ranges of lin and lout vary across different global-level objectives, whether balance factor is adaptable to all existing strategies needs further consideration.
In addition, the range of Ifet is [0, 1}, and Tppred is in the same order of magnitude as I and typically larger than Ljeat, indicating that Leeat acts as an auxiliary objective to enhance OOD fine-tuning, which is consistent with our experimental findings in Table the main paper, and Figure 5 of the supplementary material. Therefore, we first adopt the ostrich strategy that sets ?1 72 here. Interestingly, this setting is well adapted to various OOD fine-tuning strategies including EM, PEBAL, and M2A.
We then briefly explore the optimal values of the balancing factors 71 and ?2. Specifically, we fix ?1 and vary 72 E {0.01, 0.1, 1, 10, 100). The corresponding results are shown Figure 6 We observe that setting 72 1 yields the best performance. A larger 72 reduces the influence of the mediction-basee objective, leading to a noticeable performance degradation. Specifically, when 72 100, the impact of Lpred becomes negligible. As result, the direct objective for OOD
30

---

(b)
(c)
S0U PPV
SoU
SDU
Scores
60
Scores
40
Scores
60
40
20
20
20
0.01
0.1
10
100
0.01
0.1
100
100
0.01
0.1
1.0
1.00
Figure 6: Results for varying balanced factors "/1 and 72. We fix ?1 and search the best value of 72 {0.01, 0.1, 1, 10, 100}. The setting of m Y22 = 1 achieves competitive performances in most cases.
(a) EM
(b) M2A
Figure sloU results for different combinations of nearest number M and threshold SMIYC-RA. The combination of M = 1 and т = 0.7 yields superior results.
fine-tuning on the predicted anomaly scores is largely diminished. Conversely, a smaller "2 (such 7/2 (1.1) results in suboptimal outcomes, the feature-level optimization also plays crucial role in effectively separating ID and OOD representations. According to the results above, we finally select 71 = 72 1 as the balancing factors in overall objectives, this setting performs well on all of these approaches.
E.4 Thresholds of and M
Additionally, we investigate the impact of the threshold r and the number nearest neighbors M for
Lfeat- We evaluate the sloU results on EM and M2A, as shown in Figure 7 The figure shows that T 0.7 achieves the best performance. We believe that a smaller 7 introduces noise constructing ID prototypes, while a larger results in incomplete ID prototypes. Furthermore, we find that M = already yields competitive results most cases.
E,5 Ablation Study on More Datasets
In the main paper and supplementary materials above, mainly conduct ablation study S-IYC- RA. To further evaluate the generality of each component, we conduct more ablation study using M2A FS L&F and Road Anomaly datasets.
Firstly, we explore the combination of Tiglos and Tinns in prediction-based objective, and the results demonstrated in Table Morcover, also investigate the combination of Leem and Lnear in
31

---

Table Instance-level results on the validation FS Static and L&F. By incorporating LNOIB,
existing OOD fine-tuning strategies achieve higher performances. The results in bold represent better
performances compared with vanilla approaches. means higher values are better.
FS Statie
FTT&F
Approach
IPP
iAP50
iAP
iAP50
(ICCV'21)
23.7
30.1
25.8
37.5
LNOIB
33.4
37.1
33,8
47.9
PEBAL (ECCV'22)
23.8
32.1
19.6
25.3
LNOIB(ours)
29.4
40.7
26.8
37.2
M2A (PAMI
41.0
27.3
36.9
LNOIB(Ours)
37.6
49.3
34.0
47.3
Table 6: Ablation results of each component in prediction-based objective for M2A more FS L&F and Road Anomaly, Incorporating Lins contributes to further improvements.
FS L&F
(b) Road Anomaly
Vins
sloU
PPV
F
AuPRC
Lina
sloU
PPV
FJ
AuPRC
FPR
26.51
19.43
23.76
89.91
L,5
47.43
40.80
44.57
75.70
16.31
37.58
35.10
34.23
90.38
1.77
48.99
50.14
46.73
78.01
14.27
feature separation objective, and the results are presented in Table 7 Both tables demonstrate the superiority of Lins: Lsems and Leear our instance-level OOD fine-tuning framework, which is consistent to the results of SMIYC-RA.
E.6 OOD Fine-tuning with Imperfect Instance Masks
In the main experiments, adopt copy-and and-pasee paste strategy using ground-truth masks for OOD fine-tuning. However, when anomaly masks are unavailable, we should rely on predicted masks generated by several instance segmentation models, which are inevitably imperfect.
In this part, we try to explore how sensitive performance imperfect instance masks. To simulate imperfect instance masks, we replace the GT masks in COCO with the predicted masks from Mask R-CNN 19 (34.7 AP), which tend to have coarse boundaries. Using these masks, we apply the same copy-paste strategy create mix-content images. Experimental results of M2A on FS L&F and Road Anomaly are presented in Table showing that while using imperfect masks leads to slight performance drop compared GT masks, OOD fine-tune fine-tuning method remains effective. We attribute this to the fact that the fine- tuning mainly captures the overall OOD patterns. Imperfect masks may introduce noise into the OOD instance-level prototypes and thus slightly weaken the feature separation objective. But the impact limited, showcasing the robustness of our method.
Anomaly Instance Segmentation
The SMIYC and Fishyscapes benchmarks, designed specifically for anomaly semantic segmentation, do not inherently support distinguishing between different anomalies. Common approaches such as EM, PEBAL, and M2A are similarly limited in this regard. However, the instance-level metrics proposed in 37 provide a means to evaluate whether LNOIB can enhance segmentation performance, partieularly for small anomalies. These metrics are employed further confirm that LNOIB leads to tangible improvements at the instance level.
32

---

Table Ablation results of each component in feature separation objective for M2A on more FS L&F and Road Anomaly. Both sem and Lncer contribute to further improvements in L feat.
(a) L&F
(b) Road Anomaly
Leam Jnner
sloU
PPV
FI T
AuPRCT
FPRat
Leam
Lener
sloU
PPY
FI
AUPRCT
FPRas
26.51
19.43
23.76
89.91
1.83
47.43
40.80
44.57
15.70
T6.31
28.11
22.04
24.46
89.33
1.91
/
45.39
41.17
43.08
73.36
17.21
31.58
28.40
27.81
89.74
2,03
47.93
44.57
45.61
76.44
16.13
35.85
33.93
32.10
90.07
1.72
50.51
48.87
50.05
78.26
15.45
Table 8: Results of OOD fine-tuning with imperfect masks generated by Mask-RCNN.
L&F
Road Anomaly
Mask type
stoL
PPV
AuPRC
FPRg:
sloU
PPV
FI"
AuPRC
FPRus
39.16
38.10
34.80
90.54
1.68
53.70
51.46
50.97
75.70
16.31
Mask Mask-RCNN RCNN
38.39
36.60
33.86
88.73
1.91
50.58
51.07
47.92
75.37
16.70
F1 Instance-level Metrics and Results
In addition to the official component-level metrics, we also evaluate the instance-level metrics of LNOIB on the Fishyscapes benchmark [. following the setup in [37] These metrics include instance-level average precision (iAP) and iAP50. Since previous approaches do not utilize instance- level metrics, we primarily compare the performance of vanilla EM, PEBAL, and M2A with their counterparts enhanced by LNOIB. shown in Table incorporating LNOIB significantly improves iAP and P50 for OOD fine-tuning approaches, indicating enhanced performance on small anomalies. Since small and large anomalies occupy similar proportions, and large anomalics are already well segmented, these improvements likely highlight the enhanced segmentation of smaller anomalies.
F2 Instance Anomaly Segmentation Details
To compute these instance-level metrics, we utilize the anomaly semantic segmentation results along with the ground-truth instance masks to estimate the corresponding predicted instance masks, following the setup (37. Specifically, given the predicted anomaly semantic segmentation result H W we first obtain the anomaly pixel set f2 where pixels with value of 1 indicate an anomaly. Then, based on the ground-trute truth instance mask Sog of the k-th anomaly, we find the pixel set P {PI.P2. P2, ... PN,) A2 which satisfies the following properties: I) Vp E P. s.t. p Sloy Пog. Accordingly, the estimated instance mask of k-th anomaly can be represented as Gr {cr, C2,, CN2, J. where c; denotes the connected component associated with can derive the anomaly instance segmentation results and subsequently compute the iAP and iAP50 metrics.
E {0, 1)
2) #p E P. s.t.
pixel ps. Using this representation,
Note that, iAP and iAP50 are equivalent to AP and AP50 in Cityscapes 9 with the key difference being that the former evaluates the average precision of anomalous instances. Similar to [37], as SMIYC validation sets do not provide instance masks, we evaluate the instance-level metrics on FS Static and FS L&F datasets.
G More Clarification of Instance-level OOD Fine-tuning and Anomaly Instance Segmentation
The core idea of LNOIB to leverage instance-level OOD fine-tuning to enhance anomaly seg- mentation. Since OOD fine-tuning requires mixed-content dataset, use the widely adopted AnomalyMix, which pastes COCO instances onto Cityscapes images. Previous OOD fine-tuning methods do not consider instance-level contrastive loss and thus do not need differentiate between different anomalies when preparing the mixed-content dataset. In contrast, we assign unique instance labels different anomalies, enabling effective instance-level OOD fine-tuning. Note that, the instance-aware aware annotation is simple and straightforward in this copy-and -nd---nd-ttttt approach to generate the mixed-content dataset.
Unless explicitly stated otherwise, anomaly segmentation generally refers to anomaly semantic segmentation. SMIYC [3] and Fishyscapes []] and Road Anomaly datasets all focus this task,
33

---

which involves segmenting anomalous regions. The component-level metrics assess how well the anomalous components are covered, giving greater weight to small components compared to pixel-level metrics.
As main claim is that LNOIB improves performance small anomalies, evaluating instance-level metrics [37] provides further validation of this claim. Therefore, we also assess it using the anomaly instance segmentation task. While methods on SMIYC and Fishyscapes, such EM, PEBAL, and M2A, focus on anomaly semantic segmentation, we adopt the strategy outlined in Section F of the appendix simulate anomaly instance segmentation results.
Overall, both the component-level metrics anomaly semantic segmentation and the instance-level metrics in anomaly instance segmentation demonstrate that OOD fine-tuning with LNOIB improves the segmentation quality of small anomalies.
34


Reviews:
[{"review_id": "y94IoYZBI8", "reviewer": "Reviewer_qk6n", "summary": "The paper proposes LNOIB, an instance-level OOD fine-tuning framework for anomaly segmentation (AS). It addresses the limitation of existing global-level OOD fine-tuning methods (e.g., EM, PEBAL, M2A) that neglect small anomalies.", "questions": "see weakness", "limitations": "NA", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n1. Well-Motivated Problem: Addresses a critical gap—neglect of small anomalies—in OOD fine-tuning for AS.\n\n2. Theoretical Foundation: Lemma 1 and Theorem 1 formally justify why global objectives fail for small anomalies.\n\n3. Comprehensive Experiments: Evaluated on 5 benchmarks (SMIYC, Fishyscapes, Road Anomaly), showing consistent improvements in component-level (sIoU, F1*) and instance-level (iAP) metrics (Tables 1, 5).\n\n4. Practical Design: No added inference cost; compatible with existing methods (EM, PEBAL, M2A).\n\nWeakness:\n\n1. The core idea (instance-level weighting) resembles techniques in long-tail learning (e.g., class-balanced loss). The feature separation objective borrows heavily from metric learning (e.g., prototype alignment). While positioned as a \"paradigm,\" contributions feel incremental over prior OOD fine-tuning works.\n\n2. No analysis of how individual hyperparameters (e.g., $M$, $\\tau$) affect results beyond SMIYC-RA (Figure 7).\n\n3. Pixel-level metrics (AuPRC, FPR95) show marginal gains (Table 2), suggesting improvements primarily target small anomalies at the expense of broader robustness.\n\n4. Theorem 2 assumes idealized inter-class separability(\\epsilon)—unrealistic for complex urban scenes. Proofs (Appendix A) are sound but lack empirical validation of assumptions.", "was_revised": true, "final_justification": "Thanks for the authors' response. I believe most of my concerns are addressed", "rebuttal": "We sincerely appreciate your time and valuable feedback on our paper. We will give a detailed response to each of your constructive comments below.\n\n**W1. The core idea (instance-level weighting) resembles techniques in long-tail learning (e.g., class-balanced loss). The feature separation objective borrows heavily from metric learning (e.g., prototype alignment). While positioned as a \"paradigm,\" contributions feel incremental over prior OOD fine-tuning works.**\n\nThank you for raising this insightful topic! We will clarify the key differences between our approach and the method you mentioned.\n\n(1) Instance-level weighting resembles techniques in long-tail learning (e.g., class-balanced loss).\n\nWhile the intuitive concept of emphasizing rare/small classes/instances may appear similar to that of long-tail learning, our core idea differs significantly in terms of theoretical foundation, objective formulation, underlying motivation, and application scope.\n\nFirstly, our method is motivated by the theoretical findings (Lemma1, Theorem 1, and Corollary 1) that global-level objectives tend to focus more on large anomalies. These theoretical analyses are fundamentally different from long-tail learning.\n\nSecondly, our instance-level objective is formulated by the variation of existing global-level objectives, rather than a specific loss function, guaranteeing its generality in anomaly segmentation. Moreover, our method is image-specific—operating at the instance level per image—rather than dataset-wide as in long-tailed learning methods [1].\n\nThirdly, as to underlying motivation, the methods in long-tail learning often emphasize improving performance on rare categories in multi-class classification tasks. In contrast, anomaly segmentation in our work is inherently a binary problem (ID vs. OOD).\n\nAt last, for application scope, while long-tailed learning aims to rebalance rare ID classes, our approach explicitly focuses on enhancing the segmentation of small anomalies, which is a different problem with distinct challenges and goals.\n\n(2) The feature separation objective borrows heavily from metric learning (e.g., prototype alignment).\n\nThe use of prototypes is conceptually related to metric learning tasks, such as few-shot segmentation (FSS). However, our feature separation objective introduces significant novelties. \n\nFirstly, we introduce instance-level prototypes for OOD regions, which is the first method to construct prototype at the instance level using anomaly masks. In contrast, FSS methods generate category-level prototypes for new ID classes.\n\nMoreover, our method formulates prototype usage as an optimization problem, with the key objective of explicitly pushing OOD prototypes away from ID semantic prototypes. In contrast, prototype alignment methods [2] in FSS directly use prototypes to match query image pixels for classification, without involving any explicit optimization objective on the prototypes.\n\nAdditionally, we also provide a novel theoretical foundation of how these OOD instance-level prototypes enhances segmentation performance on small anomalies in the feature separation loss (Please see our response to Reviewer hxoy, W3), which is unique in anomaly segmentation.\n\nIn summary, the prototype construction (OOD instance vs. ID class), the objective of our prototype learning strategy (optimization vs. matching), and the theoretical basis differ fundamentally from the work in metric learning.\n\n(3) While positioned as a \"paradigm,\" contributions feel incremental over prior OOD fine-tuning works.\n\nFirstly, we provide a novel theoretical analysis of why prior OOD fine-tuning works fail to segment small anomalies, which lays the foundation of our work. Moreover, our method acts as a general framework that can be effortlessly integrated into existing OOD fine-tuning strategies. Due to its broad compatibility, the design of LNOIB is intentionally simple, ensuring it fits seamlessly into a variety of pipelines. While the method may not appear complex at first glance, it effectively fills a widening gap—improving the segmentation of small anomalies, which is a known limitation of existing works. Furthermore, integrating our simple yet effective method into existing OOD fine-tuning strategies (e.g., EM, PEBAL, and M2A) leads to notable performance gains, especially on component-level and instance-level metrics. These improvements highlight the practical value of our method in accurately detecting small anomalies—an essential factor for safety-critical applications in anomaly segmentation.\n\nFollowing your constructive suggestion, we will add a new subsection in Related Work to briefly introduce relevant methods you mentioned and clearly delineate the differences between our approach and existing strategies.\n\n**W2. More ablation results needed.**\n\nThank you for pointing out this issue! In our original paper, we primarily conduct ablation studies on SMIYC-RA across multiple models. Following your helpful suggestion, we further investigate whether the ablation findings generalize to other datasets. Specifically, we use M2A to explore the optimal choices of $M$ and $\\tau$ on two additional datasets: FS L&F and Road Anomaly.\n\nFollowing the same setup as in Figure 7, we evaluate 25 combinations of $M$ and $\\tau$ for each dataset. We present the results below.\n\nsIoU Results of M2A on FS L\\&F (the first row represents the values of $M$, and the first column stands for the values of $\\tau$):\n| |1|2|3|4|5|\n|-|-|-|-|-|-|\n|0.5|33.14|32.85|32.07|31.45|30.78\n|0.6|34.95|34.80|34.28|33.60|33.25\n|0.7|**35.85**|35.15|34.57|34.06|33.77\n|0.8|34.96|34.64|33.73|33.06|32.39\n|0.9|31.44|31.08|30.23|29.50|28.80\n\nsIoU Results of M2A on Road Anomaly (the first row represents the values of $M$, and the first column denotes the values of $\\tau$):\n| |1|2|3|4|5|\n|-|-|-|-|-|-|\n|0.5|49.43|48.81|48.17|48.02|47.56\n|0.6|50.45|50.22|49.86|48.91|48.12\n|0.7|**50.51**|50.47|50.19|49.03|48.54\n|0.8|50.18|50.03|49.49|48.31|47.27\n|0.9|48.20|47.81|46.10|45.55|45.15\n\nAs shown in the tables, the best performance on both FS L&F and Road Anomaly is achieved when $M=1$ and $\\tau=0.7$, which is consistent with our findings on SMIYC-RA. This suggests that the selected hyperparameters generalize well across different datasets.\n\nMoreover, we conduct further ablation studies to assess the effect of individual loss components, including $L_{sem}$ and $L_{near}$ (please see our response to\nReviewer vbkr, Q3-3), as well as $L_{glob}$ and $L_{ins}$ (please see our response to\nReviewer nMfG, Q3). These ablation experiments performed on FS L&F and Road Anomaly are consistent to the results on SMIYC-RA.\n\n**W3. Pixel-level metrics (AuPRC, FPR95) show marginal gains (Table 2), suggesting improvements primarily target small anomalies at the expense of broader robustness.**\n\nThank you for your valuable comment! We think that the potential expense of broader robustness may lie in two aspects: 1) The decreased performance on large anomaly; 2) The decreased performance on close-set performance.\n\nOn the one hand, our method enhances the performance on small anomalies, and this improvement does not come at the expense of performance on large anomalies—both pixel-level and component-level metrics are consistently improved, as shown in Tables 1 and 2. As large anomalies account for a significant portion of the pixel-level metrics, any performance degradation on them would lead to a sharp drop in the overall pixel-level metrics. Thus, our method maintains strong performance on large anomalies.\n\nOn the other hand, as shown in Figure 3 and Appendix D, fine-tuning with our method results in only a modest drop in closed-set semantic segmentation accuracy, which is on par with previous OOD fine-tuning approaches.\n\n**W4. Theorem 2 assumes idealized inter-class separability—unrealistic for complex urban scenes. Proofs (Appendix A) are sound but lack empirical validation of assumptions.**\n\nThank you for your insightful suggestion! Following your helpful advice, we conduct additional experiments to examine the similarity between category-wise prototypes on Cityscapes, which contains complex urban scene images. Specifically, we utilize EM (WideResNet38) and M2A (ResNet50) to compute the cosine similarity between prototypes, and the resulting average values are 0.13 and 0.09, respectively. While a few category pairs exhibit relatively high similarity—such as bus vs. train (0.31), car vs. truck (0.29), and traffic light vs. traffic sign (0.26)—the majority of category pairs show low similarity values (≤ 0.1). This indicates that, for well-trained semantic segmentation models on large-scale datasets like Cityscapes, most category-wise prototypes are fairly dissimilar. Although these category-wise prototypes are not rigorously orthogonal, this empirical observation suggests that the cosine similarity values are quite small in most cases.\n\nMoreover, Theorem 2 reveals the inherent limitation of using only $L_{sem}$ and highlights the necessity of introducing $L_{near}$. To convey this insight, we adopt an idealized assumption for theoretical illustration. For empirical validation, although the prototypes of different categories are not strictly orthogonal, those of different ID categories remain well separated in the feature space. However, in cases where two ID prototypes are highly similar, relying solely on $L_{sem}$ may cause the OOD prototype to collapse into the joint feature space of these similar ID categories. In such scenarios, $L_{sem}$ still yields a small value, failing to distinguish the OOD region effectively. Therefore, incorporating $L_{near}$ becomes crucial to explicitly enforce separation between the OOD prototype and the closely aligned ID prototypes.\n\n**Reference**\n\n[1] Cui Y, et al. Class-balanced loss based on effective number of samples. CVPR'19\n\n[2] Wang K, et al. Panet: Few-shot image semantic segmentation with prototype alignment. CVPR'19"}, {"review_id": "BrSHeGRfGi", "reviewer": "Reviewer_vbkr", "summary": "The paper introduces a novel framework called LNOIB aimed at improving the detection of out-of-distribution (OOD) instances in anomaly segmentation tasks. It addresses the limitation of existing OOD fine-tuning strategies that often fail to effectively segment small anomalies, proposing an instance-level fine-tuning approach to enhance the detection of both large and small anomalies. The framework is supported by a comprehensive theoretical analysis, which strengthens the soundness of the proposed method.", "questions": "1.Figure 4 is not sufficiently clear. Could the authors provide the ground-truth annotations for the images to facilitate a better understanding of the results? Additionally, it remains unclear whether the introduction of LNOIB leads to an increase in false positives—i.e., misclassifying in-distribution instances as out-of-distribution. More detailed analysis or supporting data is needed to address this concern.\n\n2.While the authors claim that LNOIB enhances the detection of small anomalies, the evidence presented is mainly visual. To substantiate this claim, quantitative metrics—such as performance breakdown by anomaly scale—should be provided. This would offer a more rigorous and comprehensive evaluation.\n\n3.The authors mention that solely adopting L_sem may cause the OOD prototype to align with the prototype of a specific ID category. Could the authors provide specific visual examples to illustrate this point? Additionally, could the authors provide more information about the datasets used, such as the number of ID categories (K) and the similarity between different ID representations? Providing detailed metrics on the performance drop when using L_sem and L_near alone on different datasets would further support the paper’s arguments.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n1.The paper is technically solid, offering a well-founded theoretical analysis and comprehensive experiments that aim to address key limitations of current out-of-distribution (OOD) strategies in anomaly segmentation.\n\n2.The manuscript is clearly written and well-structured, making complex concepts easier to understand and follow.\n\nWeaknesses:\n\n1.Despite providing detailed theoretical analysis, the instance-level loss proposed in the paper is not entirely novel. Similar instance-level losses have been used in other detection tasks, which limits the originality of this contribution.\n\n2.The authors claim that their feature separation objective significantly improves the detection of small anomalies, but these claims are not well-supported by experimental results or detailed analysis. While feature separation has been shown to improve model performance in other studies (e.g., [1]), the specific impact of the feature separation objective in this work remains insufficiently demonstrated. \n\n[1]Tang J, Lu H, Xu X, et al. An incremental unified framework for small defect inspection[C]//European conference on computer vision. Cham: Springer Nature Switzerland, 2024: 307-324.", "was_revised": true, "final_justification": "Most of my earlier concerns were addressed through clear theoretical analysis and strong experimental results. However, the core idea—instance-level loss and feature separation—has been explored in prior work in other domains, limiting novelty. Balancing these factors, I maintain my borderline accept recommendation.", "rebuttal": "We sincerely appreciate your encouraging feedback and valuable suggestions. We will give a detailed response to each of your insightful concerns below.\n\n**W1. Despite providing detailed theoretical analysis, the instance-level loss proposed in the paper is not entirely novel. Similar instance-level losses have been used in other detection tasks, which limits the originality of this contribution.**\n\nThank you for your insightful comment! Emphasizing small instances is intuitive when they are critical to the task. However, our proposed instance-level objective is grounded in a rigorous theoretical foundation, which sets it apart. Specifically, our analysis reveals that existing global-level objectives tend to converge toward large anomalies. Building on this insight, our instance-level objective introduces instance-specific adaptation as a general extension of global-level objectives, rather than being tied to any particular loss function, which is unique in the field of anomaly segmentation. Moreover, incorporating this objective leads to significantly better performance on small anomalies, aligning well with our theoretical findings. Therefore, the theoretical foundation (unique to anomaly segmentation), the objective formulation (not tied to a specific loss), and the application scope are fundamentally different from those in other detection tasks.\n\nFollowing your helpful comment, we will include an additional subsection in Related Work to more clearly highlight the originality of our approach.\n\n**W2. The specific impact of the feature separation objective in this work remains insufficiently demonstrated.**\n\nThank you for your constructive comment! We have gained valuable insights from paper [1], which helped us improve the experimental design for evaluating the feature separation objective. Specifically:\n\n1) We will provide t-sne visualization to demonstrate that using only $L_{sem}$ may lead to OOD prototypes aligning with the prototype of the \"road\" category (please refer to our response to Q3-1 for text description);\n\n2) We include more detailed information about the datasets, such as the number of ID categories and the similarity between some different ID semantic prototypes (please refer to our response to Q3-2);\n\n3) We report the individual performance of $L_{sem}$ and $L_{near}$ on the FS L\\&F and Road Anomaly datasets (please refer to our response to Q3-3 for experimental results).\n\n4) Besides your valuable advice, we also provide extra theoretical analysis of why using the OOD prototype-based feature separation objective improves segmenting small anomalies (please refer to our response to Reviewer hxoy, W3).\n\nThese analyses will be added in the revised version of our paper, and we will cite the reference you kindly provided, which has greatly inspired our experimental design and presentation of the feature separation objective.\n\n**Q1-1. Provide the ground-truth annotations in Figure 4.**\n\nThank you for your insightful suggestion! We strongly agree that including the ground-truth labels (ID vs. OOD) would greatly enhance the interpretability of the results shown in Figure 4. We will incorporate the corresponding ground-truth masks as an additional column appended to the end of each row, to provide a clearer visual comparison in the revised paper.\n\n**Q1-2. Whether LNOIB leads to an increase in false positives?**\n\nThank you for raising this important and interesting point! As shown in Table 2, the application of LNOIB consistently reduces the FPR$_{95}$ metric in most cases. It is worth noting that this metric reflects the pixel-level false positive rate, rather than an instance-level evaluation.\n\nObtaining reliable instance-level metrics in our setting is particularly challenging, as our task focuses on anomaly semantic segmentation, which aims to distinguish between ID and OOD regions at the pixel level. Although some prior work (e.g., [2]) proposes approximating instance-level performance for anomaly detection, extracting ID instances remains difficult. This is because ID regions can include both discrete thing classes and continuous stuff classes, whereas OOD regions are typically more spatially separated and instance-like. The ambiguity in delineating ID instances—especially for amorphous or continuous regions—makes instance-level evaluation less reliable in this context.\n\nTherefore, lower FPR$_{95}$ results in Table 2 indicate that incorporating LNOIB effectively reduces the number of false positive pixels.\n\n**Q2. Quantitative metrics—such as performance breakdown by anomaly scale—should be provided.**\n\nThank you for this very insightful suggestion! Following COCO, we define small anomaly as smaller than 32*32 pixels. We compare the performance of M2A with global-level objective and LNOIB. The results on FS L\\&F and Road Anomaly are presented below.\n\nResults on L\\&F of small anomaly:\n| |sIoU|PPV|F1*|AuPRC|FPR$_{95}$\n|----------|----------|----------|----------|----------|----------|\n|M2A|17.91|14.67|19.55|65.71|4.43|\n|M2A+LNOIB|**37.40**|**33.73**|**32.05**|**81.30**|**2.51**|\n\nResults on Road Anomaly of small anomaly:\n| |sIoU|PPV|F1*|AuPRC|FPR$_{95}$|\n|----------|----------|----------|----------|----------|----------|\n|M2A|24.61|25.05|18.39|52.69|21.36\n|M2A+LNOIB|**45.84**|**41.74**|**44.50**|**73.06**|**15.33**\n\nBased on the results above, both pixel-level and component-level metrics show notable improvements on small anomalies when using LNOIB, further supporting our main claim.\n\n**Q3-1. The authors mention that solely adopting $L_{sem}$ may cause the OOD prototype to align with the prototype of a specific ID category. Could the authors provide specific visual examples to illustrate this point?**\n\nThank you for your constructive suggestion! This is an excellent idea to further illustrate the limitations of using $L_{sem}$ alone. As we are unable to upload figures in the rebuttal phase, we try to describe our observations in detail and incorporate the analysis into the revised paper.\n\nOur observations reveal that many road anomalies tend to appear directly on the road surface. When these small anomalies share similar color or texture with the road, using only $L_{sem}$ often causes their features to be pulled into the feature space of the “road” category, with a cosine similarity exceeding 0.6. In such cases, $L_{sem}$ remains relatively small—since it is averaged across $K$ categories—but fails to effectively separate OOD features from the ID “road” class, leading to misclassification.\n\nIn the revised version, we will include a t-sne figure that visualizes the pixel-wise representations of such OOD instances, the “road” category, and other ID semantic categories. This will help clearly illustrate the limitations of relying solely on $L_{sem}$ and motivate the necessity of incorporating our proposed $L_{near}$ objective.\n\n**Q3-2. Provide more information about the datasets used, such as the number of ID categories ($K$) and the similarity between different ID representations.**\n\nThank you for your valuable comment! Our study involves a total of seven datasets: Cityscapes, MS COCO, SMIYC-RA, SMIYC-RO, FS-Static, FS L&F, and Road Anomaly.\n\nIn the first stage, we train a standard semantic segmentation model on Cityscapes, which contains 19 ID categories. This leads to 19×19=361 possible ID prototype pairings, making full visualization difficult. To provide a summary, we compute the average cosine similarity between category-wise prototypes using M2A with ResNet-50, which is 0.09. While a few category pairs show relatively high similarity—such as bus vs. train (0.31), car vs. truck (0.29), and traffic light vs. traffic sign (0.26)—the vast majority have low similarity (≤ 0.1), indicating well-separated feature representations.\n\nIn the second stage, we create synthetic OOD data by pasting MS COCO (with 80 semantic categories) instances onto Cityscapes images (excluding those overlapping with 19 Cityscapes categories). This forms the OOD fine-tuning phase, where the original Cityscapes content is treated as ID and the pasted MS COCO regions as OOD. After this phase, the average prototype similarity slightly increases to 0.10 when using the fine-tuned model to conduct close-set semantic segmentation on Cityscapes.\n\nIn the test phase, the remaining five datasets are binary-labeled (ID vs. OOD). Since they involve only a single ID class, computing prototype similarity among ID categories is no longer meaningful.\n\n**Q3-3. Provide detailed metrics on the performance drop when using $L_{sem}$ and $L_{near}$ alone on different datasets.**\n\nThank you for this insightful suggestion! As shown in Table 3(b), we have already reported the results with and without both objectives on the SMIYC-RA. To further validate the effectiveness of combining both objectives, we additionally present results of M2A on the FS L&F and Road Anomaly datasets below. \n\nResults on FS L\\&F:\n| $L_{sem}$ | $L_{near}$ | sIoU | PPV | F1* | AuPRC | FPR$_{95}$ |\n|----------|----------|----------|----------|----------|----------|----------|\n| | |26.51|19.43|23.76|89.91|1.85\n| ✓| |28.11|22.04|24.46|89.33|1.91\n| |✓|31.58|28.40|27.81|89.74|2.03\n|✓|✓|**35.85**|**33.93**|**32.10**|**90.07**|**1.72**\n\nResults on Road Anomaly:\n|$L_{sem}$|$L_{near}$|sIoU|PPV|F1*|AuPRC|FPR$_{95}$|\n|----------|----------|----------|----------|----------|----------|----------|\n| | |47.43|40.80|44.57|75.70|16.31 \n|✓| |45.39|41.17|43.08|73.36|17.21\n| |✓|47.93|44.57|45.61|76.44|16.13\n|✓|✓|**50.51**|**48.87**|**50.05**|**78.26**|**15.45**\n\nThe findings above are consistent across these datasets, confirming that the joint use of both objectives is indeed beneficial. We will add these findings in the revised paper to further support our paper’s arguments.\n\n**Reference**\n\n[1] Tang J, Lu H, Xu X, et al. An incremental unified framework for small defect inspection. ECCV'24.\n\n[2] Nekrasov A, et al. Ugains: Uncertainty guided anomaly instance segmentation. GCPR'23."}, {"review_id": "ZoWkKkZaPb", "reviewer": "Reviewer_Fsxc", "summary": "This paper aims to improve instance level anomaly segmentation (AS) by reconsidering common loss formulations. The paper observes that the commonly used global / pixel prediction objective function de-emphasizes small anomalies. Straightforward theoretical justifications are made to support their observations. Based on this observation a novel instance wise loss is added that is averaged over the number of pixels in the object and is therefore size independent providing an emphasis on detecting each instance. Next, the paper proposes prototype feature matching to emphasize the importance of small instances. The final objective is proposed with 4 subcomponents that are all weighted with 5 hyperparameters. This construction is generalizable across specific forms of loss functions. The paper then provides experimental support for their model showing improved performance on instance and component wise metrics while maintaining and sometimes improving pixel wise metrics. An ablation is also provided to weigh the contributions of each of the proposed losses.", "questions": "- What hyperparameter selection strategy or specific values should be considered for each of the loss weights? How likely is this to hinder adoption by the wider community?", "limitations": "The authors address limitations in the supplemental materials.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- The paper follows a clear logical flow of arguments and the writing is clear\n- The theoretical support provided is clear and supports the arguments made in the paper\n- Experiments support the aims set out in the construction of the loss\n\nWeaknesses:\n- There are several hyperparameters necessary for weighing each of the components of the loss. While a basic search has been provided over the set of parameters, this is an added complexity in future adoption of this multi-objective loss.\n\n*Note*: I do not closely follow the field of anomaly segmentation and so may not have identified missing reference to related anomaly segmentation methods and am not an expert in the datasets and benchmarks commonly used in this field.", "was_revised": true, "final_justification": "The author response provided additional guidelines for setting the several additional hyperparameters. I maintain my rating as the paper provides a method motivated by theoretical findings that will be interesting to the NeurIPS community.", "rebuttal": "We sincerely appreciate your encouraging feedback and valuable suggestion. We will give a detailed response to your insightful question below.\n\n**W1&Q1. What hyperparameter selection strategy or specific values should be considered for each of the loss weights? How likely is this to hinder adoption by the wider community?**\n\nThank you for raising this interesting and insightful topic! We think that although these hyperparameters may increase the complexity in future adoption to other segmentation tasks, they also provide different selections to suit the corresponding tasks. We will give an analysis of the selection recommendation of each hyperparameter below.\n\n1) Balancing Factor $\\alpha$ (Global vs. Instance-Level Objective):\nThe parameter $\\alpha$ controls the balance between the global-level and instance-level objectives. In anomaly segmentation, both large and small anomalies are critical for ensuring safety, so we set $\\alpha = 0.5$ by default. For tasks less sensitive to small objects, a larger value (e.g., $\\alpha = 0.75$) may be more appropriate. Conversely, for applications that prioritize accurate segmentation of small objects, a smaller value (e.g., $\\alpha = 0.25$) is preferable. Thus, the choice of $\\alpha$ should be guided by the task's sensitivity to object scale.\n\n2) Balancing Factor $\\beta$ ($L_{sem}$ vs. $L_{near}$):\nThe hyperparameter $\\beta$ adjusts the trade-off between the ID semantic loss ($L_{sem}$) and the nearest-neighbor loss ($L_{near}$). Both losses serve complementary roles: $L_{sem}$ encourages global separation of OOD prototype, while $L_{near}$ ensures local separation of OOD features. We recommend starting with $\\beta = 0.5$ as a balanced setting. If further tuning is needed, binary search around values like $\\beta = 0.3$ or $\\beta = 0.7$ can be used to identify task-specific optima.\n\n3) Weight $\\gamma_2$ (Feature Separation Objective):\nThe parameter $\\gamma_2$ controls the contribution of the feature separation objective (with $\\gamma_1 = 1$ fixed). As the prediction-based objective directly adopts OOD masks for optimization, this objective serves as an auxiliary mechanism to enhance the separability of OOD features in the representation space. A very large $\\gamma_2$ (e.g., $\\geq 10$) may overly emphasize this component and is generally not recommended. Conversely, very small values (e.g., $\\leq 0.01$) may render it ineffective. We suggest starting with $\\gamma_2=0.1$ and $\\gamma_2=1$, and using binary search for fine-grained explorations.\n\n4) Threshold $\\tau$ (Prototype Filtering):\nThe threshold $\\tau$ filters pixel-level features to build ID category prototypes. A larger $\\tau$ yields purer but less informative prototypes, while a smaller $\\tau$ retains more feature variation at the cost of increased noise. We recommend setting $\\tau \\in [0.5, 0.9]$ depending on task requirements. For tasks that value prototype richness and can tolerate some noise, $\\tau = 0.5$ is a reasonable starting point. For applications demanding high purity, initial value like $\\tau = 0.7$ is more suitable. Further refinement can be conducted using binary search.\n\n5) Number of Neighbors $M$ (in Nearest Neighbor Loss):\nThe parameter $M$ determines how many ID prototypes are considered when computing the nearest neighbor loss. This loss encourages OOD prototypes to stay distant from the closest ID prototypes. Setting $M$ too large may dilute this effect. We recommend starting with $M = 1$ and incrementally increasing it up to $M = 5$ to find the best value. This small range is generally sufficient for most tasks.\n\nOnce again, we sincerely appreciate your insightful suggestion! It has encouraged us to consider the broader applicability and adaptability of our method across diverse segmentation tasks."}, {"review_id": "1pt8QNSghF", "reviewer": "Reviewer_nMfG", "summary": "This paper presents a fine-tuning strategy for instance-level segmentation in images with small anomalies. The strategy is flexible, and the authors apply it to three segmentation approaches. Along with extensive benchmarks and experiments, the authors provide a theoretical analysis of why prior approaches fail at this task. Finally, some qualitative results are also provided.", "questions": "1. Does $L_{pred}$ contain $L_{in}$ twice? It contains $L_{glob}$ which contains $L_{in}$ and $L_{ins}$ which also contains $L_{in}$. I am concerned about the relative weight of the $L_{in}$ ($\\alpha$ becomes pretty meaningless) in the optimization. Is this a problem? Did you notice any increased focused on the in distribution pixels due to this if so?\n\n2. \"the selection of these metrics and datasets aligns with official guidelines\". I think the datasets and metrics seem appropriate, but what official guidelines are you referring to? Being more explicit about this may strengthen your paper and choice of these metrics and datasets.\n\n3. Are the ablation results consistent on the other datasets? Why was the SMIYC-RA chosen for the ablation? \n\n4. Does focusing on improving segmentation on small anomalies hurt segmentation on large anomalies? The quantative results don't seem to clearly distinguish between them.", "limitations": "The limitations are kept as a short paragraph in the appendix. Perhaps they should be moved to the main text of the paper.\n\nNo potential negative societal impacts are discussed. As the paper aims to improve image segmentation for OOD scenarios, these may be worth mentioning even if they are not unique to this paper.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "## Strengths\n\n- The proposed approach has strong quantitative results across multiple datasets and against numerous baselines.\n- The approach can be applied on top of existing methods to improve segmentation.\n- The paper provides a theoretical analysis of why prior approaches may not work well.\n- The paper is well written and detailed.\n\n## Weaknesses\n\n- The qualitative results are not convincing. For example, consider Figure 4. It is unclear what is ID or OOD in these images. In row 2, the small traffic cones are small anomalies. Is the large one also an anomaly? In the top row, the LNOIB method has some yellow artifact to the right of the anomaly. Does this consistently appear in images segmented using this method? Maybe demonstrating the qualitative performance on different images from the other datasets could convince the readers that the approach practically works well.\n\n- The authors presented means of 3 random seeds as the results, but the variance or confidence bounds are not included.\n\n- Please see my questions for more weaknesses and concerns\n\n## Quality\n\nThe paper is well written and the figures supplement the text well. Overall, the paper is high quality.\n\n## Clarity\n\nThe paper is detailed and the appendix is extensive. The approach is relatively simple and seems clear when considering other approaches in this sub-field.\n\n## Significance\n\nAccording to the results presented in the paper, the approach seems to improve small anomaly segmentation. This is likely significant for many CV applications. However, some of the results seem to be minor improvements with no confidence intervals provided, so it is hard to know whether there is a statistical or practical improvement in some cases (specifically in pixel-level scenarios, component-level seems more obvious).\n\n## Originality\n\nThe fine-tuning strategy and theorems presented seem novel.", "was_revised": true, "final_justification": "During the rebuttal period, the authors addressed each weakness presented in my initial review. The authors included updated tables and descriptions of figures to be included in the paper that address my main initial concerns of qualitative results, missing confidence bounds, and limited ablation. I believe this is a solid conference paper that will be interesting to the NeurIPS and anomaly segmentation communities. Therefore, I recommend acceptance.", "rebuttal": "We sincerely appreciate your encouraging feedback and insightful suggestions. We will give a detailed response to each of your constructive comments below.\n\n**W1-1.The qualitative results are not convincing. For example, consider Figure 4. It is unclear what is ID or OOD in these images. In row 2, the small traffic cones are small anomalies. Is the large one also an anomaly?**\n\nThank you for pointing out this issue! In the revised version, we will include the ground-truth annotations (ID vs. OOD) for each image shown in Figure 4 to enhance clarity. Specifically, in row 2, all the traffic cones—including the large one—are considered anomalies. This is because anomalies are defined as objects that fall outside the semantic categories of Cityscapes. Therefore, traffic cones are treated as anomalies in the evaluation datasets.\n\n**W1-2. In the top row, the LNOIB method has some yellow artifact to the right of the anomaly. Does this consistently appear in images segmented using this method?**\n\nThank you for your sharp observation! After identifying this issue, we evaluate the fine-tuned M2A model with two additional random seeds (as reported in the main results) and observe no noisy information near the anomaly in this image. We also conduct OOD fine-tuning with our method using two other seeds, and the predictions for this anomaly remain clean. Additionally, we use the original M2A model from Figure 4 to visualize results on approximately 50 images across five datasets. Most anomalies are accurately segmented, with only about 2 anomalies (each covering less than 0.5% of the image) showing relatively noisy information around the boundaries. We suspect this is due to the downsampling and upsampling processes in the network, which may randomly affect a few small anomalies. While this has minimal impact on overall performance—since most small anomalies are well handled—it highlights an interesting direction for future work. \n\nIn the revised paper, we will provide an improved illustration generated using other seeds in Figure 4. Additionally, we will include these minority imperfect cases and give a discussion of the potential reasons in the Limitations and Future Work section to highlight potential avenues for further research.\n\n**W1-3. Maybe demonstrating the qualitative performance on different images from the other datasets could convince the readers that the approach practically works well.**\n\nThank you for your insightful suggestion! We have used the M2A model to visualize results on approximately 50 images across five datasets. The results show that, with OOD fine-tuning using our method, most small anomalies are well segmented, significantly outperforming previous global-level strategies. As image uploads are not permitted during the rebuttal phase, we will include additional visualization results from FS L\\&F and Road Anomaly datasets in Figure 4 of the revised paper to better illustrate this improvement.\n\n**W2. The authors presented means of 3 random seeds as the results, but the variance or confidence bounds are not included.**\n\nThank you for highlighting this important point! We present the mean and standard deviation of our main results across three runs below.\n\nResults on component-level metrics:\n| |  | SMIYC-RA| | |SMIYC-RO| | |FS-Static| | |FS L\\&F| | |Road Anomaly| |\n|-|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|**Method**|**sIoU**|**PPV**|**F1***|**sIoU**|**PPV**|**F1***|**sIoU**|**PPV**|**F1***|**sIoU**|**PPV**|**F1***|**sIoU**|**PPV**|**F1***|\n|EM+LNOIB|58.73|47.42|43.90|50.86|58.80|48.57|44.38|37.20|37.79|34.74|29.56|23.97|35.72|33.18|27.30|\n| |(±0.26)|(±0.21)|(±0.35)|(±0.38)|(±0.22)|(±0.09)|(±0.30)|(±0.41)|(±0.25)|(±0.17)|(±0.24)|(±0.10)|(±0.18)|(±0.05)|(±0.29)\n|PEBAL+LNOIB|52.17|39.55|33.48|32.88|12.90|13.46|38.84|29.75|28.44|27.36|35.78|17.73|37.68|30.91|32.65|\n| |(±0.41)|(±0.13)|(±0.44)|(±0.30)|(±0.06)|(±0.16)|(±0.25)|(±0.19)|(±0.09)|(±0.26)|(±0.33)|(±0.14)|(±0.33)|(±0.15)|(±0.12)\n|M2A+LNOIB|63.15|57.37|60.08|61.50|73.71|70.25|52.08|44.93|42.96|39.16|38.10|34.80|53.70|51.46|50.97|\n| |(±0.48)|(±0.53)|(±0.27)|(±0.07)|(±0.51)|(±0.46)|(±0.18)|(±0.20)|(±0.30)|(±0.31)|(±0.17)|(±0.19)|(±0.38)|(±0.25)|(±0.08)\n\nResults on pixel-level metrics:\n| |SMIYC-RA| |SMIYC-RO| |FS-Static| |FS L\\&F| |Road Anomaly| |\n|-|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|**Method**|**AuPRC**|**FPR$_{95}$**|**AuPRC**|**FPR$_{95}$**|**AuPRC**|**FPR$_{95}$**|**AuPRC**|**FPR$_{95}$**|**AuPRC**|**FPR$_{95}$**|\n| EM+LNOIB|86.26|15.28|84.47|2.39|86.95|8.89|81.68|5.18|72.42|19.74\n| |(±0.47)|(±0.11)|(±0.35)|(±0.06)|(±0.21)|(±0.10)|(±0.24)|(±0.07)|(±0.68)|(±0.17)\n| PEBAL+LNOIB|61.46|22.03|31.30|18.44|83.81|4.47|64.50|3.82|65.35|22.77\n| |(±0.44)|(±0.18)|(±0.07)|(±0.14)|(±0.52)|(±0.03)|(±0.40)|(±0.11)|(±0.14)|(±0.10)\n| M2A+LNOIB|92.20|11.34|92.58|0.33|75.19|8.27|90.54|1.68|79.38|13.29\n| |(±0.11)|(±0.07)|(±0.42)|(±0.04)|(±0.55)|(±0.11)|(±0.58)|(±0.05)|(±0.22)|(±0.08)\n\nAs shown in the tables above, our method demonstrates stability across different seeds, with standard deviations typically below 1\\%. We will add these results to Tables 1 and 2 in the revised paper.\n\n**Q1. Does $L_{pred}$ contain $L_{in}$ twice? It contains $L_{glob}$ which contains $L_{in}$ and $L_{ins}$ which also contains $L_{in}$. I am concerned about the relative weight of the $L_{in}$ ($\\alpha$ becomes pretty meaningless) in the optimization. Is this a problem? Did you notice any increased focused on the in distribution pixels due to this if so?**\n\nYes, although $L_{pred}$ contains $L_{in}$ twice, this does not pose a problem. When expanding $L_{pred}$ in Eq. (6), we find that the coefficient of $L_{in}$ is 1, while the coefficients of the second terms in $L_{glob}$ (Eq. (1)) and $L_{ins}$ (Eq. (5)) — denoted as $L_{out}$ and $L_{outins}$ — are $\\alpha$ and $(1-\\alpha)$, respectively.\n\nTherefore, the ID loss $L_{in}$ remains independent of $\\alpha$, which solely controls the balance between the global-level OOD loss $L_{out}$ and the instance-level OOD loss $L_{outins}$. Moreover, the weight of $L_{in}$ is 1, and the weight of the combined OOD losses ($L_{out}+L_{outins}$) is also equal to 1 ($\\alpha+1-\\alpha$), ensuring that no excessive emphasis is placed on the ID regions. This balanced formulation keeps the contrastive objective of ID region and OOD region within the same range, enabling effective optimization for both ID and OOD regions.\n\n**Q2. Being more explicit about \"the selection of these metrics and datasets aligns with official guidelines\".**\n\nThank you for pointing out this issue! Under your kind reminder, we will use \"official metrics\" instead of \"official guidelines\" in the revised paper to better convey our intention. The datasets we use adopt official pixel-level metrics to evaluate overall performance on OOD regions, and component-level metrics to assess average accuracy across all anomalies. These official metrics have been commonly used in prior works. In contrast, the instance-level metric is a more recent addition, which we find helpful in validating our main claim on segmenting small anomalies.\n\n**Q3. Are the ablation results consistent on the other datasets? Why was the SMIYC-RA chosen for the ablation?**\n\nThank you for your constructive concern! The ablation results are consistent on other datasets such as FS L\\&F and Road Anomaly. The reason we chose SMIYC-RA for the ablation is that we think one dataset is okay and we choose the commonly used one in prior works.\n\nMotivated by your helpful concern, we conduct more ablation studies with M2A on FS L\\&F and Road Anomaly datasets, including the combination of $L_{glob}$ and $L_{ins}$ in prediction-based objective (please refer to the tables below), the combination of $L_{sem}$ and $L_{near}$ in feature separation objective (please refer to our response to Reviewer vbkr, Q3-3), as well as the hyperparameters of $\\tau$ and $M$ in in feature separation objective (please refer to our response to Reviewer qk6n, W2).\n\nResults on FS L\\&F:\n| $L_{glob}$ | $L_{ins}$ | sIoU | PPV | F1* | AuPRC | FPR$_{95}$ |\n|-|-|-|-|-|-|-|\n|✓| |26.51|19.43|23.76|89.91|1.85\n| |✓|25.18|15.69|17.22|57.30|8.97\n|✓|✓|**37.58**|**35.10**|**34.23**|**90.38**|**1.77**\n\nResults on Road Anomaly:\n|$L_{glob}$|$L_{ins}$|sIoU|PPV|F1*|AuPRC|FPR$_{95}$|\n|-|-|-|-|-|-|-|\n|✓| |47.43|40.80|44.57|75.70|16.31\n| |✓|35.80|31.68|37.39|53.04|21.54\n|✓|✓|**48.99**|**50.14**|**46.73**|**78.01**|**14.27**\n\nBased on the results obtained on the FS L&F and Road Anomaly datasets, our approach demonstrates robust ablation performances across different datasets.\n\n**Q4. Does focusing on improving segmentation on small anomalies hurt segmentation on large anomalies? The quantative results don't seem to clearly distinguish between them.**\n\nThank you for your insightful concern! We would like to clarify that the OOD fine-tuning with our proposed method does not compromise the segmentation performance on large anomalies. As evidenced in Table 2, the pixel-level metrics even improve when using LNOIB. Given that large anomalies contribute substantially to pixel-level scores, any degradation in their segmentation would have led to a noticeable drop in overall performance. The observed improvements thus indicate that segmentation on large anomalies is well preserved.\n\n**L1. The limitations are kept as a short paragraph in the appendix. Perhaps they should be moved to the main text of the paper.**\n\nThank you for your great suggestion! We will move the limitations to the main text of our paper in the revised version.\n\n**L2. No potential negative societal impacts are discussed.**\n\nThank you for your helpful comment! As to the potential negative societal impact, while our method shows significant improvements over prior approaches in autonomous driving scenarios, it still cannot guarantee 100% accuracy. Therefore, there remains a potential risk when deploying it in safety-critical applications. We will add this point to the revised paper."}, {"review_id": "Oqzonq4NHT", "reviewer": "Reviewer_hxoy", "summary": "The paper targets anomaly (OOD) segmentation in autonomous-driving datasets and argues that existing OOD fine-tuning methods optimize global pixel-wise objectives, thereby overlooking small OOD instances. The authors (1) provide a theoretical analysis showing that a dominant large anomaly can mask gradient signal for small ones, (2) propose adding an “instance-level objective” Lins that averages the loss over each OOD instance rather than over all pixels, and (3) introduce a feature-separation loss. The combined framework LNOIB is positioned as a plug-in for three existing fine-tuning methods (Entropy Maximization, PEBAL, Mask2Anomaly).", "questions": "- How sensitive is performance to imperfect instance masks?\n\n- Can LNOIB operate when anomaly masks are unavailable?\n\n- Provide mean, std over the 3 runs.", "limitations": "See weaknesses and questions.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths\n\n- Well-motivated focus on small anomalies. Simple, generic objectives that can be inserted into multiple existing pipelines.\n\n- Extensive experiments on five datasets.\n\n- No additional inference cost and tiny drop on ID classes.\n\nWeaknesses\n\n- Novelty is incremental. Instance-wise re-weighting and prototype separation are known ideas in few-shot segmentation and object re-ID; paper does not clearly situate itself w.r.t. those.\n\n- The experiments are reproduced by authors. Why do not follow the benchmark by original papers like M2A?\n\n- The bounds in the theoretical part ignore feature-separation loss. \n\n- Authors run 3 seeds but only averages are shown; no std/CI or p-values for the headline tables.\n\n- Construction of “OOD instances” relies on copy-paste from COCO with instance labels. Can LNOIB operate when anomaly masks are unavailable?", "was_revised": true, "final_justification": "Thanks for the authors' response. I believe most of my concerns are addressed. However, I would encourage the authors to state clearly why reproduced results are used in the experiment section to enhance reproducibility. Also, the results of \"imperfect instance masks\" shall be presented in the paper, which would significantly enhance the proposed methods' applicability.\n\nI maintain my score as BA", "rebuttal": "We sincerely appreciate your encouraging feedback and helpful suggestions. We will give a detailed response to each of your insightful concerns below.\n\n**W1. Novelty is incremental. Instance-wise re-weighting and prototype separation are known ideas in few-shot segmentation and object re-ID; paper does not clearly situate itself w.r.t. those.**\n\nThank you for your valuable comment! While the use of prototypes has been explored in few-shot segmentation (FSS) and object re-identification (re-ID), our method is fundamentally different from them: 1) In our work, OOD prototypes are constructed based on each individual OOD instance, rather than representing an ID semantic category (in FSS) or a specific ID object (in re-ID). This construction of OOD prototype is a fundamental innovation tailored to the unique nature of OOD detection. 2) We provide a novel theoretical foundation of why using OOD prototypes enhance small anomaly segmentation in the feature separation loss, which is unique in anomaly segmentation. Please see the response to W3. 3) In FSS and re-ID tasks, prototypes are directly used to match query image pixels for classification [1]. In contrast, our method formulates prototype usage as an optimization problem, where the key objective is to explicitly push the OOD prototypes away from the ID semantic prototypes. Although [2] in FSS aims to separate ID features, its goals and challenges are different from those in our OOD task. Our work is the first to explore the separation between OOD instance prototypes and ID category prototypes in anomaly segmentation. In summary, the prototype construction, the theoretical foundation, and the target of our feature separation strategy differ fundamentally from the work in FSS and re-ID.\n\nAs to the instance-wise re-weighting strategy, this idea naturally arises from the intuition that critical small anomalies deserve more focus. To our knowledge, this is the first work that explicitly targets small anomalies at instance-level in anomaly segmentation, supported by our rigorous theoretical foundation. We identify the limitations of previous global-level objectives and propose an extensive instance-level paradigm, not tied to any particular loss function. This is the first instance-based fine-tuning strategy designed for anomaly segmentation and it yields remarkable progress in segmenting small anomalies.\n\nFollowing your helpful comment, we will include a new subsection in Related Work to more thoroughly compare our approach with prior FSS and re-ID methods, especially in terms of prototype construction, optimization objectives, and theoretical foundations of our instance-level design for both objectives.\n\n**W2. The experiments are reproduced by authors. Why do not follow the benchmark by original papers like M2A?**\n\nThank you for pointing out this issue! As our core contribution lies in enhancing existing OOD fine-tuning methods with our instance-level objective, we believe it is reasonable to compare the results with and without this extension under the same experimental settings. Importantly, these settings strictly follow those adopted in prior works, rather than being designed by ourselves. We also note that some of the reproduced baseline results even outperform those originally reported, further confirming the robustness of our implementation.\n\n**W3. The bounds in the theoretical part ignore feature-separation loss.**\n\nThank you for pointing out this issue! Below, we provide a theoretical analysis of the feature separation loss when *globally separating each OOD pixel-wise representation from ID prototypes*. This analysis shows that, without instance-level OOD prototypes, small anomalies are likely to be ignored in the feature separation loss.\n\n$$L_{gsem}=\\frac{1}{K|\\Omega_{out}|} \\sum_{q_{c}\\in Q} \\sum_{x_{j}\\in \\Omega_{out}}cosSim(F_{x_{j}},q_{c})$$\n\n**Lemma 2.** The total loss of $L_{gsem}$ decomposes into instance-specific components:\n\n$$L_{gsem}=\\frac{1}{K|\\Omega_{out}|} \\sum_{q_{c}\\in Q} \\sum_{o_{k}\\in O} |\\Omega_{o_{k}}|\\cdot {\\mathbb{E}}_{x_j \\sim \\Omega\\_{o\\_{k}}} cosSim(F\\_{x_j},q_c) \\triangleq \\sum\\_{k=1}^{N} r\\_{k}L\\_{gsem,o\\_{k}}$$\n\nwhere $r_{k}=\\frac{|\\Omega_{o_{k}}|}{|\\Omega_{out}|}$ is the normalized weight of instance, and $L\\_{gsem,o\\_{k}}=\\frac{1}{K} \\mathbb{E}\\_{x\\_j \\sim \\Omega\\_{o\\_{k}}} \\sum\\_{q\\_{c} \\in Q}  cosSim(F\\_{x_j},q\\_c)$ is the mean ID semantic loss over $\\Omega\\_{o\\_{k}}$.\n\nProof:\n\n$$L_{gsem}=\\frac{1}{K|\\Omega_{out}|} \\sum_{q_{c}\\in Q} \\sum_{x_{j}\\in \\Omega_{out}}cosSim(F_{x_{j}},q_{c})=\\frac{1}{K|\\Omega_{out}|} \\sum_{q_{c}\\in Q} \\sum_{o_{k}\\in O}\\sum_{x_{j}\\in o_{k}}cosSim(F_{x_{j}},q_{c})=\\frac{1}{K|\\Omega_{out}|} \\sum_{q_{c}\\in Q} \\sum_{o_{k}\\in O} |\\Omega_{o_{k}}|\\cdot {\\mathbb{E}}_{x_j \\sim \\Omega\\_{o\\_{k}}} cosSim(F\\_{x_j},q_c)$$\n\nBased on the definition of $L_{gsem,o_{k}}$, we have:\n\n$$L_{gsem}=\\frac{1}{|\\Omega_{out}|} \\sum_{o_{k}\\in O} |\\Omega_{o_k}| \\cdot L_{gsem,o_k} \\quad\\square$$\n\n**Theorem 3.** If there exists $t\\in\\{1,2,...,N\\}$, such that $|\\Omega_{t}|\\ge (1-\\epsilon)|\\Omega_{out}|$ for some $\\epsilon \\in(0,1)$, then $L\\_{gsem}$ is bounded by $(1-\\epsilon)L\\_{gsem,o\\_{t}}\\le L\\_{gsem} \\le (1-\\epsilon)L\\_{gsem,o\\_{t}}+\\epsilon \\cdot \\max\\_{k\\ne t} L\\_{gsem,o\\_{k}}$.\n\nProof: The proof follows a reasoning analogous to that of Theorem 1.\n\n**Corollary 2.** If $\\frac{|\\Omega_{t}|}{|\\Omega_{out}|}\\rightarrow 1$, then $L_{gsem}\\rightarrow L_{gsem,o_{t}}$ in probability.\n\nProof: This proof mirrors the approach taken in Corollary 1.\n\nBased on the theoretical analysis above, globally separating each OOD feature from ID prototypes also overlooks the optimization for small anomalies, thus our ID semantic objective $L_{sem}$ in Eq. (9) is more reasonable, which equally optimizes each anomaly. Moreover, the theory of using $L_{near}$ is the same.\n\n**W4&Q3. Authors run 3 seeds but only averages are shown; no std/CI or p-values for the headline tables.**\n\nThank you for this great suggestion! We provide the mean and std of our results in Tables 1 and 2 below.\n\nResults on component-level metrics:\n| | | SMIYC-RA| | |SMIYC-RO| | |FS-Static| | |FS L\\&F| | |Road Anomaly| |\n|-|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|**Method**|**sIoU**|**PPV**|**F1***|**sIoU**|**PPV**|**F1***|**sIoU**|**PPV**|**F1***|**sIoU**|**PPV**|**F1***|**sIoU**|**PPV**|**F1***|\n|EM+LNOIB|58.73|47.42|43.90|50.86|58.80|48.57|44.38|37.20|37.79|34.74|29.56|23.97|35.72|33.18|27.30|\n| |(±0.26)|(±0.21)|(±0.35)|(±0.38)|(±0.22)|(±0.09)|(±0.30)|(±0.41)|(±0.25)|(±0.17)|(±0.24)|(±0.10)|(±0.18)|(±0.05)|(±0.29)\n|PEBAL+LNOIB|52.17|39.55|33.48|32.88|12.90|13.46|38.84|29.75|28.44|27.36|35.78|17.73|37.68|30.91|32.65|\n| |(±0.41)|(±0.13)|(±0.44)|(±0.30)|(±0.06)|(±0.16)|(±0.25)|(±0.19)|(±0.09)|(±0.26)|(±0.33)|(±0.14)|(±0.33)|(±0.15)|(±0.12)\n|M2A+LNOIB|63.15|57.37|60.08|61.50|73.71|70.25|52.08|44.93|42.96|39.16|38.10|34.80|53.70|51.46|50.97|\n| |(±0.48)|(±0.53)|(±0.27)|(±0.07)|(±0.51)|(±0.46)|(±0.18)|(±0.20)|(±0.30)|(±0.31)|(±0.17)|(±0.19)|(±0.38)|(±0.25)|(±0.08)\n\nResults on pixel-level metrics:\n| |SMIYC-RA| |SMIYC-RO| |FS-Static| |FS L\\&F| |Road Anomaly| |\n|-|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|**Method**|**AuPRC**|**FPR$_{95}$**|**AuPRC**|**FPR$_{95}$**|**AuPRC**|**FPR$_{95}$**|**AuPRC**|**FPR$_{95}$**|**AuPRC**|**FPR$_{95}$**|\n|EM+LNOIB|86.26|15.28|84.47|2.39|86.95|8.89|81.68|5.18|72.42|19.74\n| |(±0.47)|(±0.11)|(±0.35)|(±0.06)|(±0.21)|(±0.10)|(±0.24)|(±0.07)|(±0.68)|(±0.17)\n|PEBAL+LNOIB|61.46|22.03|31.30|18.44|83.81|4.47|64.50|3.82|65.35|22.77\n| |(±0.44)|(±0.18)|(±0.07)|(±0.14)|(±0.52)|(±0.03)|(±0.40)|(±0.11)|(±0.14)|(±0.10)\n|M2A+LNOIB|92.20|11.34|92.58|0.33|75.19|8.27|90.54|1.68|79.38|13.29\n| |(±0.11)|(±0.07)|(±0.42)|(±0.04)|(±0.55)|(±0.11)|(±0.58)|(±0.05)|(±0.22)|(±0.08)\n\nBased on the mean and std across three runs, we find that the std remains consistently low—typically within 1% of the mean—indicating that our method exhibits stable performance across different runs.\n\n**W5&Q2. Can LNOIB operate when anomaly masks are unavailable?**\n\nThank you for raising this important point! LNOIB cannot function without anomaly masks. This is because the instance-level objective in Eq. (5) relies on the presence of anomaly masks to identify OOD instances. Similarly, the feature separation loss in Eq. (13) depends on OOD prototypes, which are constructed based on the anomaly masks in Eq. (7). Please note that, the copy-paste method is easy and common in previous OOD fine-tuning methods to generate mix-content images, while they treat all anomalies as the whole OOD region.\n\n**Q1. How sensitive is performance to imperfect instance masks?**\n\nThank you for raising this interesting topic! To simulate imperfect instance masks, we replace the GT masks in COCO with the predicted masks from Mask R-CNN [3] (34.7 AP), which tend to have coarse boundaries. Using these masks, we apply the same copy-paste strategy to create mix-content images. Experimental results of M2A on FS L\\&F and Road Anomaly are presented below, showing that while using imperfect masks leads to a slight performance drop compared to GT masks, our OOD fine-tuning method remains effective. We attribute this to the fact that the fine-tuning mainly captures the overall OOD patterns. Imperfect masks may introduce noise into the OOD instance-level prototypes and thus slightly weaken the feature separation objective. But the impact is limited, showcasing the robustness of our method.\n\nResults on FS L\\&F:\n|Mask Type|sIoU|PPV|F1*|AuPRC|FPR$_{95}$|\n|-|-|-|-|-|-|\n|GT|39.16|38.10|34.80|90.54|1.68 \n|Mask-RCNN|38.39|36.60|33.86|88.73|1.91\n\nResults on Road Anomaly:\n|Mask Type|sIoU|PPV|F1*|AuPRC|FPR$_{95}$|\n|-|-|-|-|-|-|\n|GT|53.70|51.46|50.97|75.70|16.31 \n|Mask-RCNN|50.58|51.07|47.92|75.37|16.70\n\n**Reference**\n\n[1] Wang K, et al. Panet: Few-shot image semantic segmentation with prototype alignment. CVPR'19\n\n[2] Okazawa A. Interclass prototype relation for few-shot segmentation. ECCV'22\n\n[3] He K, et al. Mask r-cnn. ICCV'17"}]

Author rebuttal:

Meta-review: This work addresses the problem of out-of-distribution (OOD) fine-tuning for anomaly segmentation. To improve performance, especially on small anomalies where global-level objective strategies often fail, the authors propose an instance-level OOD fine-tuning framework, LNOIB. They provide theoretical justification for why global-level strategies are insufficient and argue that instance-level approaches are more effective. In addition, they introduce a feature separation objective that explicitly constrains the representation of anomalies. Experimental results demonstrate that the proposed method achieves strong performance compared to baseline approaches.

Most reviewers are in favor of accepting this work, with others leaning toward borderline acceptance. Please revise the final version according to your rebuttal. Overall, I recommend acceptance of this work.
Decision: Accept (poster)