OVID: Open-Vocabulary Intrusion Detection
Authors: Fujun Han
Jingqi Ye
Chenglong Zhang
Peng Ye
Venue: ICLR 2026 Poster (2026)
Keywords: Open-Vocabulary Intrusion Detection
Datasets
Framework
Strategy
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Various vision intrusion detection models have achieved great success in many scenarios, e.g., autonomous driving, intelligent monitoring, and security, etc. However, their reliance on pre-defined classes limits their applicability in open-world intrusion detection scenarios. To remedy these, we introduce the Open-Vocabulary Intrusion Detection (OVID) project for the first time. Specifically, we first develop a novel dataset, Cityintrusion-OpenV for OVID,  with more diverse intrusion categories and corresponding text prompts. Then, we design a multi-modal, multi-task, and end-to-end open-vocabulary intrusion detection framework named OVIDNet. It achieves open-world intrusion detection via aligning visual features with language embeddings. Further, two simple yet effective strategies are proposed to improve the generalization and performance of this specific task: (1) A Multi-Distributed Noise Mixing strategy is introduced to enhance the location information of unknown and unseen categories. (2) A Dynamic Memory-Gated module is designed to capture the contextual information under complex scenarios. Finally, comprehensive experiments and comparisons are conducted on multiple dominant datasets, e.g., COCO, Cityscape, Foggy-Cityscape, and Cityintrusion-OpenV. Besides, we also evaluate the universal applicability of our model in real scenarios. The results show that our method can outperform other classic and promising methods, and reach strong performance even under task-specific transfer and zero-shot settings, demonstrating its high practicality.

Published as a conference paper at ICLR 2026
OVID: OPEN- VEE-NVOCABULARY INTRUSION DETECTION
Fujun Han!.2, Jingqi Yel.3, Chenglong Zhang Peng Yel.
Shanghai Artificial Intelligence Laboratory
2 School of Data Science, The Chinese University of Hong Kong. Shenzhen
3 University of Science and Technology of China, 4 The Chinese University of Hong Kong han fujuuun cuhk. edu.
ABSTRACT
Various vision intrusion detection models have achieved great success in many scenarios, e,g,, autonomous driving, intelligent monitoring and security. However, their reliance on pre-defined classes limits their upplicability in open-world intru- sion detection scenarios. To remedy these, introduce the Open- Vocabulary Intrusion Detection (OVID) project for the first time. Specifically, we first de- velop a novel dataset named Cityintrusion-Openn for OVID,, with more diverse intrusion categories and corresponding text prompts. Then, we design a multi- modal, multi-task, and end-to-end open-vocabulary intrusion detection framework named OVIDNet. It achieves open-world intrusion detection via aligning visual features with language embeddings. Further, two simple yet effective strategies are proposed to improve the generalization and performance of this specific task: (1) A Multi-Distribute Noise Mixing strategy is introduced to enhance the lo- cation information of unknown and unseen categories. (2) A Dynamic Memory- Gated module is designed to capture the contextual information under complex scenarios. Finally, comprehensive experiments and comparisons conducted and on multiple dominant datasets, egg., COCO, Cityscape, applicability of our Cityintrusion-OpennV Besides, also evaluate the universal model in real scenarios. The results show that our method can outperform other classic and promising methods, and reach strong performance even under task- specific transfer and zero-shot settings, demonstrating its high practicality.
INTRODUCTION
Vision-basee intrusion detection tasks have numerous applications in life, e.g., security, intelligent monitoring, and autonomous driving (Ye et al. 2024 Han & Ye 2025 ei al. 2025b ab. Intrusion detection aims to determine whether potential objects go into specific restricted Area-of- (Aol) (Sun al., 20200 [Shi et al., 2022). Based on whether the camera is moving or not, intrusion detection tasks can be divided into static and dynamic intrusion detection. Static intrusion detec- tion relatively easy due to the fixed Aol and achieves great progress by some promising strate- gies, e.g., Histogram of Oriented Gradients (HOG) (Zhang et al. 2015), Conditional Random Field (CRF) (Matern et al. 2013), Adaptive Background Subtraction (ABS) (Stauffer & Grimson| 20000). However, static intrusion detection not meet the requirements of real-time and accuracy under dynamic scenes. Fortunately, with the continuous development of computer vision, some promising detection and segmentation frameworks are proposed (Wang et al., 2023 Chen al. 2018), which provide new schemes and paradigms for solving the problem of eynamic-view intrusion detection, i... based on overlapping pixel points between objects and Aol (Sun et al. 2020: Shi et al. 2022). Nevertheless, these proposed models can only detect single intrusion category, i.e., Pedestrians. Considering the lack of sufficient practicality, MF-ID (Han et al. (024b) and MMID-bench (Han) et al. (024c) propose the first concept and task of multi-category and multi-domains intrusion de- tection, which successfully extends the intrusion categories and domains, i.e., 1-:4. Meanwhile, Ada-iD (Han et al. 2024a) proposes
active domain adaptation intrusion detection method to further improve the performance of intrusion detection in adverse environments. Although these promising works extend the intrusion categories and scenarios by effective strategies, egg., Unsuper-
"oorrsponning Author

---

Published as a conference paper at ICLR 2026
vised Domain Adaptation, Active Domain Adaptation, their reliance pre-defined object classes limits their applicability in Open- Weeed intrusion detection scenarios, as shown in Figure A pre- defined intrusion detection framework can only detect specific or labeled categories in training sets and can not detect categories unseen and undefined, which severely limits the practicality of intru- sion detection. For some unseen intrusion categories, e-g. Car Truck, the previous models can not give correct intrusion detection results. To address this problem, we propose the Open Vocabulary Intrusion Detection, namely OVID, detect unseen and undefined intrusion categories effectively.
To accomplish the above OVID task, the greatest difficulty that there is still a lack of relevant datasets. Currently, some promising datasets, e.g., ImageNet (Deng et al.) 2009), COCO (Lin et al.] 2014) and specific autonomous driv- ing datasets, e.g., Cityscapes (Cordts) let al. 2016b), BDDI00K (Yu et al. 2020) are proposed. These datasets not suitable for our intrusion detection task due to the lack of intrusion la- bels. Fortunately, in recent works, some encouraging intrusion detection datasets have been proposed. e.g., Cityintru- sion (Sun et al., 2020), Cityintrusion- Multicategory (Han et al.) 2024b). and Multi Multi-Category Datasets (Han et al. 2024c). These datasets pro- vide multi categoy, multi-domain and intrusion labels ('N/"Y'), i.e., 'N' and 'Y' d'denotes No-intrusion and Intrusion. Although these datasets contain multi- ple categories and domains, they can not meet the requirements of the OVID task. On one hand, these datasets still lack lots of common intrusion categories, e.g., Car, Bus, Truck. On the other hand, these datasets solely contain image labels without matching text labels, which impairs their practicality in open-world intrusion detec- tion where the model is required to generalize to new, unseen objects. To this end, we propose an extensive and comprehensive intrusion detection dataset, Eltyintrusion-Openl for the OVID task.
(a) Singlecategary titessos drinction
M.. Desction
Unowder
Dovedle
Ethoot
PIDNA /CnesttiDNa
Bnadee
Deceder
Thire
MMiLl-brach / MD--D
Tci Upen-rocaisiaar istzaina deliction
becoker
Mak
Doveler
Text
Pertin
Ridee
Sei Bal
Mt
Massccck
Figure 1: Workflow comparisons of different intru- sion detection methods. Here, (a), (b), and (c) denote the Single-category, Multi-category, and proposed Open- vocabulary intrusion detection paradigms, respectively. "?' denotes the missed detection (False Negative). We can find that previous works can only detect the pre- defined intrusion category; framework can detect more categories correctly, which demonstrates the valid- ity of our paradigm.
The second difficulty is that there is still lack of an effective and efficient open-vocabulary intrusion detection framework. Although some promising multi-task intrusion detection frameworks (Sun) et al. 2020 Han et al., 2024b) and open-vocabulary detectors are proposed (Kim et al.] 2023) Yao iet al. 2023), these detectors still can not meet the requirements of the OVID task. The main reason is that the former is constrained by limited detection categories, and the latter can only perform detection task. Inspired by promising works (Han et al.] 2024b), we propose an effective, multi- modal, multi-task, and end-to-end open vocabulary intrusion detection framework, OVIDNet, to accomplish the task. The input of OVIDNet contains two different modalities, images and text. Subsequently, these inputs are sent to encoders extract features, then decoded and predicted using the decoder. In the decoder, two simple yet effective strategies are designed to boost the performance of OVIDNet: A Multi-Distributed Noise Mixing strategy is introduced to enhance the location information of unknown and unseen categories. 2) A Dynamic Memory-Gated module is designed to capture the contextual information in complex scenarios. Finally, the intrusion detection results are determined jointly by the upper and lower branches.
In summary, our contributions are as follows: (1) Novel task and dataset. To the best of our knowledge, the task of dynamic-view Open Vocabulary Intrusion Detection is proposed for the first time. This is the first multi-modal try in the vision-based intrusion task. A new benchmark, in- cluding a dataset called Cityintrusion-Operr Effective design and strategy. An effective, multi-modal, multi-task, and end-to-end framework, OVIDNet, is designed as strong baseline for this new benchmark. Besides, two effective strategies are proposed to improve the generalization and performance of the framework, including the Multi- Distributed Noise Mixing Strategy and the Dynamic Memory- Gated module. (3) Adequate exper-
and some strong baselines, is given for this task. (2)
2

---

Published as a conference paper at ICLR 2026
iments and strong results. Comprchensive experiments and comparisons are conducted verify the effectiveness of the proposed framework and methods. The results show that our framework not only reaches the current SOTA level but also maintains strong high practicality with task-specific transfer and zero-shot prediction abilities.
2 RELATED WORKS
Traditional Vision-based Intrusion Detection, Intrusion detection can be divided into static and dynamic intrusion detection. Static intrusion detection has been explored in detail due its sim- plicity, e.g., Adaptive Background Subtraction (Stauffer & Grimson 2000), Histogram of Oriented Gradients (Zhang et al. 2015). However, static intrusion detection does not meet the requirements of dynamic intrusion detection. The primary reason is that images captured by cameras can change any time, which imposes higher real-time and accuracy requirements. Fortunately, with the rapid development of computer vision, some promising works are proposed to solve the dynamic-view intrusion detection, egg., PIDNet (Sun et al., 2020), Cross-PIDNet (Shi et al., 2022). However, the practice of these works is limited due to the single category of intrusion. To compensate for the lim- itations, some encouraging works are designed to border the intrusion category, e.g., MF-ID (Han) et al. 2024b), MMID- (Han al. 2024c). Although intrusion detection has expanded signif- icantly, the intrusion category remains fixed and single, which seriously hinders the practicality in the real world. To address these, propose an open vocabulary intrusion detection task (OVID) for the first time, to solve the problem with limited intrusion categories and improve its practicality.
Open- Vocabulary Detection. Open-vocabulary detection (OVD) aims detect unseen classes in the training stage in a zero-shot manner. Some promising open vocabulary detectors are designed to solve the OVD task, e.g., Grounding DINO (Liu al.) 2024), YOLO-worid (Cheng et al. (202). and achieve excellent performance on real-world detection tasks. However, these OVD works are not suitable for our intrusion detection task. A major reason that our OVID task is a multi-task with detection and segmentation simultaneously, not only a single detection task. OpenSeeD (Zhang et al. 2023) proposes a new framework for joint detection and segmentation, but this framework cannot meet the requirements of our OVID task due to a lack of intrusion judgment capability. For this reason, we propose a new framework, OVIDNet, to meet the needs of our OVID task.
3 SYSTEMATIC DATASETS
To compensate for the lack of richness in the categories of intrusion detection datasets, we de- velop an Open- Vocabulary Intrusion Detection dataset, namely Cityintrusion-Openn. for the first time. The detailed automatic generation method of the proposed datasets is presented in Appendix A.1 Our new dataset is established on the promising Cityscape (Cordts et al. 2016b). Inspired by promising work MMID-bench (Han et al., (024c), our new dataset also contains multi-categories and multi-domains. Differently, proposed datasets have more intrusion categories, not 4 categories in Multi-Domain Multi-Category datasets, but all potential/possible 8 intrusion categories in the
(2) Original (CVPR'16) Intrusion Categories: None
(b) PIDNe (ACM MM # Intrusion Categorie:
(6) MF-ID' (T-ASE224) Latrusion Categories: 4
(d) MMID 24) Intrusion Categories
( Clus (Ciniamiiin-QpaaV)) # Intrusion Categaries.
Figure 2: The visualization comparison between our datasets and other promising intrusion detection datasets. I and denote fine-grained and multiple domains, respectively.
3

---

Published as a conference paper at ICLR 2026
eityscape dataset. Following some promising works (Han et al. 2024c). the detail Intrusion( (YY))/oo intrusion( (N') labels are provided by Automated Label Processes. And the judgment threshold is also set to 20. To demonstrate the superiority of our dataset, first present some visualization comparison, as shown in Figure 2
We can observe that our datasets provide the correct intrusion and no-intrusion labels and have richer categories of intrusions.
Then, we compare the quantitative results be- tween the proposed Cityintrusion-Openll and other promising intrusion detection datasets, as shown in Table We can find that our dataset contains more intrusion categories and has more sufficient 'Y'/'N' cases per image (18.03 cases per image in the whole dataset), which significantly improves the intrusion de- tection dataset richness ( about 2x up compared to others). More detailed information, data statistics, and more visualization comparisons are presented in Appendix A
Table 1:
The comparison between previous intru- sion detection datasets and our datasets. i denotes multiple domains.
Ciapienni Detedier Derset Nares
Categrics
"Y/N''
Clisitnoiin Maltiaineo- Abaas Derram Maate-Cragpey
5431/23588
9.59
94330220977
M.99
Oura (Chrernann
2475357799
1830
4 THE PROPOSED FRAMEWORK AND METHODS
4.1 PRELIMINARY
In the proposed OVID task, given specific object detection dataset Dd D ( M;),
and segmentation dataset D", D* {(r, M;) where I and E denotes the detection and segmentation samples, M M; denotes the corresponding labels. For My, we usually use four bounding box labels (b) and a category label (c*) to express it, b E R cd E Ca. Cd denotes the category space of detection dataset (D"). For M., we usually use fine labeling (assigning a category ea to each pixel in the sample If), c* C*, C* denotes the category space of segmentation dataset (D*). Besides, we divide the detection and segmentation dataset into (D Dj,,, (D D DJ. DE denotes the training and validation of detection dataset. Dj- D denotes the training and validation of segmentation dataset. Following the Open- Vocabulary detection and segmentation paradigms, we can express the training and validation category C% and Cy, where C# is base categories and Cf CV. The category space of the new categories in CV named as CN,. and CN =C C7 * Ø. The same is true for the segmentation dataset. For I, we the text Encoder (ET) convert text information into text embeddings, i.e., FT E (Text) and use the image Encoder (E') to extra the image feature, i.e.FF' = E (Img). All features sent a Decoder (D) for decoding. Finally, the segmentation Aol and bonding box are extracted to calculate the intrusion results and express it as
I, = J (
(FT F+) S (Box², }
(Box)AAo))}
(1)
where I, denotes the final intrusion results. J denotes the intrusion judgment module. - denotes the extract two key information from the Decoder (D). Box?, Auif denote the prediction of the bounding box and Aol, respectively.
4.2 OVERALL FRAMEWORK
In this section, we introduce the proposed OVIDNet, as shown in Figure 3 We improve the original OpenSeeD (Zhang et al. 2023) to make more suitable for our OVID task. Firstly, we use two different encoders, text and image encoders, to extract features for text prompts and images. The features are sent to the decoder. In the decoder, we design two effect methods to improve the generalization of the proposed framework: Multi-Distribtedd Noise Mixing and Dynamic Memory- Gated Module, respectively. Finally, we extract detection results and the segmentation results in the decoder to calculate the overlapping pixel values and determine whether it constitutes an intrusion. Once the overlapping pixel value exceeds a certain threshold (t), a warning ('Y/N, Class') ) will be added to the detected intruder surround box. Note that 'Y', 'N', and 'Class' denote the Intrusion, Non-intrusion and Class name (Here, class abbreviations are used instead of complete labels to easily show results. The detailed correspondence between the two is shown in Appendix A.4
4

---

Published as a conference paper at ICLR 2026
OVID: Opea--ooa Intrusion Detection
Intrasion categories
person' "bus".. ....
Feature Extraction
Intrusion judgment
thing classes
CLIP
Multi-
Bounding
Distributed Nuise Mising
cleoss
lest Fincader
Text Embeeding
Intrasion
(stressl, "road" ..)
Swin transSormer
Dynamic
Intrusion resalt
Mediule
Original images
Imaga
Multizccale
Mask (Aol)
Multi-Distrihtte Noiae Mixing
Dyramic Memory--Gatol Medille
Adjisted
Uniform
Seatures
Synamic Misbale
Original Bbox
Combination noise
Finhanced Blox B,
Menooy Networ
Gaussian noise
Mistniry uimi
N(-N(
Siimitaitt manching
Mensory oulpul
Concal
N.
Laplace noise
X-1.00.1)
Weelliing
IxTConv
Bcc-Hle
Add
judgment
product
connected
Sigmold
The overall framework and pipeline of our proposed OVIDNet. The input OVIDNet consists of two different modalities: Text and Images. The text includes some customizable and common intrusion categories and scenarios. The image denotes the corresponding original images. Then, the text and images sent to different encoders extract features, i.e., clip and tiny-swin- transformer, respectively. These features will be sent to the decoder for decoding and prediction. In the encoder, we design a multi-distributed noise mixing strategy and dynamic memory- gaaeer module to enhance generalization in open scenarios. Finally, we extract the predicted bounding box and predicted Aol mask to calculate the overlapping pixels and give the final intrusion results. Once the overlapping pixels are greater than the threshold (f), it will be judged as an intrusion. Otherwise, will be judged as no-intrusion. We use abbreviations to represent the full name better express the intrusion results. The detailed correspondence can be found in Appendix A.4
Figure 3:
4.3 MULTI-DISTRIBUTED NOISE MIXING STRATEGY
In the original OpenSeeD model, noise generation methods usually use uniform noise distribution and fixed percentage of noise dynamics. Therefore, can express
B = C (B. N, O A ② 0, 1).
where B, denotes the set the bounding box, B, can expressed by center point (x, y) and width, height (w, h). A denotes the range of the disturbance and A { 1 h). T is constant noise scaling factor. N, denotes the random distribution and N, 2 1). @ denotes the element- wise product. C denotes that all value clamped between and 1. However, in the real world, this method not adapt dynamic environments and scenarios, e.g., different sizes and changing objects, and challenging intrusion scenarios. Therefore, to address this issuc, we propose a new Muulti-Distribute Noise Mixing Strategy, shown in Equation 3 The idea of the proposed Multi- Distributed Noise Mixing Strategy is very simple yet effective. When confronted with complex dynamic environments, models need cope with the variations of different targets and scenarios. Specifically, for tiny objects, fine-grained perturbations are used to preserve their detailed features. Meanwhile, large-scale perturbations strengthen the global features for large objects.
B c (B. + (a No, + + N, + + N,) BAAG, 1).
(3)
where N., U (-1, N, (0, and N; (O, 1). L denotes the Laplace distribution. a, 8 and 7 is the coefficient of U, N. and L distributions, respectively. Note that 3 1. 0 denotes the proposed dynamic varying noise ratio based on the detection area of the bounding box, and θ r (1 + log(1 + A)). A denotes the of the bounding box and and are defined as the same as the Equation 2 Our detailed algorithm is shown Algorithm and the mechanism proof of the Multi- Multi-Distributed Noise Mixing strategy is shown in Appendix
h. Besides, C
5

---

Published as a conference paper at ICLR 2026
Algorithm Multi-Distributee Noise Mixing Strategy
Require: Bounding box parameters Be, Noise scale 7; Uniform noise weight a, Gaussian noise weight 3, Laplace noise weight 7
Ensure: Augmented bounding box parameters By-
1: E> D is tensor of the same shape as Be
2: Initialize <-
3: >> Compute the area of each bounding box
4: A <- [:, 2] Be [t. 3:.
5: D Compute dynamic noise scale for each box
6: - T (1 + log A))
7: >>Dffine perturbation directions for center and size
8: A l:: < B
[. 2:2/2
Perturb center Perturb width and height
9: A [
B. 2
10: >> Generate noise from multiple distributions
N. ~ Ng N (0, and N (0,
12: >> Compute the weighted combination of noise
Nm <a N₂ 3 Ng gg + N₁
14: >> Add scaled noise to bounding box parameters
B, < <BB
(Nm O & ))
16: >> Clamp augmented bounding boxes to the valid range
17: B, + -Cammp (By,
18: return By
4.4 DYNAMIC MEMORY-GATED MODULE
To address the challenges of insufficient long-ter -erg-termmo dependency modeling and poor dynamic scene adaptation in our OVID task, we propose Dynamic Memory-Gated Module. Given input feature BBCCxX we first use global average pooling (GAP) to extract a global context query vector ( BxC), express it as GAP(X), where B, C, H and W denotes the batch_size, channel, height and width. Then, we introduce a dynamic memory retrieval module and express it as
QM Vd
On softmax
My.
(4)
where Q E RBxC denotes query vector. MÅ E BBC denotes the memory key, and is the number of memory units. My E RMMxC denotes the memory value, and My is used to store the feature information corresponding to the key. Om E B denotes the memory output by retrieving. Finally, retrieved memory output (0m) and input features (X) fused by concatenation and 1xl Conv. Therefore, we can express this principle as
(5) where X, denotes the fusion feature. W denotes the generate dynamic weights and W (W2 (W..U( ())- Wi, W₂ denotes the weight of fully connected networks. Wi RCxd
X, Conv Ix1 (ConcattX @ W, Wm)))
W2 E R denotes the sigmoid function.
5 EXPERIMENTS AND ANALYSES
5.1 EXPERIMENTAL SETTINGS
Implementation Details. We conduct all experiments on computer with NVIDIA GeForce RTX 2080Ti GPUs. Unless specified, the Max.Iter, Batch_size.total, CHECKPOINT.PERIOW
EVAL.PERIOD of all experiments are set to 15000, 8, 15000, and 15000, respectively. The image encoder and text encoder adopt we-sswi--traaffommn (Liu et al. 2021) and Clip (Radford et al. 2021 respectively. More hyperparameter details can be found in Appendix
Datasets. Our experiments conducted in some publiy datasets, e.g., COCO Lin et al. 2014
Citytscape (Cordis et al. 2016a), Foggy- Setyy-Cityscape (Sakaridis et al.] (018)) and Cityintrusion Open's V. In addition, provide more visualization results, also test and report visualization demo results

---

Published as a conference paper at ICLR 2026
on other datasets, e.g., the ShanghaiTech Campus dataset (Luo et al. 2017). and the UA-DETRAC (Wen et al. 2020). Note that our experiment, we adopt two manners, i.e. zero-shot and task- specific transfer, to evaluate the performance of the model.
Metries. In order to report the quantitative results of our experiments more comprchensively, in- spired by some previous promising work (Han al. 2024c|b), b), the mlOU(%) and mAP(%) utilized to evaluate the sub-task performance of segmentation and object detection. For the in- trusion detection performance, we also use three different intrusion detection metrics: AccY(%) AccN(%), and Ac)(%) to quantify. Besides, some additional metrics, e.g. panoptic segmentation metries, PQ(%). (Q,%), RQ(%). and every AP(%), AP@ .5%) of intrusion categories, are reported to evaluate the zero-shot performance of the model.
Comparison Models. We compare with the OpenSeeD (Zhang et al. 2023) because of its promising multi-task capability and performance in open- veca-vocayllary tasks. The multi-task feature is consistent with our task. Besides, we also compare the latest intrusion works, e.g., PIDNet (Sun et al. 2020). Cross-PIDNer (Shi et al. 2022). MMID-bench (Han et al. 2024c), MF-ID (Han et al. 2024b).
5.2 MAIN RESULTS
Compared with promising open-vocabulary works. We first compare the multiple performances with the promising OpenSeeD model and report three zero-shot detection performances, i.e., PQ(%), SQ(%), RQ(%), and three task-specific transfer intrusion performances, i.e., AccY( AccN(%), Ac)(%), shown in Table We find that in different tasks, for the panoptic segmentation performance (PQ), compared with OpenSeeD, our methods can improve it by 2.19% and 1.12%, respectively. Besides, for intrusion detection performance (Acc), our model surpass by 3.43% and 3.45%, % respectively, which verifies the effectiveness of the proposed model and strategies.
Table 2: The zero-shot and task-specific transfer comparison results between promising multi-task
open-vocabulary work and OVIDNet in different datasets. More results are shown in Appendix D
Zeno-shot Detection (Panuptic segmentation)
Task-specific Transfer (Intrusion delection)
Model
Test data
RQ(%)
SQ(%)
PQ(%)
Test data 2
AccY(S))
AccN(%)
Acc (b)
Cityscape
18.22
43.68
14.03
Ours (Normal)
18.72
36.19
29.36
OpenSecD
Foggy-Cityscape
18.07
36.71
14.28
Ours (Foggy)
22.04
25.88
24.38
Cityscape
20.36
36.17
16.22
Ours (Normal)
24.43
38.16
32,79
OVIDNet (Ours)
Foggy-Cityscape
19.05
33.71
15.40
Ours (Foggy)
27.72
27.90
27.83
Compared with promising intrusion detection works. We also compare some intrusion detection works, shown in Table 3 Note that the detailed results of model can be seen in Tabel [4] of Appendix E.1 We can see that, compared with previous intrusion works, our model not only has an open structure but also detects
intrusion categories. More importantly, our model has strong generalization capability and achieves zero-shot detection, which is not only pre-trained/pre- undefined categories. In addition, can observe that the task difficulty increases, i.e., common intrusion detection task (PIDNet, Cross-PIDNet, MF-ID)-edomain adaptation intrusion detection task (eeiI--bencc)-Opee--oooe category continuously decreases. The main reason is that, as the difficulty of different intrusion de- tection tasks increases, the requirements of different intrusion detection frameworks are also raised in the open world, especially their generalization and zero-shot capabilities.
intrusion detection task (OVID), the performance of each
Table 3: The comparison between our work and promising intrusion detection works. 'close" and 'open' denote the different detection structures. 'ZSD' denotes Zero-shot detection. / and X denote the intrusion category assessable not assessable, respectively. denotes that the backhone is BNet.
Metbod
Venue
Structure
2Sb
R(%)
Mc))
Bar
TK
Bu
Tar
C(
PIDNet Sun
2020
ACM MM'20
close
x
x
63.3
67.1
x
x
X
x
x
X
x
X
Cross-PIDNet Shi al.
2022
T-IV'21
close
X
72.1
74.7
x
x
x
x
x
x
X
MF-ID Hap et al
2024b
T-ASE'24 24
close
X
45.8
39.8
34.5
38.2
x
x
x
MMIlD-hecch Han
X024c
x
x
x
x
OVIDNet (Ouns)
open
/
7

---

Published as a conference paper at ICLR 2026
Zero-shot and Task-specific transfer evaluation results on proposed strate- gies. We then test the zero-shot/tusk- specific transfer performance of the pro- posed strategies. Specifically, our model on the COCO dataset and val- idate on the Cityscape datasets ob-
Table 4: Zero-shot and Task-specific transfer quantitative
results of the proposed different strategies.
DVG
MENM
mlODES
mAPR 5(90
Aud V(S)
Acchr6)
Accr
x
train
15.30
28. 78
29.16
20.05
37.56
10.72
7
15.33
29.40
29.56
21.01
38.64
31.75
16.22
29.57
28.98
24.43
38.16
32.79
tain the segmentation and detection performance in zero-shot manner. Besides, we also test the intrusion detection performance on the proposed Cityintrusion Openn datasets by task specifcc transfer manner, as shown in Table B denotes the baseline. We can observe that as different strate- gies are added, multiple performances are improved, not only intrusion detection but also zero-shot performance, e.g., PQ (%) and mIOU(%)). Compared with the baseline, the intrusion performance (Acc) can surpass it by 3.43%. In addition, the zero-shot performance can surpass it by 2.19% (PQ) and 1.03% (mIOU), respectively. More detailed results can be found in Appendix E.1
Generalization Verification cross-domain tasks. We further test the performance of our OVID- Net framework and strategies in cross adveree weather tasks, e.g., Normal to verify gen- eralization capabilities. Note that all performance results are given by the pre-trained (in Normal weather) and inference (in adverse weather) manners, as shown in Figure We can find that our OVIDNet is effective even in adverse weather and exhibits promising intrusion performance. Under three different foggy coefficient setting, i.e., 0.005, = 0.01, a 0.02, our OVIDNet can sur- pass the baseline model by 2.96%, 3.22%, and 3.45%, respectively. Besides, our strategies can also improve the zero-shot performance under cross-domain tasks, e.g., compared with the baseline of three different foggy coefficient settings, the PQ(%) in the Normal- tasks can surpass them by 1.29%, 1.21% 16, and 1.12%, respectively. More details results can be found in Appendix E.2
Rudia
OMION DMG
1MDIN 2010
Danka
sabees (MENO ONG
Figure 4: Generalization Verification in cross-domain tasks.
Visualization Comparisons. We also present some visualization comparison results to verify the zero-shot performance and effectiveness of the proposed framework and methods, as shown in Fig- ureß] We can find that our framework can present promising visualization detection results, not only detecting intrusion behaviors correctly but also giving correct Intrusion bels, which proves the effectiveness of our framework and approach. Note that our OVIDNet can improve the zero-shot segmentation performance of Aol; in this case, the Aol is the road. More visualization comparison results are presented in Appendix E.3
('N') la-
(n) Original Images
Ground Truth
(c) Raseline
(4) Ours
Figure 5: The visualization comparison results.
5.3 ABLATION EXPERIMENTS
Multi-Diste Hulti-Distributed Noise Mixing Strategy. We analyze the proposed multi-distributeed noise mixing strategy and conduct extra ablation experiments to verify its effectiveness. The detailed results
8

---

Published as a conference paper at ICLR 2026
shown in Table 5 performance can reach the best, with 31.75% intrusion accuracy. The main reason that, in task- specific transfer, the model focuses more texture features and spatial perturbations. Besides, the transfer task is performed during normal weather. Thus, the need for weather changes and light perturbations is low. In this paper, we the a, B, y to 0.5, 0.1 and 0.4, respectively.
We find that when the a=0.5, B=0.1, -=0.4, the intrusion detection
Dynamic Memory-Gated Module. We also ex- plore the effect of different memory unit sizes on intrusion detection performance, as shown in Ta- ble IOU" denotes the zero-shot segmentation results of the road. We can see that the best in- trusion performance can reached when M=40. The main reason is that the larger memory can help capture richer history and global features, espe- cially in open-world intrusion detection. However, larger memory units also introduce more irrele- vant information, making it difficult to focus key memory features. Conversely, fewer memory units help the model focus features relevant to intrusion detection, but if the memory units are low, will lead to a loss of diversity and complexity required for the intrusion task, affecting the understanding of complex intrusion scenarios. In this paper, we set the memory units to 40,
Table 5: The ablation experiments of the pro- posed strategies. denotes the baseline.
Ablation I: Mutti-Disrrbutedd Nose Mixing Strategy
Metbeds
Metris
(ZA())
N
#C61
PQ
mAP(%)
Anx
Z()
14,03
74.1
27.6
29.36
704
30.25
7022
0.33
14.78
68.8
25.8
30.48
(1.3)
(00
14.53
78.5
25.9
31,03
7.05)
700.3
V
15.35
31,75
Ablalion
Modile
Meltosh
Monerey
1000
Aso
X
14.0%
27.6
29.
(il-30)
14.84
76.0
27.5
30.31
15.80
76.5
29.5
36.72
30.34
5.4 MORE INSIGHTFUL AND INTERESTING EXPERIMENTS
Experiment 1: Why is the performance result of category 'Rider' is '0.0?'
In some table, we find that the performance of category 'Rider' '0.0" egg., Table a4d Table 15 To answer this question, we first investigate some of the latest open-vocabulary works (Bianchi ct al. 2024 Ma et al. (025). Some works denote that the understanding of fine-grained properties of objects and their parts is important. From this view, we conduct some experiments and provide the visualization comparison, as shown in Figure 6 We can find that our model recognizes the category 'Rider" as the category 'Person'. The main reason that these two categories have similar features. 2) Besides, in the training dataset, the number of cate- gory 'Person' is much larger than the category 'Rider'' which leads to category imbalance. Therefore, these
Table The performance of GCRS.
Performance
Rider_Y
Rider_N
Rider
Acc
two factors will make it difficult to recognize the fine- grained category 'Rider', To compensate for this gap, we design a simple yet effective reasoning enhance-
OVIDNet (Ouns)
0
0
0
32.79
+ GCRS
11.95
31.52
23.56
33.55
ment strategy, i.e., Geometric Constraint Reclassifica- tion Strategy (GCRS). The detaled principles of GCRS is shown in Appendix F Then, we conduct several experiments to verify the effectiveness of the GCRS strategy, as shown in Table We can find that when applying the proposed strategy, the performance of the rider category improved from to 23.56. Additionally, intrusion detection performance has improved, i.e., 32.79-+33.55
Figure 6: Some cases of recognizing 'iider (R)' as Person (P)'. denotes the case locations.
Experiment 2: Real-scenario application exploration. To verify the high generality and universal applicability of our framework, we also provide some visualization results under different static sce- narios, e.g., intelligent monitoring, and security. Since static scenarios lack relevant intrusion detec- tion datasets and labels, the specific quantitative evaluation results cannot be measured and given. However, inspired by some super-resse super-resolution works (Gandikota & Chandramouli 2024] Korkmaz al. 2024), can report some demo visualization results. Here, directly use framework

---

Published as a conference paper at ICLR 2026
cnl Sacurter (biin)) (0) Sccarto (iammll beatloon
t8 Orggnd images
Ub 2 Text prumpss Persen Read
ponint Pusson, Cat
5tt g'ieet preepex Peson, Riler Car, Track Rus, Tiars, Moterryck, Biegcll, Road
Figure 7: The visualization demo results in real scenarios. We directly utilize our framework to infer public static scenario datasets without any retraining process. We give three different text prompts customizable results, i.e., text prompts, 3 text prompts, and 9 text prompts, respectively.
to infer public static scene datasets without any retraining process, e.g., the ShanghaiTech Campus dataset (Luo et al. 2017), and the WA-DETRAC (Wen et al. 2020). Note that for different scenarios and intrusion categories, the number of text prompts can be customized. In our paper, we report multiple visualization results with different customizable text prompts Besides, we use different domains, e.g., Normal, Rainy, and Night, evaluate the model's generalization performance, as shown in Figure 7 We can observe that our framework can detect and judge intrusion behavior, demonstrating the practicality and effectiveness of the proposed framework.
DISCUSSIONS
To consider more distribution shift types and en- hance the diversity of intrusion scenes in open- world deployment, we created new intrusion detection dataset for the OVID task, namely The new dataset is built based on the BDD- 100K datasets (Yu et al. 2020). The detailed information can be found in Appendix G Our new datasets contain rich in- trusion scene types.We We evaluate the performance of our model on the datasets, as shown in the Ta- 7 We can find that, in different domain shifts, our strategies still present promising performance improvements. Compared with the baseline, our model can surpass it by 4.58 % which verifies the strong robustness of our model and the effectiveness of the proposed strategies.
Table 7: The detailed experimental results Cityintrusion CeeessiinnOpne-VVVee V-BDD dataset.
MDNM
AscYi%)
AccN(%)
Acc
x
x
20.99
17.22
18.69
Cityintrusion-Opent
x
20.16
19.27
19.62
40.93
20.36
20.38
20.33
+1.64
25.79
21.66
23.27
+4.58
CONCLUSION
In this paper, we propose a new and vital intrusion detection task, Open- Vocabulary Intrusion De- tection (OVID). This is the first multi-modal attempt in the vision-based intrusion detection task. A new benchmark, including relative dataset, an efficient multi-modal framework, and some strong baselines, is given for the specific task. Besides, two effective strategies are proposed to improve the generalization and enhance the performance the intrusion detection task in open scenarios, i.e., the Multi- ciitti-Dittibuttdd Noise Mixing and the Dynamic Memory-Gatee module. Finally, rich exper- iments and comparisons are done to demonstrate the effectiveness of the proposed framework and strategies. In the future, will further explore useful methods to improve performance.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
This work was supported by a locally commissioned task from the Shanghai Municipal Government and the Shanghai Artificial Intelligence Laboratory (AlLab). We also express our gratitude to our collaborators (Fujun Han and Jingqi Ye) for their valuable contributions during their internships at AlLab, both in in-person and remotely. We sincerely thank all authors for their contributions.
REFERENCES
Lorenzo Bianchi, Fabio Carrara, Nicola Messina, Claudio Gennaro, and Fabrizio Falchi. The devil is in the fine-grained details: Evaluating open-vocabulary object detectors for fine- ane-grained under- standing. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recogni- tion, pp. 22520-225222 22529,, 2024.
Olivier Chapelle, Jason Weston, Léon Bottou, and Vladimir Vapnik. Vicinal risk minimization. Advances in neural information processing systems, 13, 2000.
Liang-Chies Chigg Chen, Yukun Zhu, George Papandreou, Florian Schroff, and Hartwig Adam. Encoder- decoder with atrous separable convolution for semantic image segmentation. In Proceedings of the European conference on computer vision (ECCV), pp. 801-818, 2018.
Tianheng Cheng, Lin Song, Yixiao Ge, Wenyu Liu, Xinggang Wang, and Ying Shan. Yolo-world Real-time open-vocabulary object detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16901-16911, 2024.
Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene understanding. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3213-3223, 2016a.
Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene understanding. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3213-3223, 2016b.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scabl scale hi- erarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 248-255. leee, 2009.
Kanehana Vaishnavi Gandikota and Paramanand Chandramouli. Text-guided explorable image super-resolution. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 25900-25911, 2024.
Fujun Han and Peng Ye. Milm-isu: The first-ever comprehensive benchmark for multimodal large language models based intrusion scene understanding. The Thirty-ninth Annual Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2025.
Fujun Han, Peng Ye, Shukai Duan, and Lidan Wang. Ada-id: Active domain adaptation for intrusion detection. In Proceedings of the 32nd ACM International Conference on Multimedia, pp. 7404- 7413, 2024a.
Fujun Han, Peng Ye, Ke Li, Shukai Duan, and Lidan Wang. Mf-id: benchmark and approach for multi-category lin-gnrined intrusion detection. IEEE Transactions on Automation Science and Engineering, 2024b.
Fujun Han, Peng Ye, Chunyan She, Shukai Duan, Lidan Wang, and Derong Liu. Mmid-bench: A comprehensive benchmark for multi-domain multi-category intrusion detection. IEEE Transac- tions on Intelligent Vehicles, 2024c.
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Liang Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. Ielr, 1(2)33 2022.
11

---

Published as a conference paper at ICLR 2026
Dahun Kim, Anelia Angelova, and Weicheng Kuo. Region-aware pretraining for open- vocabbocaullary object detection with vision transformers. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pp. 11144-11154, 2023.
Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al. Segment anything. In Proceed- ings of the IEEE/CVF International Conference Computer Vision, pp. 4015-4026 2023.
Cansu Korkmaz, A Murat Tekalp, and Zafer Dogan. Training generative image super-resolution models by wavelet-domain losses enables better control of artifacts. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5926-5936, 2024.
Ke Li, Fujun Han, Yining Wang. Kewen Huang, Bo Yang, and Yan Tian. Mmidnet: A multi-level mutual information disentanglement network for cross-domain infrared small target detection. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2025a.
Ke Li, Yining Wang, Fujun Han, Hu Wang, Zige Xiong, and Yan Tian. Hstnet: A hybrid spatial- channel sparse transformer network for infrared small target detection. IEEE Transactions on Geoscience and Remote Sensing, 2025b.
Tsung-Yi Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer EssionEECCV 2014: 13th European Conference, Zurich, Switzerland. September 6-12, 2014, Proceedings, Part V 13, pp. 740-755. Springer. 2014.
Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Qing Jiang, Chunyuan Li, Jianwei Yang, Hang Su, al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. In European conference on computer vision, pp. 38-55. Springer, 2024.
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 10012-10022, 2021.
Timo Lüddecke and Alexander Ecker. Image segmentation using text and image prompts. In Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 7086- 7096, 2022.
Weixin Luo, Wen Liu, and Shenghua Gao. A revisit of sparse coding based anomaly detection in stacked rnn framework. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), Oct 2017.
Yuqi Ma, Mengyin Liu, Chao Zhu, and Xu-Cheng Yin. Ha-fgovd: Highlighting fine-grained at- tributes via explicit linear composition for open-vocabulary object detection. IEEE Transactions on Multimedia, 2025.
Dierck Matern, Alexandru Paul Condurache, and Alfred Mertins. Automated intrusion detection for video surveillance using conditional random fields. In MVA. pp. 298-298 088-011,22 2013.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference machine learning, pp- 7748-8763., PMLR, 2021.
Christos Sakaridis, Dengxin Dai, Simon Hecker, and Luc Van Gool. Model adaptation with synthetic and real data for semantic dense foggy scene understanding. In Proceedings of the european conference on computer vision (ECCV), pp. 687-704, 2018.
Zhenyu Shi, Shibo Hc, Jingchen Sun, Tao Chen, Jiming Chen, and Hairong Dong. An efficient multi-task network for pedestrian intrusion detection. IEEE Transactions Intelligent Vehicles, 8(1):649-660, 2022.
Chris Stauffer and W. Eric L. Grimson. Learning patterns activity using real real-time tracking. JEEE Transactions pattern analysis and machine intelligence, 2248)7747777, 2000.
12

---

Published as a conference paper at ICLR 2026
Jingchen Sun, Jiming Chen, Tao Chen, Jiayuan Fan, and Shibo He. Pidnet: An efficient network for dynamic pedestrian intrusion detection. In Proceedings of the 28th ACM International Conference Multimedia, pp. 718-726, 2020.
Chien- Yao Wang, Alexey Bochkovskiy, and Hong- Yuan Mark Liao. Yolov7: Trainable bag-of- freebies sets new state-of-the-ard for real-time object detectors. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 7464-7475, 2023.
Longyin Wen, Dawei Du, Zhaowei Cai, Zhen Lei, Ming- Chigg Chang, Honggang Qi, Jongwoo Lim, Ming-Hsuan Yang, and Siwei Lyu. Ua-detrac: A new benchmark and protocol for multi-object detection and tracking. Computer Vision and Image Understanding, 193:102907, 2020.
Yunyang Xiong, Bala Varadarajan, Lemeng Wu, Xiaoyu Xiang, Fanyi Xiao, Chenchen Zhu, Xiao- liang Dai, Dilin Wang. Fei Sun, Forrest landola, al. Efficientsam: Leveraged masked image pretraining for efficient segment anything. In Proceedings of the IEEE/CVF Conference on Com- puler Vision and Patlern Recognition, pp. 16111-16121, 2024.
Lewei Yao, Jianhua Han, Xiaodan Liang. Dan Xu, Wei Zhang, Zhenguo Li, and Hang Xu. Det- clipv2: Scalable open-vocais ven-voocbbulrry object detection pre-training via word-region alignment. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 23497-23506, 2023.
Jingqi Ye, Haonan He, Minglei Li, Fujum Han, Tao Chen, and Peng Ye. Gradient intrinsic dimen- sionality alignment: Narrowing the gap between low-rank adaptation and full fine-tuning. In The Fourteenth International Conference on Learning Representations, 2026.
Tao Ye, Xiao Cong, Yuliang Li, Zhikang Zheng, Xiangpeng Deng, Xiangming Yan, Xiaosong Li, and Xi Zhang. Few-shot railway intrusion detection without forgetting via double rpn and detec- tor. IEEE Transactions on Intelligent Vehicles, pp. 1-16, 2024. doi: N0.1109/TVV2202433377770
Fisher Yu, Haofeng Chen, Xin Wang, Wenqi Xian, Yingying Chen, Fangchen Liu, Vashisht Madha- van, and Trevor Darrell. Bdd100K: A diverse driving dataset for heterogencous multitask learn- ing. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp- 2636 2645, 2020.
Hao Zhang, Feng Li, Xueyan Zou, Shilong Liu, Chunyuan Li, Jianwei Yang, and Lei Zhang. A sim- ple framework for open-vocabulary segmentation and detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1020-1031, 2023.
Yong-Liang Zhang, Zhi-Qin Zhang, Gang Xiao, Rui-Dong Wang, and Xia He. Perimeter intrusion detection based on intelligent video analysis. In 2015 15th International Conference Control, Automation and Systems (ICCAS), pp. 1199-1204. IEEE, 2015.
Hengshuang Zhao, Jianping Shi, Xiaojuan Qi, Xiaogang Wang, and Jiaya Jia. Pyramid scene parsing network. In Proceedings of the IEEE conference computer vision and pattern recognition, pp- 2017.
Xu Zhao. Wenchao Ding, Yongqi An, Yinglong Du, Tao Yu, Min Li, Ming Tang, and Jinqiao Wang. Fast segment anything. arXiv preprint arXiv:2306.12 12156, 2023.
13

---

Published as a conference paper at ICLR 2026
APPENDIX OVERVIEW
Table of contents:
A: Cityintrusion-Open Dataset
8 A.I: Method and Statistics for Proposed Dataset
8 A.2: More Visualization Results
S A.3: More Intrusion Dataset Comparisons
8 A.4: The Correspondence Between Full Name, Text Prompt, and Abbreviation
S A.5: Framework Design Motivation
B: Mechanism Proof for Multi-Disse Distributed Noise Mixing strategy
C: More Experiment Settings
D: The Comparisons with Other Strong Open-vocabulary Detection Systems
E: More Results for OVID Task
8 E.1: Quantitative Results of Different Categories
E.2: More Results of Cross-domain Task
E.3: More Visualization Comparison Results
F: The Principles of Geometric Constraint Reclassification Strategy
G: The Detailed Information of Cityitrrasion-OpenVBBDD Dataset
H: Limitations and Further Works
A CITYINTRUSION-OPENV DATASET
In this subsection, we present additional information and details for the proposed Cityintrusion- Open' dataset, including data statistics, visualization results, intrusion Dataset Comparisons, and the correspondence between full name, text prompt, abbreviation, and framework design motivation.
A.1 METHOD AND STATISTICS FOR PROPOSED DATASET
Automatic Generation Method. Inspired by promising intrusion detection works (Sun al. 2020
Shi et al. 2022 Han et al. 2024b), our Cityintrusion V dataset is built based on the Cityscape dataset (Cordts et al.] 201 The main reason that the Cityscape datasets have segmentation and detection labels for the same original image, which provides prerequisite for our multiple intrusion detection tasks. Additionally, following the relevant works (Han et al. 2024c), we also design an automatic labeling program to generate Intrusion ('Y") and No-intrusion (NN) labels. Note that the
final intrusion detection overlapping pixel points are also set to 20 (Sun et al. 2020). The specific processes shown follows.
Step 1: We first clean the original Cityscape (Cordts et al., Saall/ooog--Cyyycaapne (Sakaridis) et al. 2018). After cleaning, conduct frame alignment for these datasets. Note that small number of objects that we don't care about or are incorrectly labeled will be removed in this process.
Step 2: Based on the results in step 1, we read the bounding box coordinates of the interested FoyyeeCycyyyCtecppeeeeeee datasets. Additionally. we also read the area-
intrusion objects from the of-interest (Aol) in the Cityscapes segmentation dataset (Cordts al. 2016b).
Step 3: For the obtained area-of-interest (Aol) in step 2, we binarize them with 0 and 1.
Step 4: After step 3, the bounding box coordinates from step projected into the binarized area-of-interest (Aol).
Step 5: We calculate the overlapping pixel values between Aol and bounding box in step
14

---

Published as a conference paper at ICLR 2026
Original images
Segmentatio-Latee
Detaction-Label
(d) Ours(Intruiion class text Isbel)
Figure 8: More visualization results of our Cityintrusion-Open V. Unlike previous intrusion detection datasets (Sun al. 2020 Han al. 2024b), our datasets encompass all common/possible intrusion categories in Cityscape datasets, providing richer and varied labels that meet the requirements the proposed OVID task.
Step 6: To get the final intrusion/no-inteii intruusion labels: 'N/Y, Class', we compare overlapping pixel values in step 5 with setting threshold, where 'N' denotes Non-Intrusion, 'Y' denotes Intrusion, and 'Class' denotes names intrusion objects. Note that, following previous work (Sun et al., 2020), the threshold is set to 20.
Step 7: To obtain and present our final intrusion detection dataset better, we blended the segmented images containing the intrusion labels in step 6 with the original images in step 1.
Step 8: Finally, to ensure the quality and accuracy of the proposed datasets, a team of three students organized to manually inspect and verify the annotations.
Statistical Analysis. Then, we conduct detailed statistical analysis, as shown in Table We provide details of the number of intrusion and non-intrusion cases for each category in the training and validation sets of the dataset, along with the total average. The total average of the whole dataset can reach 18.03, surpassing previous promising intrusion detection datasets greatly. Rich labels can meet the requirements and provide data foundation for the proposed OVID task.
Table 8: The detailed statistics of proposed datasets. T and V denote the training and validation datasets, respectively. denotes the total average in the whole dataset.
Person
Rider
Car
Truck
Bus
Train
Motorcycle
Bicycle
Categories
T|V
TJV
T/V
T|V
TTV
T/V
T/V
TVV
Intrusion cases ('Y")
3567/716
698/226
14545/2493
246/52
219/67
88.9
270/55
1138) 361
Non-Iatrusion cases ("N")
14427|2703
1109 330
12610/2174
243(41
166/31
83/14
469/94 994
2591 [814
Total cases
17994/3419
1807 556
27155|4667
489/93
385/98
171
739|
3175
Total average! (Per very image)
18.03
A.2
MORE VISUALIZATION RESULTS
In order to better present our proposed dataset, we also provide more visualization results, as shown in Figure Different from the previous single category (President) (Sun et al. 2020) Shi et al., 2022) and four categories (President, Motorcycle, Rider, Bicycle) (Han et al. [0244b c). we can find that our Pivintrusion-Opent dataset contains multiple different intrusion categories, not only single or four categories. All possible intruder categories can be considered in our dataset, e.g. Person, Rider, Car, Truck, Bus, Train, Motorcycle, Bicycle. Our new dataset can provide the prerequisite for the OVID task. Here, because of the labels, utilize abbreviations instead of labels in order to easily show our results, e.g., 'N, P denotes the NN--Intuusoo,, Person', Text prompt: Person. 'Y, C' denotes the 'Ittrusion, Car', Text prompt: Car. 'Y, Bu" denotes the 'Intrusion, Bus", Text
15

---

Published as a conference paper at ICLR 2026
Table 9: The correspondence between the full name, text prompt, and abbreviation. Italic denotes thing classes (Aol). All categories are customizable in different scenarios.
# No.
Full name
# Text prompt
# Abbreviation
# 1
'Person"
'Person'
'P'
'Rider"
'Rider'
# #3
'Car'
'Car'
"C'
"Truck"
'Truck'
'Tk'
# 5
'Bus"
'Bus'
'Bu'
"Train"
"Train'
'Tn'
# 7
'Motorcycle'
'Motorcycle'
'M'
#
'Bicycle'
'iccycle'
'Bc'
# 9
'Road"
'Road'
'Ro'
# 10
prompt: Bus. 'Y, M' denotes the 'Intrusion, Motorcycle', Text prompt: Motorcycle. The detailed correspondence between full name, text prompt, and abbreviation and can be found in Table 9
A.3 MORE INTRUSION DATASET COMPARISONS
In this subsection, we further compare our proposed datasets with other promising intrusion detec- tion datasets and provide more comparison results to verify the superiority of our dataset, as shown in Figure 9 Compared to previous promising intrusion detection datasets (Sun et al. 2020 Han al. 2024b|c). our dataset exhibits much superior and richer labels. Besides, the proposed datasets contain 8 intrusion categories, surpassing the previous works 1 or 4 categories. More importantly, our Sityintrusion-pper dataset contains text labels, which compensate for the lack of relevant datasets and meet the needs of the proposed OVID task.
A.4 THE CORRESPONDENCE BETWEEN FULL NAME, TEXT PROMPT, AND ABBREVIATION
To better help understand the different intrusion categories and the abbreviations in our paper, we provide the detailed correspondence between the full name, the text prompt, and the abbreviation.
The detailed correspondence is shown in the Table g e.g., 'Person' (# Full name) - 'Person' (# Text prompt) + 'P' (# Abbreviation). 'Rider' (# Full name) - 'Rider' (# Text prompt) + 'R' (# Abbreviation), 'Car' (# Full name) + 'Car' (# Text prompt) - "C' (# Abbreviation), 'Truck' (# Full name) > 'Truck' (# Text prompt) + 'Tk' (# Abbreviation), 'Bus' (# Full name) - 'Bus' (# Text prompt) > 'Bu' (# Abbreviation).
A.5 FRAMEWORK DESIGN MOTIVATION
In this subsection, We first explore two basic yet important questions motivations for our ap- proach. (I) Why do we conduct open vocabulary intrusion detection research? Our goal is to break through the dependencies and limitations of pre-defined categories. Truly enable intrusion detection in the open world. (2) How to achieve the specific OVID task? A simple idea is that we can leverage collaborative model with vocebuoocyylary segmentation (OVS), e.g., SAM (Kirillov al. 2023),
FastSAM (Zhao et al. 2023). EfficientSAM (Xiong et al.) 2024). and Open-vocabulary detection (OVD), egg., DetClip Yao et al. 2023), Grounding DINO (Luu al. 2024), YOLO-world (Cheng et al. 2024) to train/infer and get final Intrusion/No-intrusii
labels. As shown in Table we list and compare some feasible schemes. Unfortunately, although the model of "OVD+OVS" is a feasi- ble solution, it is not suitable for intrusion detection. The main reason is that the training cost of the End-to-End strategy combined with two LLVMs (Large Language Vision Models) is very expen- design new efficient framework for the proposed OVID task, namely OVIDNet. Our framework is established based on OpenSeeD (Zhang et al.) 2023). Finally, the OVIDNet is leveraged to collaborate give the bounding box and mask image for the OVID
sive. To alleviate this problem,
16

---

Published as a conference paper at ICLR 2026
Original (CYPR'16) Intrusion Calegnries Noue
(b) Intrusied Categuriss
(SE A Intrasion Canegories
(M)MMDD'(T (V # Intrusion Categories: 4
[c] Ours (CoporraiinnCrarn)) Intrusion Categosies:
Figure 9: The compression between our datasets with other promising intrusion detection datasets. Unlike previous intrusion datasets (Sun et al. 2020 Han et al. 2024b), datasets encompass broader range of potential intruders. Our datasets can be used to train/cvaluate the performance of the OVID task and validate the effectiveness of the proposed strategies.
task, and the final intrusion labels ("N/Y") are given by the intrusion post- post-processing judgments. The overall framework and pipeline of OVIDNet are illustrated below.
The comparison of some feasible schemes for the proposed OVID task. OVD and OVS denote the open-vocabulary detection and segmentation models. Pre-trained TSM denotes the pre- trained traditional segmentation models, e.g., DeepLabv3+ (Chen et al. 2018), PspNet (Zhao et al.) 2017). Retrain denotes whether the model needs to be retrained under different scenarios. We can find that our scheme is End-to-End and has low training cost.
Table 10:
OVD
OVS
Scheme
Pre-trained TSM
End-to-Fin Fnd
Open- Vocahulary?
Retrain
Training Cost
Train
Infer
Train
Infer
S1
X
Very Large Large
S2
/
S3
X
X
Low
Ours
x
Low
B MECHANISM PROOF FOR MULTI-DISTRIBUTED NOISE MIXING STRATEGY
To clarify why the proposed noise mixture improves generalization toward unknown categories, provide a theoretical proof based on Vicinal Risk Minimization (VRM) (Chapelle et al. 2000). In Equation perturbation can be written as
By C C + (a N., + + +
To facilitate subsequent derivation, we set Z = a- N.NN. +3 3-NN, + N,, And No N, denote noise sampled from Uniform, Gaussian, and Laplace distributions, respectively. Since Z is combination of three independent noise sources, its distribution can be expressed as
(2(2) = (2)(2) + (2) + (p).
(7)
which mixed distribution containing (1) fine-grained bounded perturbations (U), (2) moder-
ate Gaussian variations (Gaussian), (3) heavy-tailed structural deviations (Laplace). This directly
yields mixed vicinal distribution the Vicinal Risk Minimization (VRM) framework. Following previous work (Chapelle et al. 2000), the vicinal risk can be written as
(f (lou)~DBeuppe {(s(x z).v).
(8)
17

---

Published as a conference paper at ICLR 2026
Because pz is mixture, VRM can be expressed
RM = oRu( +R)
(9)
(((==)) where 1 () (x) Bz-p, (((x+)),
Thus, instead of training against single perturbation model, our model effectively optimizes risk under three complementary vicinal neighborhoods simultaneously.
Let the real open-world perturbation distribution be q(2). If y and pz share support and < pz, then the Radon-Nikodym ratio can be expressed as
K == sup (2)
(10)
pz (2)
and the K finite, giving
g(2) Kpz(z).
(11)
Because pz(=) (z) + (2,(2) + (2) covers bounded signals (U), smooth variations (Gaus- sian), and heavy-tailed structural changes (Laplace). has wider support and a smaller worst-case ratio K than any single distribution. Therefore, the true risk is upper-bounded by
(f( E + 2). S K RM--
(12)
A smaller constant K implies tighter bound. Hence, model can better generalize unknown object shapes, sizes, and distributions, and improve localization for novel or rare categories. Because their perturbations are more likely fall inside the large support of the mixture pz.
C MORE EXPERIMENT SETTINGS
In this section, will introduce more implementation details and settings in the experiments. We present more setting details for the experiment, as shown in Table [1 Due to the limitation of our GPUs, we have to set the Image_size makes some of our results lower than those in the original model (image_size size: 1200 1200, max iter: 368750) (Zhang al., 2023). To ensure fairness and verify the correctness of our method, we also set CHECKPOINT PERIOD and EVAL PERIOD to 15000, respectively, for all experiments. Ad- ditionally, we retrain the baseline and verify the method's s validity. Note that our OVIDNet frame- work is built based on the OpenSeel) (Zhang et al. 2023). The OpenSeelD is a simple but efficient framework for open-vocabulary segmentation and detection. Differently, we modify the original framework meet the requirements of the OVID task. We first add an effective intrusion detection judgment module to obtain the capability for intrusion detection. Then, we propose two strategies for improving generalization and intrusion performance in the open world and verify the effective- ness on multiple dominant datasets and tasks. Therefore, to be fair, our setting mainly refers to Openseed and is adapted our own tasks. Our OVIDNet framework consists of text encoder (Clip (Radford et al., (20TT), image encoder (Tiny-swin-rransfornee (Liu et al. (02T)), decoder, and intrusion detection prot-progsssing module. Note that the final overlapping pixel threshold set to 20 (Sun et al.) (2020).
800 and reduce the iterations 15000, which inevitably
D THE COMPARISONS WITH OTHER STRONG OPEN-VOCABULAN DETECTION SYSTEMS
To further validate the effectiveness of our model. we conduct a thorough investigation for other vocabuucyyy detection models, e.g., YOLO-World (Cheng et al.) 2024) and Grounding DINO (Liu et al. 2024), and find that they primarily focus on the open- vocabulary detection do- main. However, our task requires not only detection sub-tasks but also segmentation sub-tasks and intrusion detection sub-tasks. Therefore, open-vocabulary detection models alone cannot fulfill our intrusion detection requirements. Besides, we also find that some models support the open- vocabulary instance segmentation. However, these models also cannot meet the needs of the OVID task, as shown in Table 12 Nevertheless, to validate the effectiveness of our proposed model, conducted some experiments on the YOLO-world and GroundingDINO models. Specifically, we follow previous intrusion detection work (Sun al. 2020 Han al. 2024b) and adopt combined
18

---

Published as a conference paper at ICLR 2026
Table 11: The detailed illustration of the experiment setting.
# Name
Setting Category
Value
# Name
Setting Category
Value
TOKENIZER
CLIP
WINDOW SIZE
CONTEXT.LENGTH
18
PATCH_SIZE
WIDTH
TEXT
512
EMBED DIM
BACKBONE
96
HEADS
8
DEPTHS
12, 2, 2]
LAYERS
12
NUMCHEADS
13 12,
# Name
Sctting Category
Value
# Name
Setting Category 4
Value
(GNORE_VALUE
255
NHEADS
LOSS. WEIGHT
1.0
CLASS. WEIGHT
CONVS_DIM
256
MASK. WEIGHT
5.0
MASK_DIM
256
DICE_WFIGHT
5.0
ENCODER
DECODER
COMMON STRIDE
4
BOX, WEIGHT
5.0
TRANSFORMER LANERS
GIOU, WEIGHTT
2.0
TOTAL NUM FEATURE LEVELS
HIDDEN DIM
256
NUM FEATURE LEVELS
NUM QUERIES
model approach for testing, ie., open-vocabulary detection (OVD) open-vocabulary segmenta- tion (OVS). Here, open-vocabulary detection models contain YOLO-world (Cheng et al. 2024). Besides, the open-vocabulary segmentation model adopts the Clipseg (Lüddecke & Ecker 2022)
to complete the experiments. Note that for the latter aopeb-vocabulary segmentation model), we initially intended to adopt the SAM model (Kirillov et al. 2023 However, we find that the origi- nal SAM requires point or bounding box information for object segmentation. Therefore, in these experiments, we employed the Clipseg model for the open-vocabulary segmentation subtask. The specific experiment results are as shown in Table 13
Table 12: The proposed OVID task analyses in different open- open-vocabulary works.
Name
Instance segmentation work
Panoptic segmentation work
Does it segment stuff (road/other)
X
Are output pixels fully covered?
X
Can it meet the requirements the OVID task?
X
Table 13: The comparison results between different combined models and our OVIDNet model.
Model
Type
Domain
Acc
Parameter
Normal
22.64
Y Cheng et al.
2024 Clipseg Läddecke & Eeker, 2022
OVD+OVS
163.75M
Foggy
17.75
Normal
30.08
Yorl-World-L
Cheng et al.
2024 +Clipseg Lüddecke & Ecker 2022
OVD+OVS
198.75M
Foggy
24.34
OVIDNet(OUrs)
End-to-End
Normal
32.79
120.32M
Foggy
27.83
We can find that, compared with the combined model (YOLO-Wald-L-Clipse) model can surpass it by 2.71 % (normal domain) and 3.49% (foggy domain). These performance gains demon- strate the effectiveness of our model.
E MORE RESULTS FOR OVID TASK
In this section, we will provide additional results to test the effectiveness of our framework and strategies. We first report quantitative results of different categories in normal and cross-domain conditions. Then, we present more visualization results. The specific results are shown below.
E. QUANTITATIVE RESULTS OF DIFFERENT CATEGORIES
We first present additional results from various categories using the proposed strategies. Note that give two types of metrics, i.e., segmennation/ddtectioon metrics (IOU, AP, AP@.5) and intrusion
19

---

Published as a conference paper at ICLR 2026
detection metrics (AceY, AceN, Ace), respectively. The former is obtained via zero-shot manner, the latter via a task-specific transfer manner. We conduct experiments in COCO, Cityscape, and SityintuuionnOpenny as shown in Table [4 From Table [4 we can find that when the proposed methodology is added, multiple metrics in multiple categories are improved to a certain extent. Compared with the baseline model, our strategies can surpass it by 3.97% (IOU) and 0.93% (AP), respectively, which verifies the effectiveness of the proposed strategies.
Table 14: The more quantitative results of different intrusion categories. Task: COCO-Cittscaa Cityscae,, Cityintrusion-Open We provide quantitative results for all possible intrusion categories to test the effectiveness of the proposed strategies. Besides, to comprchensively measure the results across different categories, we report two distinct metrics, i.e., segmenntaiion/éelcctoo metrics (IOU, AP, AP@.5) .5) and intrusion detection metrics Ac AccN, Acc), respectively. The bold is the best result.
brireiee Calgarioo Taak: COCO :Clysaape Clpistraaknnnleee
Segroetalias and Deiviion Mitrios
Badine
DMO
MUNM
PenserS)
Rader())
Cariio
Trukk/
Balli)
Bloyketk)
Angetty
XoIjAP
KIU/ATJAPEES
KHNJARJAPESS
KAAAARAPPS
23
42./52455.5
3A382213
04.3881224
44.3]]442441
1pt 485.5
2421445555
2/55
06 15.330
561000 925
24144:99
06511/0286
112522.13255
60.2
1RA11 9272
7251222765
47.71155.6 28.56
Saskgka
Istreies Detectina Marko
Badiee
DMG
MDNN
Proverk)
Trve
Be(S)
Tole())
Ruy:k(1)
All Eiregeiis
Accit (ach/Ace
AccY/AocN Ase
Accit Aec
Acct Acci Ace
AccN/Ac
AccY/
AutTAAscN Apc
/
x
1240445 28/90.2
0:000.000000
22.34 25.46/25
31,34 20032 65
009100
145321 24 18.29
11877.1777.. 35
5000
52.90 16
Dreep 00g0.0
2880p6091110 16
1801/11 17/95 93
24 38.72
2131/990606612
0,001.000000
38.8232.20344
12.3323.40144 1177/1148.70
19:44
20,224226688 26/3554
1
00006.00100
26122 40.1633333
32.0973233..1
32.8425010011
DJ8002091.00
344309.0033355 33.79
Table 15: The more quantitative results of different intrusion categories in the cross-domain task. We further test the effectiveness of the proposed strategies with a task-specific transfer manner. Task: COCO- FFggy- Seegy--ittsscpe,, Cityintrusion-Oper Open Three foggy conditions are used to conduct comprehensive experiments, i.e., a=0.005, -0.00.01000000 The bold is the best result.
Jainaka Catopelos COCO Eogge-CCppppee Cltp-gp, Qppaaeraien--pp
Seskgies
Badis
DMG
MUNN
Bagycees)
Caragetts
AschiAAe
AscYf AucNiAss
AacTAAsdiAa
AccY)AecNAAss
X
000000000000
25.43 54/24/44
01,12500000
00000
36
10 11gh8
211949
000000000000
23,77 3000627 66
288532220031
DUKeO D09100
23 2464
11.5041 15/54.21
212554.7 28,49
8
20.9 183445
0 800
2407 95
250845000000
6000 14
14 2162
18666,000000 N8:52
00000000000
%
11 47
30.661 32/11 51
30 25
Dx000 cop000
22 2o 1533.1
22 24/39
24 sepe $1 31.40
Saciegios
Bodin
DMO
MUSN
Ponse(R)
Rale(()
Truc::3
Baili
Manscyyk(t))
Blcyck(1)
All Caraeeta
AscT) Auc
Asch! Auchi
AscY Auch Acc
Accht Auch//uzz
11:541 66/95 98
0000
1:11 20001/5666
70.77 88/14444
15.5 14 1474
Dreeo 00
1% 1116 17/56
0000400 too0
24.79 21.3126.0
26.5442002221
22.0272
2480222342144
2022 54153.29
1482200
0000
30,55/20 14
388913309931
0000
33
000001001:0
23:5015 10115-1113:.00
Persert)
Ralerti
Carlii
Recti
Toseec)
Begekett)
Ciregvies
Bedine
DMO
MINM
AccY)Aash/Asz
Accht AuchiAAc
AccY/Acch)Aoc
AcdY Acch Acc
Accf AucNiies
x
x
143833 2292.7
0.009.20 8000
26.56 14.8221
28.5 12,0823
CX00 s09000
152411/9/2..3
22 2214425.9 34.99
V
V
x
00001000100
76.55 65
No 19:55
Dreeo rcpu00
7164p4 (04p4-073.00
3hoq11o01p 74
23 1aapp00335777
8
07 00000:000000
27.96 26 94277
907733444144 26
35.92 1422:19:00 19:2000 6
16 1430000 29
34pp 56
25
/
8
000010031000
32.25 2330/2333
32.6116-1977
27 1333pm0 23.83
E.2 MORE RESULTS OF CROSS- -DOMAN TASK
Furthermore, we present additional intrusion detection results for various intrusion categories across different cross-domain tasks. In this experiment, we adopt three different foggy coefficients, i.e., a=0.005, a=0.01, and a=0.02. In these experiments, We conduct experiments in COCO, Foggy- Cityscape, and Cityintrusion-O0 Op as shown in Table [5] We can observe that, in various cross- domain tasks, our strategies enhance intrusion detection performance. In different tasks, compared
20

---

Published as a conference paper at ICLR 2026
with the original baseline model, framework can improve them by 2.96%, 3.22%, and 3,45%, respectively. Furthermore, our proposed approach can effectively improve the performance of intru- sion detection for various categories. These performance improvements demonstrate the effective- ness of our approach, as well as the ability of our framework to generalize.
E.3 MORE VISUALIZATION COMPARISON RESULTS
Finally, we also present more visualization comparison results to verify the effectiveness of the pro- posed framework and strategies. We set the text prompt of stuff felffffclssssssese Road and set the text prompt of thing classes as Person', 'Rider', 'Car', 'Truck', 'Bus', 'Train', 'Motorcycle', 'Bicycle', shown in Figure I0] From Figure we can find that our framework can present promising visu- alization detection results, not only detecting all intruders correctly but also giving correct Intrusion ('Y')
('N') labels, which proves the effectiveness of our approach.
O DOR
[sl Original Images
1b| Ground Truth
[c] Baseline
CdI Durs
Figure 10: The visualization comprising results. Here, (a), (b), (c), and (d) denote Original images, Ground truth, Baseline results, and Ours, respectively. Text prompt: Road (stuff classees. 'Person', 'Rider', 'Car', 'Truck', 'Bus', 'Train', 'Motorcycle', 'Bicycle', (thing classes). For clanseclssses the abbreviations are used instead of complete labels easily show our results. The correspondence between the abbreviation and full name can be referred to in the Table
F THE PRINCIPLES OF GEOMETRIC CONSTRAINT RECLASSIFICATION STRATEGY
In subsection 5.4, we analyze why the performance of the category 'Rider' is '0.0' in depth. To compensate for this gap and enhance the practicality of model, we design simple yet effective reasoning enhancement strategy, iee. Geometric Constraint Reclassification Strategy. Our GCRS strategy implemented within the proc-srogssiing reasoning phase, specifically designed to miti- gate local class ambiguity, e.g., category similarity. The ambiguity occurs when our model predicts person in the vehicle as a standalone Subject Box Bp (Person), rather than the intended com- posite Rider class. Therefore, the principle of our GCRS strategy relies on utilizing the spatial topological constraints between predicted bounding boxes. Specifically, by calculating the IoU be- quantify tween the detected Bp and any potential Accessory Box By (Bicycll/Mootrcyylle) Threshold т, their degree of spatial coupling. If this coupling exceeds a predefined it constitutes strong geometric constraint. Then, mandatory class is reassigned to enforce seman- tie consistency. The core correction criterion is formally expressed as a conditional reclassification operation, and express it
21

---

Published as a conference paper at ICLR 2026
If Class(Bp) (B)) 'Person' and (3Bv E {('Bicycle', 'Motorcycle)) s.t. [BU Bv) r): (13)
Then Class(Br) + 'Rider',
where T is threshold. The loU metric used to quantify the overlap is defined
Area( Bp Bv) Area Bp Bv)
loU(Br, Bv)
(14)
Our GCRS strategy acts a mamaain-knowlede-ddrvvee geometric filter. And GCRS can signifi- cantly enhance our model's accuracy on some challenging composite instances (e.g., Rider category) by prioritizing geometric evidence primary model scores.
G THE DETAILED INFORMATION OF CIYYNTRISSONNOPENV--DD V-BDD DATASET
To consider more distribution shift types and enhance the diversity of intrusion scene in open-world world deployment, we created a new intrusion detection dataset for the OVID task, namely Cityintrusion- V-DDD- The dataset is built based on the BDD-1000 datasets (Yu al. 2020 We clean the original dataset based on the proposed OVID task features. The detailed method can refer to Appendix A.I our paper. Our new datasets contain rich intrusion scene types, e.g., multiple different weather (Clear, Cloudy, Rainy, Foggy, Night), different geographic environments (City, Highway, Suburban/Rural), different period of time (Daytime, Dusk, Night), and Different trans- portation environments (Heavy Traffic, Empty Road). These new domains can meet the experi- ment's requirements in different distribution shifts. Finally, our datasets contain 1482 training data and 449 evaluation data. We evaluate the performance of our model on these datasets, shown in the Table 7 We can find that, in different domain shifts, our strategies still present performance im- provements. Compared with the baseline model, our model can surpass it by 4.58 %, which verifies the strong robustness of our model and the effectiveness of the proposed strategics.
H LIMITATIONS AND FURTHER WORKS
Intrusion Detection (OVID) project for the first task, efficient framework, and a strong benchmark for vision-based in- trusion detection. Additionally, we design corresponding strategies to enhance intrusion detection performance in real-world scenarios and increase the practicality of the model. However, there still some limitations that need to be addressed in the future: (1) Enhance the ability to recognize fine-grained categories and improve generalization performance in the real world. (2) Inspired by methods for earfieter-efficient fine-tuning (Hu et al.) [2022; Ye et al.) 2026). we will explore effi- cient ways to enhance the intrusion detection performance of models at lower fine-tuning cost.
In this paper, we introduce the time, including
22


Reviews:
[{"review_id": "AGcXw90cE5", "reviewer": "Reviewer_v3N4", "summary": "This paper establishes a new task of dynamic-view Open Vocabulary Intrusion Detection, including a correlated dataset Cityintrusion-OpenV. An end-to-end model, OVIDNet, is designed as a baseline for the proposed new benchmark. Two strategies are proposed to improve the generalization and performance of OVIDet.", "questions": "1. What does \"model singularity\" in line 088 mean?\n2. What does \\textbf{B} denote in Table 4 and Table 5?\n3. Is it sufficient to solve the OVID problem by adding the intrusion judgment capability to an open-vocabulary detection and segmentation model? In this work, the implementation of intrusion judgment is simple and learning-free, which suggests that simply combining the two should work for the OVID task. The authors should make an in-depth discussion or analysis to reveal the inherent significance of the proposed OVID task.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The proposed dataset extends the future research area of vision intrusion detection.\n2. The overall method is easy to follow.\n3. Comprehensive experiments and comparisons are conducted to verify the effectiveness of the proposed framework and methods.", "weaknesses": "1. The designed Multi-Distributed Noise Mixing Strategy is an incremental enhancement of the noise generation in OpenSeeD.\n2. While the Dynamic Memory-Gated module is designed to capture the contextual information under complex scenarios, no discussion or analysis demonstrates whether this challenge is inherently significant in OVID. It is also unclear how this module functions in the overall model.\n3. The open-vacabulairy capability relies on existing OVD and OVS practices. The overall method lacks in-depth insight into tackling the proposed OVID task.\n4. While the established challenge is an open-vocabulary problem, the experiments are conducted under zero-shot and task-specific transfer settings (lines 306-307).\nOther typo issues, including but not limited to\n1. Line 010, using e.g. and etc. in the same sentence.\n2. Line 173, \"(D_T^d, D_V^s)\" . I believe this should be \"(D_T^d, D_T^s)\"\n3. Line 315, \"duo to\"", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Zg7FuEggr5", "reviewer": "Reviewer_7Wz9", "summary": "This paper proposes the first Open-Vocabulary Intrusion Detection task, builds the Cityintrusion-OpenV dataset (with 8 intrusion categories), and designs the end-to-end OVIDNet framework. Then they provide OVIDNet, their end-to-end framework that pairs CLIP for text and a tiny Swin Transformer for images, checking overlaps between bounding boxes and AOIs to judge intrusions.", "questions": "- When the paper mentions “multi-domain,” is it only normal and foggy weather? Most tests switch between those two, but there’s no word on other domains like rainy or night. \n- Line 168 has a typo—“detention” should be “detection.”", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "Open-vocab was a gap in intrusion detection, and the Cityintrusion-OpenV dataset provides new annotations for this task.", "weaknesses": "1. The core logic for “intrusion judgment” feels rather vague. As it stands, the method appears to detect any object that appears on the road, rather than accurately identifying what constitutes an intrusion. Although the paper mentions using the overlap between bounding boxes and the AoI to determine intrusions, it never clearly defines what constitutes an intrusion in different scenarios. For instance, are regular vehicles considered intruders? What about pedestrians or bicycles? Are there specific time periods or restricted zones where entering the AoI becomes an intrusion? Without these clarifications, the task essentially boils down to “object detection plus overlap checking,” rather than genuine intrusion detection, which should focus on identifying unauthorized entries. In fact, if the goal is simply to detect everything on the road, this can be easily achieved using an MLLM or by combining a segmentation model with an open-vocabulary detector.\n2. Two core improvements lack mechanism proof. For the Multi-Distributed Noise Mixing strategy, the paper only states it mixes three types of noise and adjusts ratios dynamically, but fails to explain why this specific noise combination enhances unknown category location information or how dynamic ratios avoid damaging normal features. For the Dynamic Memory-Gated Module, it only describes the structural process (global average pooling → memory retrieval → feature fusion) without clarifying why the query vector matches memory units effectively.\n3. The method performs poorly on basic categories like “Person” and “Rider,” weakening its practicality for common intrusion scenarios.\n4. The figures are too small to be visible clearly, hindering verification of visualization results.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "3qk4QAG8GE", "reviewer": "Reviewer_jM2n", "summary": "The paper introduces Open-Vocabulary Intrusion Detection (OVID), which defines a new task, provides a Cityscapes-derived dataset (Cityintrusion-OpenV), and proposes a multi-modal, multi-task framework (OVIDNet) that aligns image features with text prompts to jointly perform detection, segmentation, and intrusion judgment. The method uses pixel overlap between predicted boxes and AoI masks to identify intrusions. Two components, Multi-Distributed Noise Mixing (MDNM) for adaptive perturbations and a Dynamic Memory-Gated (DMG) module for context, are designed to improve zero-shot generalization and open-world robustness. Reported gains over OpenSeeD are +2.19 PQ and +3.43 intrusion accuracy under lighter training settings (800 resolution, 15K iterations). The dataset extends prior intrusion benchmarks from 4 to 8 categories with Y/N labels and text prompts (18.03 cases per image). Qualitative results on ShanghaiTech and UA-DETRAC show promptable deployment without retraining, although quantitative metrics are missing for those examples.", "questions": "Please follow Strengths and Weaknesses. \n\nWhy is only selectively synthetic fog evaluated? For open-world deployment, it would be important to also consider other distribution shifts such as common corruptions [1] (rain, snow, and similar variations adapted to Cityscapes) and viewpoint or perspective changes [2] (for example, Möbius or MPD augmentations). The paper should clearly define the scope and reasoning behind focusing solely on fog. Exploring how AoI–box consistency behaves under viewpoint changes and visual artifacts could provide valuable insights into the model’s robustness.\n\n[1] - Hendrycks, Dan, and Thomas Dietterich. \"Benchmarking neural network robustness to common corruptions and perturbations.\" arXiv preprint arXiv:1903.12261, 2019. \n[2] - Chhipa, P. C., Chippa, M. S., De, K., Saini, R., Liwicki, M., & Shah, M.  Möbius transform for mitigating perspective distortions in representation learning. In European Conference on Computer Vision, 2024.\n\nWhat happens if the models are trained at higher resolution (1200) and for longer schedules? Do the reported gains hold?\n\n\n\nI am open to revising my score.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper is the first to formally define open-vocabulary intrusion detection and presents a multi-modal framework that combines detection, segmentation, and intrusion judgment. It connects open-vocabulary detection with security applications in an interesting and practical way.\n\nThe Cityintrusion-OpenV dataset expands existing intrusion datasets to 8 categories and introduces text prompts with a much higher number of Y/N cases per image (18.03), which is a solid step toward open-world evaluation.\n\nThe experimental coverage is broad, including zero-shot transfer (COCO→Cityscapes), task-specific transfer (Cityintrusion-OpenV), fog domain testing, ablations, and qualitative real-world demonstrations.\n\nThe ablations for MDNM and DMG demonstrate how different configurations (α, β, γ values, and memory unit sizes) affect performance and reveal consistent, though modest, improvements.\n\nThe figures and algorithm descriptions are clear and easy to follow, and the appendix includes sufficient details for reproduction.\n\nThe approach shows potential for practical deployment, as it can handle new categories via prompts without retraining and appears somewhat robust under foggy conditions.", "weaknesses": "The baseline comparisons are not entirely fair. The authors trained with a reduced image size (800) and fewer iterations (15K), while OpenSeeD used 1200 and 368,750. Without matching settings or reporting variance over multiple seeds, it is hard to interpret the reported gains.\n\nThe ShanghaiTech and UA-DETRAC results are only qualitative, but the paper makes strong claims of universal applicability. Quantitative results would be needed to back that up.\n\nThe technical novelty is limited since OVIDNet largely builds on OpenSeeD with an added intrusion decision module. MDNM’s mixture of distributions and logarithmic area scaling lacks theoretical justification, and DMG feels similar to standard attention mechanisms without a clear distinction.\n\nThe reported improvements are small (1–3 percent), and in some cases, performance drops, showing that gains are not stable.\n\nThe absolute performance remains low (32.79 percent intrusion accuracy), and the Rider class fails completely (0.0 percent) due to confusion with Person and class imbalance. The paper does not address this failure.\n\nComparisons with other strong open-vocabulary detection systems, such as YOLO-World and Grounding DINO, are missing, which weakens the SOTA claim.\n\nThe dataset labeling relies on an automatic 20-pixel overlap rule with limited manual verification. There is no inter-annotator agreement, threshold sensitivity analysis, or bias auditing.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "mqnv2LVoiF", "reviewer": "Reviewer_aUMD", "summary": "The paper introduces OVID, an “open-vocabulary intrusion detection” setting, a new dataset (Cityintrusion-OpenV) built from Cityscapes with 8 intruder classes plus AoI masks, and a baseline model OVIDNet built on OpenSeeD with two add-ons: Multi-Distributed Noise Mixing (MDNM) for box jitter and a Dynamic Memory-Gated (DMG) module. Results show modest gains over OpenSeeD and prior intrusion systems, plus qualitative demos.", "questions": "Besides mentioned weakness, following are few questions for authors - \nQ1. What are the exact novel classes (if any) absent from training in your zero-shot claims?\nQ2. How is class text prompted at test time—single label (“bus”) or templates (“a photo of a bus”)?\nQ3. Did you try class re-weighting or few-shot prompts to fix Rider?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "S1. Problem formulation: Open-vocabulary intrusion detection as a two-stage process: first produce text-conditioned boxes and masks, then make an intrusion decision by measuring overlap with the area of interest. This separation makes the approach easy to follow.\n\nS2. Dataset contribution: Cityintrusion-OpenV extends prior intrusion datasets to eight intruder categories and pairs every image with area-of-interest masks and text prompts. It reports about eighteen interactions per image and gives class-wise counts. This supports both open-vocabulary evaluation and the intrusion decision.\n\nS3. Plug-and-play memory module: The Dynamic Memory-Gated block retrieves a small set of scene features from a fixed memory and fuses them with the decoder through a learned gate. It is a lightweight add-on that aims to capture repeated road layouts and common spatial patterns in driving scenes. The module can be dropped into similar transformer decoders without major changes.\n\nS4. Multi-Distributed Noise Mixing perturbs boxes rather than images. It mixes uniform, gaussian, and Laplace jitters with weights and scales the jitter by box area using a log-like schedule. Small boxes get fine perturbations, large boxes get broader jitter. This is well aligned with an intrusion task where a few pixels of overlap can flip the decision.", "weaknesses": "W1. The zero-shot part is mainly domain shift (COCO→Cityscapes). The novel-class aspect is unclear because most classes (person, car, bus, truck, train, motorcycle, bicycle) appear in COCO. Provide a base/novel split where some intruder classes are withheld during training and evaluated only via text prompts (standard OVD protocol). Compare to strong OVD baselines under the same split. (Table 2, Table 10 show no explicit novel-class setup.)\n\nW2. The intrusion decision uses overlap > 20 pixels (fixed) following Sun et al. (Appendix A.1; Sec. 4.2), independent of image scale (800 vs 1200, Table 9 p. 17). This is confusing to me. Report analyses with a normalized overlap (e.g., fraction of box pixels inside AoI) and calibration curves/ROC, and show sensitivity to the threshold t.\n\nW3. The paper argues that OVD+OVS pipelines are costly (Appendix A.5 Table 8 p. 16) but does not report numbers. Please include GroundingDINO(+CLIP) + panoptic/road segmenter (e.g., OpenSeeD/SAM-family) as a task baseline, and a pure OpenSeeD baseline with the exact same training/compute. Also test YOLO-World style detectors with AoI overlap. This is necessary to show OVIDNet improves over obvious modular alternatives. \n\nW4. Improvements over OpenSeeD are ~+2.19 PQ and +3.43 Acc (Table 2 p. 7), and category-wise results contain zeros (e.g., Rider = 0 in several tables; Fig. 6 p. 9). The practical impact of MDNM/DMG is seems modest; please include statistical variance (≥3 seeds) and report confidence intervals, if you have it.\n\nW5. Because of GPU limits, training uses 15k iters and 800px images (Table 9 p. 17). This deviates from OpenSeeD’s stronger recipe (368,750 iters, 1200px). Comparisons must either match schedules/crops or adjust for capacity (e.g., stronger OpenSeeD backbones). \n\nW6. Many visuals imply Road is the AoI (Fig. 5 p. 8; Fig. 10 p. 18), but practical systems often include sidewalks/rails or multiple AoIs. Please clarify whether stuff classes beyond Road are supported and evaluate multi-AoI cases.\n\nW7. Why selectively synthetic fog is only evaluated? For open-world deployment, other distribution-shifts such as common corruptions [1] (rain, snow, so on adapted to Cityscapes) and viewpoint change and perspective distortion [2] (Möbius/MPD augmentation). The scope and reasoning should be clearly defined in paper. Such investigation on robustness of AoI–box consistency under viewpoint change and artifacts can provide worthful insights.\n[1] - Hendrycks, Dan, and Thomas Dietterich. \"Benchmarking neural network robustness to common corruptions and perturbations.\" arXiv preprint arXiv:1903.12261 (2019).\n[2] - Chhipa, P. C., Chippa, M. S., De, K., Saini, R., Liwicki, M., & Shah, M. (2024, September). Möbius transform for mitigating perspective distortions in representation learning. In European Conference on Computer Vision (pp. 345-363).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)