Joint Geometrical and Statistical Domain Adaptation for Cross-domain Code Vulnerability Detection
Authors: Qianjin Du
Shiji Zhou
Xiaohui Kuang
Gang Zhao
Jidong Zhai
Venue: EMNLP 2023 Main (2023)
Keywords: Contrastive Learning
Code Vulnerability
Cross-domain Detection

Abstract: In code vulnerability detection tasks, a detector trained on a label-rich source domain fails to provide accurate prediction on new or unseen target domains due to the lack of labeled training data on target domains. Previous studies mainly utilize domain adaptation to perform cross-domain vulnerability detection. But they ignore the negative effect of private semantic characteristics of the target domain for domain alignment, which easily causes the problem of negative transfer. In addition, these methods forcibly reduce the distribution discrepancy between domains and do not take into account the interference of irrelevant target instances for distributional domain alignment, which leads to the problem of excessive alignment. To address the above issues, we propose a novel cross-domain code vulnerability detection framework named MNCRI. Specifically, we introduce mutual nearest neighbor contrastive learning to align the source domain and target domain geometrically, which could align the common semantic characteristics of two domains and separate out the private semantic characteristics of each domain. Furthermore, we introduce an instance re-weighting scheme to alleviate the problem of excessive alignment. This scheme dynamically assign different weights to instances, reducing the contribution of irrelevant instances so as to achieve better domain alignment. Finally, extensive experiments demonstrate that MNCRI significantly outperforms state-of-the-art cross-domain code vulnerability detection methods by a large margin.

Joint Geometrical and Statistical Domain Adaptation for Cross-domain Code Vulnerability Detection
Qianjin Du', Shiji Zhou², Xiaohui Kuang", Gang Z Jidong Zhai
'Dpparmeett of Computer Science and Technology, Tsinghua University
*Department Automation, Tsinghua University
National Key Laboratory of Science and Technology on Information System Security daj20@mails tswsshua..na... .dw. cn; zhaijidong@tsinghual edu. cn zhoushiji@00gmaiil. com com
xhkuang@bupt.cm edu. cn; bi bisezhaog@ 63. com
Abstract
et al.. 2016; Li et al.. 2018; Zou al., 2019; Zhou et al., 2019; Feng et al., 2020) have attempted introduce deep learning and NLP techniques for automated vulnerability detection. Those deep learning-basee detection methods mainly focus on in-domain code vulnerability detection. That is, the training and test datasets are assumed to be drawn from the same distribution (i.e., from the same soft- ware project). However, in practice, most software projects lack labeled vulnerability datasets. Label- ing data is labor-intensive and expensive, which re- quires that the models trained on a label-rich source domain (i.e., a software project with abundant la- beled data) are applied to a new target domain (i.e., software project without labeled data). Owing to program style, application scenarios, and other fac- tors, different software projects may obey different probability distributions. The models trained on the source domain suffer from severe performance loss on the target domain due to the obvious dis- tribution discrepancy between two domains. To solve this problem, some research works have at- tempted to introduce domain adaptation to perform cross-domain code vulnerability detection. Domain adaptation is technique that aims to eliminate distribution discrepancy and transfer knowledge from a labeled source domain to an unlabeled tar- get domain (Ganin and Lempitsky, 2015). Lin et al. (2018) first proposed a function-Level cross-domain code vulnerability detection framework. But they only hypothesize that the learned high-level repre- sentations are transferable, and they do not propose method to explicitly climinate the domain discrep- ancy. Lin et al. (2019) proposed a cross-domain detection framework that utilizes two LSTMs learn two groups of transferable feature represen- tations, and then combine both groups of feature representations to train a classifier. Nguyen et al. (2019) proposed a new code domain adaptation framework named SCDAN that incorporated con- ventional domain adaptation techniques such
code vulnerability detection tasks, de- tector trained label-rich source domain fails to provide accurate prediction on new or unseen target domains due to the lack of la- beled training data on target domains. Previ- ous studies mainly utilize domain adaptation perform cross-domain vulnerability detec- tion. But they ignore the negative effect of private semantic characteristics of the target domain for domain alignment, which easily causes the problem of negative transfer. In ad- dition, these methods forcibly reduce the dis- tribution discrepancy between domains and do not take into account the interference of irrele- vant target instances for distributional domain alignment, which leads to the problem of ex- cessive alignment. To address the above is- sues, we propose novel cross-domain code vulnerability detection framework named MN- CRI. Specifically, we introduce mutual near- est neighbor contrastive learning to align the source domain and target domain geometri- cally, which could align the common seman- tic characteristics of two domains and sepa- rate out the private semantic characteristics of each domain. Furthermore, we introduce an instance re-weighting scheme to alleviate the problem excessive alignment. This scheme dynamically assign different weights in- stances, reducing the contribution of irrelevant instances so as to achieve better domain align- ment. Finally, extensive experiments demon- strate that MNCRI significantly outperforms sfte-of-the--at cross-domain code vulnerabil- ity detection methods by large margin.
1 Introduction
With the development of the Internet, the number vulnerabilities software has increased rapidly, which leaves software to face huge security threats. There are multiple efforts (Lin et al., 2017; Grieco
"Corresponding authors.

---

Source Domain
Sample
z
Input
0000
CE
Classifier
intb 1000
Input xj
(M4))
zj
Instonce Re-weighting
O
Feature Extractor
MMD--R
Target Domain Sample
Retrieve
Controstive Learning Paradigm Based on Mutual Nearest Neighbors
Output
00 O
RuH
Ccn
Target sample
Push
RuB
Positive source sample
00
O0 00
A Negative source sample
Memory Bank
Mutual Neighboors
Figure 1: The framework of MNCRI.
DDAN, MMD, and DIRT-T (Ganin and Lempitsky. 2015; Long et al., 2015: Shu et al., 2018). CD- VulD (Liu et al., 2020) employed a metric transfer learning framework (MTLF) (Xu et al., 2017) to learn cross-domane representations by minimizing the distribution divergence between the source do- main and the target domain. Lastly, Li et al. (2023) proposed detection framework called VulGDA that utilized graph embedding method extract comprehensive vulnerability features and a feature generator to generate domain-invariant invariant feature rep- resentations by minimizing Maximum Mean Dis- crepancy (MMD).
and the respective private semantic characteristies should separated simultaneously. (2) There will always exist some target instances that are not relevant to source instances. Current methods treat the instances equally when conducting statis- tical distribution domain alignment. Intuitively, the relevant instances should be given larger weights, and irrelevant instances should be assigned smaller weights so to decrease their importance and con- tribution domain alignment. Ignoring the rel- evance of instances will make models learn sub- optimal cross-domain representations. To address these issues, we propose novel cross-domain code vulnerability detection framework named MNCRI (as illustrated in Figure 1) based mutual near- est neighbor contrastive learning and the instance re-weighting scheme, which jointly achieves ge- ometric domain alignment and statistical domain alignment. First, we argue that geometric domain alignment should follow a principle: semantically consistent instances should be geometrically ad- jacent to each other, whether within across domains. Based on this principle, we introduce mutual nearest neighbor contrastive learning achieve the alignment of common semantic charac- teristics. Specifically, we construct mutual nearest neighbor pairs across the source domain and the target domain by leveraging the mutual neighbor rule. Then we introduce a contrastive loss func- tion that aims at reducing the feature discrepancy of mutual nearest neighbor pairs. By minimizing this contrastive loss, we achieve the alignment of
Although the belove-mentioned research works achieve considerable performance improvements, there still exist two problems that restrict further performance improvements: (1) In practice, dif- ferent domains (i.e., different software projects) share common semantic characteristics and while having their own private semantic characteristics due to different application scenarios, coding style, and code implementation. etc. Previous studies reduce the domain discrepancy by forcibly min- imizing divergence of the entire source domain and the target domain. They ignore semantic dif- ferences between two domains and allow samples with private semantic characteristics to participate in domain alignment, which is prone the prob- lem of negative transfer. Eliminating the domain discrepancy should be constrained on only aligning the common semantic characteristics (also known as the common knowledge) between two domains

---

common semantic characteristics across domains and separate out private semantic characteristics of each domaín. Second, to alleviate the problem excessive alignment, we introduce the instance re-weighting scheme that assigns different weights to different target domain instances based on the relevance with source domain instances. In this way, the effect of some target domain instances that are irrelevant source domain instances is re- duced and the effect of some relevant target domain instances is highlighted, which is beneficial for re- ducing the distribution discrepancy. Finally, we conduct extensive experiments to verify the effec- tiveness of our proposed method. The experimental results show that the proposed method outperforms current baseline methods by a large margin. In summary, our contributions
trained on label-rich project dataset is applied to new no-label project, its performance drop sharply. To address this issue, researchers have attempted to introduce domain-adaptation tech- niques into cross-domain code vulnerability de- tection tasks. Nam et al. (2013) utilized migration component analysis to map software metrics in the source and target domains to the same feature space. Hellendoorn et al. (2020) attempted uti- lize local syntactic features and global semantic features to perform file-level cross-domain vulner- ability detection. Lin ct al. (2018) first proposed a cross droms-domain code vulnerability detection frame- work at the function-level, which extracts features from abstract syntax trees (ASTs) of functions and then feeds extracted features into a long short-term memory recurrent neural network (LSTM) learn higher-level representations. But they only hypoth- esize that the learned high-level representations are transferable, and they do not propose an effective method to explicitly eliminate the discrepancy be- tween the source domain and the target domain. Lin et al. (2019) presented cross-domain detec- tion framework that employed two LSTMs to learn two groups of transferable feature representations. These representations were then combined to train classifier. Nguyen et al. (2019) proposed a novel code domain adaptation network called SCDAN. SCDAN incorporated conventional domain adapta- tion network techniques such as DDAN, MMD, and DIRT-T (Ganin and Lempitsky, 2015; Long et al.. 2015; Shu et al., 2018) to eliminate distribution divergence. To further tackle the distribution di- vergence, CD-VulD (Liu et al., 2020) deployed the metric transfer learning framework (MTLF) (Xu et al., 2017) to learn tions by minimizing Mahalanobis distance. Li et al. (2023) proposed a cross-domain detection frame- work called VulGDA, which employed a graph embedding approach to extract comprehensive vul- nerability features and feature generator to gen- crate Comain-invariaan feature representations by minimizing the domain discrepancy.
follows:
We propose novel cross-domain code vulner- ability detection framework named MNCRI, which reduces the domain discrepancy from two perspectives of geometries and statistical distributions by exploiting the mutual nearest neighbor contrastive learning and the instance re-weighting scheme.
We introduce mutual nearest neighbors con- trastive learning to solve the problem of neg- ative transfer, which explores instances rela- tionships between domains and achieves the domain alignment geometrically.
We incorporate instance re- weightitng scheme into the statistical domain alignment, which dynamically weights different instances according to the relevance between instances to eliminate the negative effect of irrelevant instances for domain alignment.
representa-
We conduct extensive experiments to verify the effectiveness of MNCRI. Experimental results show that MNCRI outperforms current cross-domain detection methods and obtain obvious performance improvements.
2
Related Work
2.2 Contrastive Learning
2.1 Cross-domain Code Vulnerability Detection
Contrastive learning in the latent space has recently shown great promise, which aims to make the rep- resentation of given anchor data to be similar its positive pairs and dissimilar to its negative pairs (Tian et al., 2020; Chen et al., 2020; Khosla et al., 2020). Many state-of of-the-r methods for represen-
Since different software projects have different code styles, application scenarios and other fac- tors, they may obey different distributions. There- fore, when a code vulnerability detection model

---

tation learning tasks are based on the contrastive learning framework (Chen and He, 2021). Previous studies construct positive samples by augmenting original samples. In this paper, we utilize mutual nearest neighbors as positive pairs to achieve fea- ture alignment between domains.
source domain the target domain. Previous meth- ods mainly focus on minimizing the global distri- bution divergence between domains to achieve the domain alignment, such as Maximum Mean Dis- crepancy (MMD) (Yan et al., 2017; Li et al., 2023). In practice, the target domain may preserve its own private semantic characteristics that do not appear in the source domain. Previous methods do not consider domain private semantic characteristics and forcefully align the global distributions of two domains, which may cause the problem of nega- tive transfer. manifold learning (MeInnes et al., 2018; Chen et al., 2022), the concept of geometric nearest neighbors is commonly used to describe the similarities of semantic characteristics. For differ- ent domains, the geometrically nearest neighbors can be viewed as the most similar samples that contain the same semantic characteristics. There- fore, to align the common semantic characteristics between domains and separate out private seman- tic characteristics of each domain, we introduce mutual nearest neighbors contrastive learning.
3 Methodology
As illustrated in Figure 1, the architecture of MN- CRI contains two main components: 1) the mu- tual nearest neighbor contrastive learning module, which performs contrastive learning based on mu- tual nearest neighbors for better geometric domain aligmment; 2) the statistical distribution domain alignment module equipped with the instance re- weighting scheme, which reduces the distribution discrepancy and achieves the domain alignment statistically, Below we elaborate the details of MN- CRI.
3.1 Problem Formulation
We formalize vulnerability detection a binary classification problem. In cross-domain vulnera- bility detection, given labeled source domain set D, = ((*f, yf) ) including n, instances a;f, aims to predict the label y of the target sample x! in an unlabeled target domain set D. = (=))); where v E {0,1} 1} and s E {0, (1 means the sample is vulnerable and 0 means the sample is non-vulnerable), n and ni are the number of the source domain set and the target domain set, re- spectively. As illustrated in Figure 1, our model consists of two basic modules: (1) a feature ex- tractor f(-) that maps the code sample 2; into the embedding representation 2; (z, = fxx))). (2) a classifier g(-) to classify the class of the sample (y) = (x:))))
Specifically, for target domain sample xt. we retrieve its corresponding k nearest neighbors N in the source domain. In the same way, for source domain sample urf. we also obtain its corresponding k nearest neighbors NWW in the target domain. Then mutual nearest neighbors can be constructed by following the rule: if the source sample 2; and the target sample xf are contained in each other's nearest neighbors, they are regarded as mutual nearest neighbors. After that, we build relationship matrix to describe the mutual near- est neighbor relationship between source domain samples and target domain samples. Formally, the relationship matrix is defined as: Mst E RR, where M = 1 if only xf and 25 is the mutual nearest neighbor pair; otherwise M 0. Be- sides, to fully utilize supervised information of the source domain and learn better discriminative fea- tures for classification, we also build in-domain relationship matrix M according to ground-truth labels of the source domain.
3.2 Geometric Domain Alignment by Mutual Nearest Neighbor Contrastive Learning Paradigm
In supervised learning scenarios, the cross-entropy loss on the source domain is defined as:
In geometric domain alignment, we hope that built mutual nearest neighbor pairs can be closer to each other in the feature space, while non- geometrically samples should be far away from each other. In this paper, we introduce an con- trastive learning loss to achieve this goal.
LCE = y log log(g(((z)))+
(1- vt) log(1 g({()))))
Due to the domain discrepancy between the source domain and the target domain, the performance drops sharply when transfer the model trained on
For each training sample Ti its positive sample sets P. and negative sample sets N; in the memory

---

bank can be obtained by relationship matrices A/** and M The contrastive loss is formulated by:
as the weights of target instances during statistical domain alignment. This process can be formulated as:
211
Coo
log Pi.
U
LIMD-IR T Σ
k(zt,
25)
(xp(2s2//T)
log Eper, cxp(212pT) Eve,, emp(z.a//T)
U
w w
ΣΣ
f( (1)))((z
ΣΣ
k<
2/1
UW
W
where is temperature parameter. By optimizing the contrastive loss, it makes the model learn geo- metric neighbor relationships between the source domain and the target domain and makes the fea- ture representations of samples with common se- mantic characteristics closer in the feature space.
1-1
1-13-
f(z)) = 2+ |Probly = 1|25) 0.5l
(4)
where Prob(y} 1|2}) denotes the prediction probability that the classifier predicts the target in- stance xt is vulnerable. Introducing the instance re-weighting scheme degrade the impact of some target instances that not relevant to source in- stances, which is helpful for reducing the distribu- tional divergence between domains and achieving better statistical domain alignment.
3.3 Statistical Domain Alignment by Instance Re-weighting Scheme
As depicted in Section 3.2, the mutual nearest neighbors contrastive learning could reduce the geometrical domain discrepancy and achieve better geometrical domain alignment. In practice, reduc- ing the distributional divergence between domains and achieving the statistical domain alignment are also important for cross-domain code vulnerability detection. MMD (Gretton et al., 2008)) a popular estimator to calculate the degree of statistical align- ment of data distribution between two domains, and it defined follows:
3.4 Overall Objective
MNCRI proposed to jointly minimize the cross entropy loss of the source domain instances and reduce the divergence between the source domain and target domain. The total objective function in MNCRI is formulated
C -= == + XL + #LMMD-IRR (5)
U
V
where LC denotes source supervised loss using source labels, and Loon denotes the geometric do- main alignment loss bascd on the mutual nearest neighbor contrastive learning, and LMMD-IR de- notes the statistical domain alignment loss using instance re-weighting scheme. À and are two trade-off parameters.
LMMD =1
25)
Σ>>
UW
W
W
(3)
Previous studies (Liu et al., 2020; Li et al., 2023)
perform statistical domain alignment by minimiz- ing Maximum Mean Discrepancy (MMD). How- ever, there will always exist some instances that are not relevant the source instances. Previous methods forcefully reduce the global distributional divergence between domains and ignore the neg- ative effect of irrelevant instances, which easily leads to the problem of excessive alignment. Ide- ally, relevant target instances should be assigned large weights and irrelevant instances should be as- signed small weights during the domain alignment. Thus, in order to achieve this goal, propose to introduce the instance re-weighting scheme into the statistical domain alignment. Specifically, we adopt the prediction confidence the classifier trained on the source domain for target instances
Experiments
In this section, we conduct extensive experiments to evaluate the effectiveness of the proposed frame- work (MNCRI).
4.1 Experimental Settings
4.1.1 Dataset
In our work, we conduct experiments on three open-source software project datasets: FFmpeg, QEMU (Zhou et al., 2019) and ReVe (Chakraborty et al., 2021). There are several reasons for choosing the above datasets as experimental datasets. The first one is that they are manually labeled so that they could provide accurate supervised information.

---

Second, the number of samples of these datasets is abundant support model training. The imple- mentation details can be found in Appendix
5.41%. Meanwhile, MNCRI (CodeT3) obtains a TPR gain of 9.18% compared to VulGDA. More- over, our framework equipped with CodeBERT, GCBERT (GraphCodeBERTT also can obtain sig- nificant performance improvements compared to baseline methods. This demonstrates that the pro- posed framework has good generalization perfor- mance and it does not rely on the specific code feature extractor and can be adapted various feature extractors. Since CodeT5 adopts more ef- fective pre-training mechanisms compared Code- BERT and GCBERT (GraphCodeBERT), MNCRI using CodeT5 as the feature extractor obtains the largest performance improvements. This demon- strates that a power feature extractor is helpful for improving cross-domain vulnerability detection performance. To make the comparison with Vul- GDA, MNCRI (VulGDA-f) uses the same feature extractor as VulGDA. We can observe that MNCRI (VulGDA-f) consistently outperforms VulGDA. We can conclude that the performance improvements can be attributed to the proposed domain adaptation strategies.
4.1.2 Baselines
The state-of-the-art cross-domen code vulnerabil- ity detection methods include: VulDeePecker (Li et al., 2018), SySeVr (Li et al., 2021). Devign (Zhou et al., 2019), CodeBERT (Feng et al., 2020), GraphCodeBERT (Guo et al., 2020), CodeTS (Wang et al., 2021), SCDAN (Nguyen et al., 2019), CD-VulD (Liu et al., 2020), and VulGDA (Li et al.. 2023).
4.1.3 Evaluation Metric
In general, Fl-score is widely used metric to eval- uate the performance of vulnerability detection sys- tems. Let TPPPP be the number of vulnerable samples that are predicted correctly, FP be the number of truly non-vulnerable samples that are predicted as vulnerable and FN be the number of truly vulner- able samples that are predicted as non-vulnerable. TP Precision (P TPTFT) represents the ratio of vulnerable samples that are correctly predicted as vulnerable out of all samples predicted as vulnera- TP ble. Recall (R TP-FN) represents the ratio of vulnerable samples that are correctly predicted as vulnerable out of all truly vulnerable samples. The
By analyzing experimental results, we can ob- serve that MNCRI could significantly improve TPR (True Positive Rate) of the detector compared to other cross-domain vulnerability detection meth- ods. This observation shows that mutual nearest neighbor contrastive learning and the instance re- weighting scheme are conducive to reducing do- main discrepancy by jointly conducting geometric domain alignment and statistical domain alignment. Besides, compared to the detectors CD-VulD, Vul- GDA, and MNCRI exploiting domain adaptation techniques, traditional detectors VulDecPecker, Sy- SeVr and Devign have the worse performance in cross-domain detection tasks. This is because tra- ditional detectors do not consider the domain gap between domains. The comparison results indi- cate the importance of domain adaptation in cross- domain code vulnerability detection tasks.
Fl Fi-score (F1 = Px is the harmonic mean of precision and recall. is clear that the higher F1- score represents better classification performance. Besides, vulnerability detection tasks also focus on TPR (True Positive Rate) performance (as known Recall) of detectors. Because the higher TPR represents that the more vulnerable samples are discriminated correctly, is crucial for the perfor- mance evaluation of detectors. Therefore, our experiments, we adopt TPR (True Positive Rate) and Fi-score as our evaluation metrics.
4.2 Results
We conduct extensive experiments to verify the effectiveness the proposed framework (MN- CRI). Experimental results are shown in Table
In order to more intuitively demonstrate the clas- sification performance of MNCRI in cross-domain vulnerability detection tasks, we plot the histogram distributions of softmax-based prediction confi- dence on the target domain dataset (sce Figure 2). We can see that the prediction confidence scores of VulGDA are mainly distributed around 0.5 (see Fig- ure 2(a)). This phenomenon indicates that VulGDA has no discriminative ability for a large number of target instances and is unable to judge whether
1. Our proposed framework MNCRI (CodeT5) achieves the best performances on all cross-domain code vulnerability detection tasks, and outper- forms all baseline methods. For instance, in the TFmpeg-tt-QENNe (F Q) cross-domain detec- tion task, MNCRI (CodeT5) achieves an Fi-score 58,21%, outperforming the current statc-of-the art cross-domain detection method VulGDA by

---

F-0 Q
F-R
Q
Q
R
R
Methods
TPR
TPR
TPR
FI
TPR
F1
TPR
FT
VulDeePecker
64.82
41.31
12.59
12.47
34.85
32.04
33.98
12.04
35.57
29.01
16.57
13.88
SySeVr
66.98
43.22
12.73
13.19
35.57
33.22
35.77
12.96
37.28
30.05
17.65
14.36
Devign
68.92
45.61
14.58
13.31
37.99
35.91
39.82
13.55
35.56
31.35
18.97
16.28
CodeBert
53.22
46.66
10.66
11.22
37.44
38.15
30.02
12.97
30.66
33.59
15.21
18.34
GCBERT
55.34
47.14
10.79
11.30
39.40
38.60
31.09
13.68
30.97
33.98
16.88
18.77
CodeT5
58.57
49.50
11.23
12.11
38.80
39.42
33.70
15.02
31.77
35.73
18.22
20.39
SCDAN
70.30
52.32
13.99
12.68
40.78
43,30
4.329
17.65
38,92
37.05
15.29
22.21
CD-VulD
73.21
51.24
13.88
14.06
48.80
46.68
48.80 80
20.10
40.25
40.17
18.71
25.78
VoIGDA
76.76
52.80
15.22
14.37
47.96
47.72
50.52
20.98
40.68
42.86
19.88
26.35
MNCRI (VuIGDA-F)
81.37
54,95
16.21
15.44
50.39
50.99
51.39
22.37
41,38
45.92
21.69
29.32
MNCRI (CodeBERT)
84.55
56.57
17.11
16.68
53.22
53.96
53.88
23.10
42.56
48.10
21.69
29.32
MNCRI (GCBERT)
84.92
57.09
17.92
17.02
54.10
54.11
54.74
24.97
44.74
49.83
22,28
30.10
MNCRI (CodeT5)
85.94
58.21
18.66
17.51
56.70
55.42
57.09
25.65
46.60
50.22
24.41
31,50
Table 1: Results on cross-domain code vulnerability detection tasks. For simplicity, GCBERT denotes the Graph- CodeBERT (Guo et al., 2020) model. MNCRI (VulGDA-f) denote that MNCRI use the same feature extractor VulGDA. MNCRI (CodeT5) denotes that our framework MNCRI uses CodeTT5 as the feature extractor. F denotes the FFmpeg project dataset, and Q denotes the QEMU project dataset, and R denotes the Reveal project dataset. F Q represents the source domain is the FFmpeg project dataset and the target domain is the QEMU dataset.
4.3 Ablation Study
To analyze the impact of different components our proposed MNCRI on the performance, we con- duct an ablation study on all cross-domain vulnera- bility detection tasks and report the results in Table 2.
(a) VulGDA
Effect of Mutual Nearest Neighbors Con- trastive Learning. From the results reported in Table 2, observe that the removal of the mu- tual nearest neighbors contrastive learning ("w/o Lcon") ") sharply reduces the performance in all eval- uation metrics (TPR and F1). This indicates that contrastive learning based on mutual nearest neigh- bors guarantees geometric domain alignment by pulling mutual nearest neighbors pairs closer to each other and pulling away non non-gemmetrically close samples. Furthermore, we remove the mem- ory bank mechanism ("w/o MB") in the contrastive learning training stage and record its results. We can find that removing the memory bank mecha- nism would degrade performance. This indicates that the global semantic information containing in the memory bank is conducive to achieving more accurate domain aligmment.
(b) MNCRI
Figure 2: Histogram of softmax-based prediction confi- dence on the target domain test dataset. (a) is the predic- tion confidence score of VulGDA; (b) is prediction confidence score of MNCRL x-axis coordinate repre- sents the prediction confidence score from 0 to 1 and y-axis coordinate represents corresponding counts.
target instances are vulnerable or non-vulnerable. In contrast, our proposed framework MNCRI over- comes this phenomenon (see Figure 2(b)). We can observe that there are few prediction confidences distributed around 0.5, which shows that MNCRI learns the more accurate vulnerable patterns across domains and has the superior discriminative ability for target instances.
Effect of Instance Re-weighting Scheme. To evaluate the contribution of the instance re- weighting scheme, we abandon it and report its results in Table 2. We see that the removal of the instance re-weighting scheme ("w/o IR") leads to performance degradation. This implies that assigning different weights to different instances according to relevance is helpful for reducing dis- tributional divergence between domains, which en-
Overall, the promising performance of MNCRI can be attributed to the mutual nearest neighbor- based contrastive learning and the instance re- weighting scheme.

---

F-Q
F-R
Q
Q-R
R
R
Q
Methods
TPR
FT
TPR
TPR
TPR
FT
TPR
FT
MNCRI (Full)
85.94
58.21
18.66
17.31
56.7
55.42
$7.09
25.65
46.60
50.22
24.41
31.50
MNCRI (w/o Lcon)
79.74
54.30
16.92
15.29
49.99
49.37
$2.69
22.13
42.72
45.35
20.61
28.17
MNCRI (w/o MB)
83.49
55.76
17.24
16.66
53.85
52.28
55.37
23.48
43,92
48.50
23.69
30.22
MNCRI (w/o IR)
84.87
56.87
18.04
16.65
55.12
54.39
56.21
24.71
45.98
49.74
24.01
30.85
Table 2: Experimental results of ablation study.
domain vulnerability detection task. From the ex- perimental results shown in Table 3, we can see that MNCRI outperforms all baseline methods under the few-shot condition. This verifies the effective- and generalizability of MNCRI in dealing with both zero-shot and few-shot vulnerability detection.
Neighbors number &
(b) Loss weights À and j
Methods
TPR
Figure 3: (a) Varying nearest neighbors number k. (b) Varying loss weights \ and μ.
SCDAN
74.51
52.67
CD-VulD
79.32
53.45
VulGDA
79.05
54.88
ables the model to derive better cross-domain rep- resentations, and thus leads to improved MNCRI performance.
MNCRI
86.33
59.89
Table 3: Experimental results of few-shot condition
with 20% target domain labeled samples for training.
4.4 Sensitivity to hyper-parameter
Impact of the nearest neighbors number k. To show the sensitivity of MNCRI to the nearest neigh- bors number k. we conduct the experiments on the F F Q cross-domain vulnerability detection task, and show the results in Figure 3(a). We observe that the Fl-score increases with the increasing value of A and peaks k = 20. Further increasing the values of l results in worse performance. The Fl-score varies slightly within a wide range of k; demon- strating that MNCRI is robust to the choices of k.
4.6 Conclusion
In this paper, we propose a novel cross-domain code vulnerability detection framework named MNCRI achieve domain alignment. On the hand, we introduce mutual nearest neighbors con- trastive learning to Iearn geometric relationships between source domain samples and target domain samples, so as to achieve geometric domain align- ment. On the other hand, we introduce the instance re-weighting scheme to reduce distributional di- vergence between domains and achieve statistical domain alignment. Finally, we conduct extensive experiments to verify the effectiveness of our pro- posed framework.
Impact of loss weights À and To show the sen- sitivity of MNCRT to the loss weights À and 14, we conduct control experiments on the F + Q cross- domain vulnerability detection task. The results are illustrated in Figure 3(b). We can observe that MNCRI achieves the best performance when is set to 0.2. Within a wide range of X, the perfor- mance of MNCRI changes very little, validating that MNCRI stable to the choices of X. Simi- larly, MNCRI varies slightly with E [0.01, 0.3). showing that MNCRI is robust to the selection of μ..
Limitations
Here we summarize the limitations for further dis- cussion and investigation of the community. Our proposed k-nearest neighbor mechanism requires manually setting the optimal k* to achieve the best cross-domain detection performance, which is in- efficient. A better solution is design an adaptive k-nearest neighbor mechanism which could adap- tively seek optimal neighbor samples according the semantic information of training samples.
4.5 Analysis of Few-Shot Condition
To evaluate the generalizability of MNCRI in few- shot vulnerability detection, we also evaluate MN- CRT in the few-shot condition on the F + Q cross-

---

References
Xin Li, Yang Xin, Hongliang Zhu, Yixian Yang. and Yuling Chen. 2023. Cross-domain vulnerability de- tection using graph embedding and domain adapta- tion. Computers & Security, 125:103017.
Saikat Chakraborty, Rahul Krishna, Yangruibo Ding, and Baishakhi Ray. 2021. Deep learning based vul- nerability detection: Are we there yet. IEEE Trans- actions Software Engineering.
Zhen Li, Deqing Zou, Shouhuai Xu. Hai Jin, Yawei Zhu, and Zhaoxuan Chen. 2021. Sysevr: A frame- work for using deep learning to detect software vul- nerabilities. IEEE Transactions on Dependable and Secure Computing.
Liang Chen, Qianjin Du, Yihang Lou, Jianzhong He,
Tao Bai, and Minghua Deng. 2022. Mutual nearest neighbor contrast and hybrid prototype self-training for universal domain adaptation. In Proceedings of the AAAI Conference on Artificial Intelligence, vol- ume 36, pages 6248-6257.
Zhen Li, Deqing Zou, Shouhuai Xu, Xinyu Ou, Hai Jin, Sujuan Wang. Zhijun Deng, and Yuyi Zhong. 2018. Vuldeepecker: A deep learning-based sys- tem for vulnerability detection. arXiv preprint arXiv:
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020. simple framework for contrastive learning of visual representations. In In- ternational conference on machine learning, pages 1597-1607, PMLR.
Guanjun Lin, Jun Zhang, Wei Luo, Lei Pan, Olivier De Vel, Paul Montague, and Yang Xiang. 2019. Software vulnerability discovery via learning multi- domain knowledge hases. IEEF Transactions on De- pendable and Secure Computing, 18(5):2469 2485.
Xinlei Chen and Kaiming He. 2021. Exploring simple siamese representation learning. In Proceedings of the ICEE/CVF conference computer vision and pattern recognition, pages 15750-15758.
Guanjun Lin, Jun Zhang, Wei Luo. Lei Pan, and Yang Xiang. 2017. Poster: Vulnerability discovery with function representation learning from unlabeled projects. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Se- curity, pages
Zhangyin Feng. Daya Guo, Duyu Tang, Nan Duan, Xi- aocheng Feng, Ming Gong. Linjun Shou, Bing Qin,
Ting Liu, Daxin Jiang, et al. 2020. Codebert: A pre-trained model for programming and natural lan- guages. arXiv preprint arxiv:2002.08155.
Guanjun Lin, Jun Zhang, Wei Luo. Lei Pan, Yang Xiang, Olivier De Vel, and Paul Montague. 2018. Cross-project transfer representation learning for vulnerable function discovery. IEEE Transactions on Industrial Informatics, 14(7):328933297,
Yaroslav Ganin and Victor Lempitsky. 2015. Unsuper- vised domain adaptation by backpropagation. In In- ternational conference on machine learning, pages 1180-1189. PMLR.
Shigang Liu, Guanjun Lin, Lizhen Qu, Jun Zhang. Olivier De Vel, Paul Montague, and Yang Xiang. 2020. Cd-vuld: Cross-domale vulnerability dis- covery hased on deep domain adaptation. IEEF Transactions Dependable and Secure Computing, 19(1):438
Arthur Gretton, Karsten M. Borgwardt, Malte J. Rasch, Bernhard Schölkopf, and Alexander J. Smola. 2008. kernel method for the two-sample problem. CoRR, abs/0805.2366
Gustavo Grieco, Guillermo Luis Grinblat, Lucas Uzal, Sanjay Rawat, Josselin Feist, and Laurent Mounier. 2016. Toward acage-scale vulnerability discovery us- ing machine learning. In Proceedings of the Sixth ACM Conference Data and Application Security and Privacy, pages 85-96.
Mingsheng Long, Yue Cao. Jianmin Wang, and Michael Jordan. 2015. Learning transferable fea- tures with deep adaptation networks. In Interna- tional conference on machine learning, pages 97- 105. PMLR
Leland McInnes, John Healy, and James Melville. 2018. Umap: Uniform manifold approximation and projection for dimension reduction. arXiv preprint 182.0:680..
Daya Guo, Shuo Ren, Shuai Lu, Zhangyin Feng, Duyu Tang, Shujie Liu, Long Zhou, Nan Duan, Alexey Svyatkovskiy, Shengyu Fu, et al. 2020. Graphcode- bert: Pre-training code representations with data flow. arXiv preprint arXiv:2009.08366
Jaechang Nam, Sinno Jialin Pan, and Sunghun Kim. 2013. Transfer defect learning. In 2013 35th inter- national conference on software engineering (ICSE), pages 182-391. IEEE
Vincent Josua Hellendoom, Petros Maniatis, Rishabh Singh, Charles Sutton, and David Bieber. 2020. Global relational models source code.
Van Nguyen, Trung Lc. Tue Le, Khanh Nguyen, Olivier De Paul Montague, Lizhen Qu, and Dinh Phung. 2019. Deep domain adaptation for vulner- able code function identification. In 2019 interna- tional joint conference on neural networks (LJCNN).
Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. 2020. Su- pervised contrastive learning. Advances in neural information processing systems, 33: 33:18661-1861
pages 1-8. IEEE.

---

Rui Shu, Hung Bui, Hirokazu Narui, and Stefano Ermon. 2018. A dirt-t approach to unsupervised do- main adaptation. arXiv preprint arXiv: 1oXXx:/802... 0802.08335
Yonglong Tian, Dilip Krishnan, and Phillip Isola. 2020. Contrastive multiview coding. In Computer Vision- ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Purt XI 16, pages 794.794. Springer.
Yue Wang, Weishi Wang. Shafiq Joty, and Steven CH Hoi. 2021. Codet5: Identifier-aware unified pre-trained encoder-decoder models for code un- derstanding and generation. arXiv preprint 1rX:v:2109.00559.
Yonghui Xu, Sinno Jialin Pan, Hui Xiong, Qingyao Wu, Ronghua Luo. Huaqing Min, and Hengjie Song. 2017. unified framework for metric transfer learn- ing. IEEE Transactions on Knowledge and Data En- gineering, 2966):1158-117)
Hongliang Yan, Yukang Ding, Peihua Li, Qilong Wang, Yong Xu, and Wangmeng 7uo. 2017. Mind the class weight bias: Weighted maximum mean discrepancy for unsupervised domain adaptation. In Proceedings of the IEEE conference on computer vision and pat- tern recognition, pages 2272-2281.
Y. Zhou, S. Liu, J. Siow, X. Du, and Y. Liu. 2019. De- vign: Effective vulnerability identification by learn- ing comprehensive program semantics via graph neural networks.
Deging Zou, Sujuan Wang, Shouhuai Xu, Zhen Li, and Hai Jin. 2019. jvuldeepecker: deep learning- based system multiclass vulnerability detection. IEEE Transactions Dependable and Secure Com- puling.
A Implementation Details
We use CodeBERT, GraphCodeBERRT, CodeT5 as the feature extractor, respectively. We use cross- entropy to calculate the classification loss. The classifier is 2-layer full connected layers with 256- dimensional output, and uses MMD measure the distributional domain discrepancy. The initial learning rate set to 5e-4. We split source and target samples into different mini-batches with size 36. The total training epoch is 10. The temperature parameter 7 is set to 0.05.


Reviews:
[{"review_id": "1uI74tjmZu", "reviewer": "Reviewer_hFha", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "# About baseline methods\nIn this paper, to make a fair comparison, we adopt the same experimental conditions (including datasets and experimental settings) as the latest baseline VulGDA. Since links are prohibited in the rebuttal material, we will release the source code of baselines in the revised paper. To verify our proposed method does not rely on specific feature extractors (e.g., PLM models), we conduct supplementary experiments that adopt the same model architecture as SCDAN without using PLM models. The results are shown in the following table. Compared to SCDAN, MNCRI obtains an average relative F1 gain of 3.1%. This indicates that our method does not rely on the specific feature extractor.\n\n|Methods|F -> Q|F -> R|Q -> F|Q -> R|R -> F|R -> Q|\n|:------------------------------------|:---:|:---:|:---:|:---:|:---:|:---:|\n| SCDAN |52.32|16.66|52.28|23.48|48.50|30.22|\n| MNCRI(SCDAN-f) |56.87|16.65|54.39|24.71|49.74|30.85|\n\n# About novelty attributed to the proposed method \nAlthough our proposed method and the approach presented in [1] both employ the k-nearest neighbor mechanism, our method is fundamentally inconsistent with paper [1]. Paper [1] infers target labels by integrating the labels of $k$ neighbors of target samples from the source domain. Our method first retrieves $k$ nearest neighbors of source samples from the target domain and $k$ nearest neighbors of target samples from the source domain. Subsequently, we construct source-target mutual nearest neighbor pairs by follow the rule: if a source sample $x_i^s$ is the neighbor of a target sample $x_j^t$ and $x_j^t$ is the neighbor of $x_i^s$, $x_i^s$ and $x_j^t$ are the mutual nearest neighbor pair. Finally, we exploit a novel contrastive learning paradigm that pulls these nearest neighbors closer to each other and pushing away those non-nearest neighbor samples. In this way, it enables the model to learn more discriminative representations and achieve better semantic transfer. Compared to our method, paper [1] only uses the k-nearest neighbor mechanism to predict target labels and does not focus on learning discriminative cross-domain representations. \n\nThe re-weighting scheme is a general method that is widely used in various deep learning application scenarios, such as attention mechanisms equipped in Transformer and instance re-weighting methods for imbalanced classification. Different domains adopt different instance re-weight schemes to address their own domain issues according the different characteristics of domains.  These proposed instance re-weighting schemes bring performance improvements and have been proven to be novel. In our work, we mainly focus on improving cross-domain code vulnerability detection performance. In cross-domain vulnerability detection tasks, there are some target instances that are not relevant to source instances. Previous studies reduce the domain discrepancy by forcibly minimizing statistical distribution divergence (MMD) of the entire source domain and the target domain, which easily leads to the problem of excessive alignment. In our work, we argue that assigning large weights to relevant instances and small weights to irrelevant instances could alleviate the problem of excessive alignment and improve cross-domain code vulnerability detection performance. Compared to previous instance re-weighting research works that utilize solvers (e.g., such as the research work presented in [2]) or complex optimization methods to obtain instance weights, our proposed instance re-weighting scheme is simple yet effective. Specifically, we adopt the prediction confidence of the classifier trained on the source domain for target instances as the weights of target instances during statistical domain alignment. Introducing the instance re-weighting scheme mitigates the adverse influence of some target instances that are not relevant to source instances, which is helpful for alleviating the problem of excessive alignment. Experimental results prove that the effectiveness of the proposed instance re-weighting scheme.  \n\n[1] Sener, Ozan, Hyun Oh Song, Ashutosh Saxena, and Silvio Savarese. \"Learning Transferrable Representations for Unsupervised Domain Adaptation.\" NIPS (2016).\n\n[2] Dynamic re-weighting for long-tailed semi-supervised learning.  NIPS (2011).\n\n\n\n# About Equation 3\nWe apologize for our imprecise formula definition. In Equation 3, $U = n_s$ and $W = n_t$. Thanks for your valuable suggestions. We will fix this problem in our revised paper. \n\nThanks for your insightful suggestions. We will polish the paper based on these valuable points in the revised version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The identification of the two proposed problems, namely negative transfer and excessive alignment, provides an intriguing perspective.\n- The performance achieved in the experiments is surprisingly impressive.", "reasons_to_reject": "- There appears to be some confusion regarding the fairness of the comparison in the results. The baseline methods, such as SCDAN, CD-VulD, and VulGDA, referenced in the paper, were not evaluated under the same experimental conditions (like dataset). This has led to uncertainty about whether the competitive results presented in the paper were reproduced by the authors themselves. While the supplementary code is provided, it lacks inclusion of the above referred method. Additionally, considering that SCDAN is solely LSTM-based, it might be beneficial to incorporate PLM-based techniques along with traditional domain adaptation approaches (e.g., DANN). This would help emphasize the significance of the proposed problem—negative transfer and excessive alignment—and consequently validate the effectiveness of the proposed methodology.\n\n- The level of novelty attributed to the proposed method for cross-domain code vulnerability detection appears to be somewhat limited. It's worth noting that k-Nearest Neighbor based adaptation [1] has previously been explored, and the concept of instance weighting [2] is not a new or groundbreaking technique. But admittedly, this paper is the first to apply these methods within the specific field of cross-domain code vulnerability detection.   \n[1] Sener, Ozan, Hyun Oh Song, Ashutosh Saxena, and Silvio Savarese. \"Learning Transferrable Representations for Unsupervised Domain Adaptation.\" NIPS (2016).   \n[2] Sun, Qian, Rita Chattopadhyay, Sethuraman Panchanathan, and Jieping Ye. \"A Two-Stage Weighting Framework for Multi-Source Domain Adaptation.\" NIPS (2011).\n", "questions_to_authors": "", "paper_topic_and_main_contributions": "Centering on the objective of detecting code vulnerabilities across different domains, the paper tackles the challenges of negative transfer and excessive alignment. The primary contributions of this study encompass the introduction of an innovative framework, MNCRI, which leverages mutual nearest neighbor contrastive learning and instance reweighing to achieve enhanced alignment. Through extensive experiments, the approach is shown to markedly outperform existing state-of-the-art methods for cross-domain code vulnerability detection.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "- It seems that there is a missing illustration for Equation 3 in the context you provided. IIUC, maybe U should be written in n_s, and W should be written in n_t.\n\n- The organization of the introduction appears to be problematic, as it attempts to convey a significant amount of information within just two paragraphs spanning two pages. This dense presentation makes it challenging for readers to grasp the content effectively. "}}, {"review_id": "9H5EvFricP", "reviewer": "Reviewer_f7o6", "summary": "", "questions": "1. The details of the datasets provided in the appendix appear to have a balanced distribution of positive and negative examples. However, in real-world applications, scenarios with fewer vulnerabilities may introduce additional difficulties. \n2. If the code has many different types of vulnerabilities with significant characteristic differences, considering all diverse vulnerability features as a single category (i.e., “vulnerable”) in a binary classification model may be problematic. \n3. Can the authors provide an ablation experiment that eliminates the use of Maximum Mean Discrepancy (MMD) loss for statistical domain alignment to understand whether statistical domain alignment without the Instance Re-weighting Scheme has a negative or small positive effect on experimental results?", "limitations": "", "rating": 3, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "# About vulnerability dataset\nIn our work, in order to make a fair comparison with other baseline methods, we adopt the same datasets as baselines. The current published vulnerability datasets commonly have a balanced ratio of positive and negative examples to learn discriminative vulnerability representations. As you described, in real-world applications, the number of vulnerable samples is less than that of non-vulnerable samples. The class imbalance problem is a hot research topic in deep learning field. Some research works was proposed to solve class imbalance problem, such as focal loss, re-sampling strategies, etc. Due to the lack of imbalanced datasets in the code vulnerability detection field, we cannot report corresponding performance. But our method can solve class imbalance problem by equipping with class-imbalance learning methods (e.g., focal loss or re-sampling strategies).\n\n# About binary classification model \nThe goal of the code vulnerability detection task is to determine whether a piece of code is secure and non-vulnerable rather than to identifying types of vulnerabilities exist in the code. Following previous research works, such as VulGDA, vulnerability detection is viewed as a binary classification problem. Of course, different types of vulnerabilities may have different characteristics. But the semantic characteristics of non-vulnerable code samples are consistent and are distinctly different from the semantic characteristics of vulnerable samples. Therefore, current research works and vulnerability dataset collection works are mainly based on binary classification tasks. Furthermore, published vulnerability datasets with multiple types of vulnerabilities are scarce due to the expensive annotation costs. We appreciate your insightful suggestions. We plan to build multi-category datasets and explore multi-class code vulnerability detection tasks in our future work.\n\n# About the ablation experiment that eliminates the use of Maximum Mean Discrepancy (MMD) loss for statistical domain alignment\n\nWe conducted an ablation experiment that eliminates the use of Maximum Mean Discrepancy (MMD) loss for statistical domain alignment. The results are shown in the following table. MNCRI (w/o L_MMD) denotes that our proposed method without the MMD loss for statistical domain alignment. We can see that omitting the MMD loss for statistical domain alignment results in a decrease in performance. This demonstrates that statistical domain alignment without the Instance Re-weighting Scheme still has a positive effect on experimental results and plays a constructive role in the cross-domain code vulnerability detection tasks.\n|Methods|F -> Q|F -> R|Q -> F|Q -> R|R -> F|R -> Q|\n|:------------------------------------|:---:|:---:|:---:|:---:|:---:|:---:|\n|MNCRI (Full) |58.21|17.51|55.42|25.65|50.22|31.50|\n|MNCRI (w/o L_Con) |54.30|15.29|49.37|22.13|45.35|28.17\n|MNCRI (w/o IR)    |56.87 |16.65|54.39|24.71|49.74|30.85|\n|MNCRI (w/o L_MMD)   |55.75|16.04|53.47|22.96 |47.98 |29.92|\n\n# About implementation details of nearest neighbor instance selection \nAs shown in Figure.1, we first use a feature extractor (e.g., CodeT5) to extract semantic features of samples and the feature dimension is 768. In other words, the output of the last layer of the feature extractor serves as the semantic features of samples. Next, we employ the Cosine distance as the distance metric to measure the distance between sample semantic features. For a sample $x_i^t$ in the target domain, we retrieve its $k$ nearest neighbors $H_i^s$ in the source domain. Similarly, for a sample $x_j^s$ in the source domain, we retrieve its $k$ nearest neighbors $H_j^t$ in the target domain. Subsequently, the affinity relationship matrix $M^{st}$ is built by following the rule: $M_{i,j}^{st}$= 1 if only $x_i^t$ and $s_j^s$ are the mutual nearest neighbor pair; otherwise $M_{i,j}^{st}$= 0. Finally, we calculate the mutual nearest neighbor contrastive loss based on the built affinity relationship matrix  (refer to Equation.2 ) and optimizing model weights. We repeat the above steps in each training iteration step until the model converges. Thus, in our work, the affinity relationship matrix is updated (changed) in each training iteration step.  We will provide the missing implementation details in the revised paper.\n\n# About implementation details of the memory bank\nIn our experiments, the memory bank stores the previous five batches of samples’ features. The number of stored samples’ features is $5 \\times 36 \\times 2 = 360$ (the batch size is 36, including source domain samples’ features and target domain samples’ features). The memory bank is updated using a queue mechanism. \n\n# About the assumption that similar instances have common semantic characteristics\nIn our work, we argue the instances with common semantic characteristics have \nsimilar semantic features rather than these instances themselves being similar. As you mentioned, the same types of vulnerabilities may have different manifestations in different projects. But they still have common vulnerable patterns (common semantic characteristics). For instance, CWE-119 (Buffer Errors) vulnerability functions have different manifestations in different projects due to program style and application scenarios. But their vulnerable patterns are the same and consistent. That is, all vulnerability functions trigger data buffer overflow errors. The goal of the detection model is to extract common vulnerable patterns (common semantic characteristics) from different manifestations. Therefore, the semantic features of instances with common semantic characteristics are similar in the feature space.\n\nThanks for your insightful suggestions. We will polish the paper based on these valuable points in the revised version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. This paper addresses a significant issue in the field of code vulnerability detection, where models trained on source domains often fail to generalize to target domains. This is a relevant and practical problem of interest to the NLP community. \n2. The proposed framework, MNCRI, introduces a novel approach to cross-domain code vulnerability detection by combining mutual nearest neighbor contrastive learning and instance re-weighting, showing promising results in improving domain alignment and performance. \n3. The paper provides extensive experimental evaluations that demonstrate the effectiveness of the proposed framework, with results showing significant performance improvements over baseline methods and highlighting the contributions made by this work.", "reasons_to_reject": "1. The paper lacks detailed descriptions of the implementation details of nearest neighbor instance selection, such as the feature used for calculation and the distance metric employed. Additionally, it is unclear whether the affinity relationship matrix is initialized only once during the training process or changes as training progresses. \n2. The paper also lacks clarity regarding the details of the memory bank, including how many samples’ features are stored. \n3. The assumption that similar instances have common semantic characteristics while dissimilar instances have private semantic characteristics may be overly idealistic, as dissimilar instances may also share common semantic characteristics, such as different manifestations of the same vulnerability across different software projects.", "questions_to_authors": "1. The details of the datasets provided in the appendix appear to have a balanced distribution of positive and negative examples. However, in real-world applications, scenarios with fewer vulnerabilities may introduce additional difficulties. \n2. If the code has many different types of vulnerabilities with significant characteristic differences, considering all diverse vulnerability features as a single category (i.e., “vulnerable”) in a binary classification model may be problematic. \n3. Can the authors provide an ablation experiment that eliminates the use of Maximum Mean Discrepancy (MMD) loss for statistical domain alignment to understand whether statistical domain alignment without the Instance Re-weighting Scheme has a negative or small positive effect on experimental results?", "paper_topic_and_main_contributions": "This paper addresses the issue of cross-domain code vulnerability detection, where a detector trained on a source domain fails to provide accurate predictions on unseen target domains. The main contributions of this paper include:\n1) The proposal of a novel cross-domain code vulnerability detection framework, MNCRI.\n2) The introduction of a mutual nearest neighbor contrastive learning paradigm to align the source and target domains geometrically and separate the private semantic characteristics of each domain.\n3) The incorporation of an instance re-weighting scheme to mitigate excessive alignment and improve domain alignment in statistical domain.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "No", "Typos_Grammar_Style_and_Presentation_Improvements": "No"}, "extra_text": {}}, {"review_id": "cfgVAnptlP", "reviewer": "Reviewer_HWdy", "summary": "", "questions": "1. Please clarify the novelty of the proposed method, especially the contrastive learning module and the instance re-weighting module.\n2. Please discuss the sensitivity of the hyperparameters.", "limitations": "", "rating": 3, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "# The novelty of the proposed method\nIn this paper, we propose a cross-domain code vulnerability detection framework, which jointly achieves geometric domain alignment and statistical domain alignment by introducing \nthe contrastive learning paradigm and the instance re-weighting scheme. \n\n**The novelty of the contrastive learning module.**  In cross-domain code vulnerability detection tasks, different domains may have their own private semantic characteristics. Previous studies forcibly align the source domain and target domain and do not consider the negative effect of private semantic characteristics for domain alignment, which easily causes the problem of negative transfer. To address this problem, we propose a mutual nearest neighbor contrastive learning paradigm to achieve geometric domain alignment. Specifically, we first propose a novel geometric domain alignment principle: semantically consistent instances should be geometrically adjacent to each other, whether within or across domains. Based on this principle, we propose the mutual nearest neighbor contrastive learning paradigm, which pulls instances with the consistent semantic across the source and target domains closer to each other and pushes away those semantically inconsistent instances. In this way, it achieves the alignment of common semantic characteristics across domains while the private semantic characteristics of each domain are separated, which solves the problem of negative transfer. Compared to previous methods, our proposed mutual nearest neighbor contrastive learning paradigm achieves geometric domain alignment by exploring the geometric structure relationship of source and target domains in feature space. \n\n**The novelty of the instance re-weighting module.**  The re-weighting scheme is a general method that is widely used in various deep learning application scenarios, such as attention mechanisms equipped in Transformer and instance re-weighting methods for imbalanced classification. Different domains adopt different instance re-weight schemes to address their own domain issues according different characteristics of domains.  These proposed instance re-weighting schemes bring performance improvements and have been proven to be novel. In our work, we mainly focus on improving cross-domain code vulnerability detection performance. In cross-domain code vulnerability detection tasks, some target instances that are not relevant to source instances. Previous studies reduce the domain discrepancy by forcibly minimizing statistical distribution divergence (MMD) of the entire source domain and the target domain, which easily leads to the problem of excessive alignment. In our work, we argue that assigning large weights to relevant instances and small weights to irrelevant instances could alleviate the problem of excessive alignment and improve cross-domain vulnerability detection performance. Compared to previous instance re-weighting research works that utilize solvers or complex optimization methods to obtain instance weights, our proposed instance re-weighting scheme is simple yet effective. Specifically, we adopt the prediction confidence of the classifier trained on the source domain for target instances as the weights of target instances during statistical domain alignment. Introducing the instance re-weighting scheme mitigates the adverse influence of some target instances that are not relevant to source instances, which is helpful for alleviating the problem of excessive alignment. Experimental results prove that the effectiveness of the proposed instance re-weighting scheme.\n\n\n\n\n# The sensitivity of the hyperparameters\nAs shown in Figure 3(b), our method achieves the stable detection performance when $\\mu \\in \n[0.1,0.3]$. When $\\mu$ is set to 0.01, the performance drops. But it cannot conclude that the F1 score is sensitive to minor changes of hyperparameters, as 0.01 is ten times smaller than 0.1. The performance degradation caused by differences in order of magnitude cannot be avoided in many machine learning application scenarios. In our work, the total objective function is formulated as:\n\n$L = L_{CE} + L_{Con} + L_{MMD-IR}$ \n\nwhere hyperparameter $\\mu$ represents the weights of the instance re-weighting module in the overall objective function optimization. If $\\mu$ is set to a relatively small value (e.g., $\\mu=0.01$), it indicates that the instance re-weighting module ($L_{MMD-IR}$) will play a relatively small role on the overall objective optimization.  For instance, if the detection performance does not significantly decrease when $\\mu$ is set to a relatively small value (e.g., $\\mu=0.01$), it can be concluded that the instance re-weighting module is useless and performance improvements come from other modules rather than the instance re-weighting module. Because the instance re-weighting module is close to being banned. Therefore, as illustrated in Figure3(b), when $\\mu=0.01$, the performance degradation is due to the instance re-weighting module being disabled. On the contrary, when $\\mu$ is set to a large value (e.g., $\\mu \\in [0.1, 0.3]$), our method achieves stable detection performance, showing that our proposed method is robust to the selection of $\\mu$.  \n\n\n# About potential applications\nWe propose a novel cross-domain code vulnerability detection framework named MNCRI, which aims to align the common semantic characteristics between domains and separate out private semantic characteristics of each domain. Certainly, our method is not only applicable to cross-domain code vulnerability detection tasks. It can be applied to some knowledge-transfer application scenarios, such as cross-lingual sentiment classification. Some languages derive from the same language family, and they have some common ways of expressing emotions. Simultaneously, due to differences in regions, these languages also have their own unique ways of expressing emotions. Our method MNCRI can be employed to transfer common emotional expression patterns between different languages while separating their respective unique emotional expression patterns, so as to improve the performance of cross-lingual sentiment classification. Furthermore, our method can be applied to some cross-domain news classification tasks where different types of news share common semantic characteristics while also having their own private semantic characteristics. \n\nIn this paper, we mainly focus on improving the performance of cross-domain code vulnerability detection.  Potential applications are not discussed in our original paper. Thanks for your insightful suggestions. We will depict the above potential applications of MNCRI in the revised paper.\n\n\n\nThanks for your insightful suggestions. We will polish the paper based on these valuable points in the revised version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. Overall, this paper is clearly organized and well written. The studied problem of cross-domain code vulnerability detection is very interesting. \n2. The authors revealed limitations in existing work, and further motivated the MNCRI approach. \n3. The proposed method can help address the negative transfer problem, which is a long-standing problem in transfer learning. \n4. Experimental results show that the proposed method performs much better than baselines. Also, ablation studies verified the contributions of each component of the proposed model.", "reasons_to_reject": "1. The novelty of proposed method is not significant. In particular, the proposed model combines contrastive learning and MMD. The instance re-weighting scheme is only a minor modification of the well known MMD approach.\n2. The proposed model has two hyperparameters, \\lambda and \\mu. Figure 3(b) shows that the F1 score is quite sensitive to minor changes of hyperparameters, considering the small range of [0.01, 0.3].\n3. It seems the proposed MNCRI is a general approach, and the authors may discuss its potential applications in other domains.", "questions_to_authors": "1. Please clarify the novelty of the proposed method, especially the contrastive learning module and the instance re-weighting module.\n2. Please discuss the sensitivity of the hyperparameters.", "paper_topic_and_main_contributions": "This paper studies a very interesting problem, i.e., cross-domain code vulnerability detection. The authors argued that prior work ignores the negative effect of private semantic characteristics of the target domain for alignment, and proposed a new approach based on mutual nearest neighbor contrastive learning. Furthermore, the authors introduced an instance re-weighting scheme to enhance the alignment between two domains. Experiments on three open-source software project datasets demonstrate the effectiveness of the proposed method.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "N/A", "Typos_Grammar_Style_and_Presentation_Improvements": "N/A", "Justification_for_Ethical_Concerns": "N/A"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The reviewers were enthusiastic about the paper and agreed that the paper has significant technical merits that were well-justified and supported well by the experiments in the paper. The rebuttal process helped clarify the technical contributions of the work. While the new results will need to be included before the paper is published, there is strong interest in this line of work and the excitement on the proposed approaches is high.
Decision: Accept-Main