Joint Geometrical and Statistical Domain Adaptation for Cross-domain Code Vulnerability Detection (2023)

Abstract: In code vulnerability detection tasks, a detector trained on a label-rich source domain fails to provide accurate prediction on new or unseen target domains due to the lack of labeled training data on target domains. Previous studies mainly utilize domain adaptation to perform cross-domain vulnerability detection. But they ignore the negative effect of private semantic characteristics of the target domain for domain alignment, which easily causes the problem of negative transfer. In addition, these methods forcibly reduce the distribution discrepancy between domains and do not take into account the interference of irrelevant target instances for distributional domain alignment, which leads to the problem of excessive alignment. To address the above issues, we propose a novel cross-domain code vulnerability detection framework named MNCRI. Specifically, we introduce mutual nearest neighbor contrastive learning to align the source domain and target domain geometrically, which could align the common semantic characteristics of two domains and separate out the private semantic characteristics of each domain. Furthermore, we introduce an instance re-weighting scheme to alleviate the problem of excessive alignment. This scheme dynamically assign different weights to instances, reducing the contribution of irrelevant instances so as to achieve better domain alignment. Finally, extensive experiments demonstrate that MNCRI significantly outperforms state-of-the-art cross-domain code vulnerability detection methods by a large margin.

## introduction with the development of the internet, the number of vulnerabilities in software has increased rapidly, which leaves software to face huge security threats. there are multiple efforts (lin et al., 2017; grieco * corresponding authors. et al., 2016; li et al., 2018; zou et al., 2019; zhou et al., 2019; feng et al., 2020) have attempted to introduce deep learning and nlp techniques for automated vulnerability detection. those deep learning-based detection methods mainly focus on in-domain code vulnerability detection. that is, the training and test datasets are assumed to be drawn from the same distribution (i.e., from the same software project). however, in practice, most software projects lack labeled vulnerability datasets. labeling data is labor-intensive and expensive, which requires that the models trained on a label-rich source domain (i.e., a software project with abundant labeled data) are applied to a new target domain (i.e., a software project without labeled data). owing to program style, application scenarios, and other factors, different software projects may obey different probability distributions. the models trained on the source domain suffer from severe performance loss on the target domain due to the obvious distribution discrepancy between two domains. to solve this problem, some research works have attempted to introduce domain adaptation to perform cross-domain code vulnerability detection. domain adaptation is a technique that aims to eliminate distribution discrepancy and transfer knowledge from a labeled source domain to an unlabeled target domain @xcite . @xcite first proposed a function-level cross-domain code vulnerability detection framework. but they only hypothesize that the learned high-level representations are transferable, and they do not propose a method to explicitly eliminate the domain discrepancy. @xcite proposed a cross-domain detection framework that utilizes two lstms to learn two groups of transferable feature representations, and then combine both groups of feature representations to train a classifier. @xcite proposed a new code domain adaptation framework named scdan that incorporated conventional domain adaptation techniques such as ddan, mmd, and dirt-t @xcite @xcite . cd-vuld @xcite employed a metric transfer learning framework (mtlf) @xcite to learn cross-domain representations by minimizing the distribution divergence between the source domain and the target domain. lastly, @xcite proposed a detection framework called vulgda that utilized a graph embedding method to extract comprehensive vulnerability features and a feature generator to generate domain-invariant feature representations by minimizing maximum mean discrepancy (mmd). although the above-mentioned research works achieve considerable performance improvements, there still exist two problems that restrict further performance improvements: (1) in practice, different domains (i.e., different software projects) share common semantic characteristics and while having their own private semantic characteristics due to different application scenarios, coding style, and code implementation, etc. previous studies reduce the domain discrepancy by forcibly minimizing divergence of the entire source domain and the target domain. they ignore semantic differences between two domains and allow samples with private semantic characteristics to participate in domain alignment, which is prone to the problem of negative transfer. eliminating the domain discrepancy should be constrained on only aligning the common semantic characteristics (also known as the common knowledge) between two domains and the respective private semantic characteristics should be separated simultaneously. (2) there will always exist some target instances that are not relevant to source instances. current methods treat the instances equally when conducting statistical distribution domain alignment. intuitively, the relevant instances should be given larger weights, and irrelevant instances should be assigned smaller weights so as to decrease their importance and contribution to domain alignment. ignoring the relevance of instances will make models learn suboptimal cross-domain representations. to address these issues, we propose a novel cross-domain code vulnerability detection framework named mncri (as illustrated in figure 1 ) based on mutual nearest neighbor contrastive learning and the instance re-weighting scheme, which jointly achieves geometric domain alignment and statistical domain alignment. first, we argue that geometric domain alignment should follow a principle: semantically consistent instances should be geometrically adjacent to each other, whether within or across domains. based on this principle, we introduce mutual nearest neighbor contrastive learning to achieve the alignment of common semantic characteristics. specifically, we construct mutual nearest neighbor pairs across the source domain and the target domain by leveraging the mutual neighbor rule. then we introduce a contrastive loss function that aims at reducing the feature discrepancy of mutual nearest neighbor pairs. by minimizing this contrastive loss, we achieve the alignment of common semantic characteristics across domains and separate out private semantic characteristics of each domain. second, to alleviate the problem of excessive alignment, we introduce the instance re-weighting scheme that assigns different weights to different target domain instances based on the relevance with source domain instances. in this way, the effect of some target domain instances that are irrelevant to source domain instances is reduced and the effect of some relevant target domain instances is highlighted, which is beneficial for reducing the distribution discrepancy. finally, we conduct extensive experiments to verify the effectiveness of our proposed method. the experimental results show that the proposed method outperforms current baseline methods by a large margin. in summary, our contributions are as follows: ## methodology as illustrated in figure 1 , the architecture of mn-cri contains two main components: 1) the mutual nearest neighbor contrastive learning module, which performs contrastive learning based on mutual nearest neighbors for better geometric domain alignment; 2) the statistical distribution domain alignment module equipped with the instance reweighting scheme, which reduces the distribution discrepancy and achieves the domain alignment statistically. below we elaborate the details of mn-cri. ## experiments in this section, we conduct extensive experiments to evaluate the effectiveness of the proposed framework (mncri).