Generalization-Enhanced Code Vulnerability Detection via Multi-Task Instruction Fine-Tuning (2024)

Abstract: Code Pre-trained Models (CodePTMs) based vulnerability detection have achieved promising results over recent years. However, these models struggle to generalize as they typically learn superficial mapping from source code to labels instead of understanding the root causes of code vulnerabilities, resulting in poor performance in real-world scenarios beyond the training instances. To tackle this challenge, we introduce VulLLM, a novel framework that integrates multi-task learning with Large Language Models (LLMs) to effectively mine deep-seated vulnerability features. Specifically, we construct two auxiliary tasks beyond the vulnerability detection task. First, we utilize the vulnerability patches to construct a vulnerability localization task. Second, based on the vulnerability features extracted from patches, we leverage GPT-4 to construct a vulnerability interpretation task. VulLLM innovatively augments vulnerability classification by leveraging generative LLMs to understand complex vulnerability patterns, thus compelling the model to capture the root causes of vulnerabilities rather than overfitting to spurious features of a single task. The experiments conducted on six large datasets demonstrate that VulLLM surpasses seven state-of-the-art models in terms of effectiveness, generalization, and robustness.

## conclusion in this paper, we introduce vulllm, a novel framework for code vulnerability detection utilizing llms. by innovatively integrating a vulnerability interpretation task into our multi-task learning framework alongside data augmentation strategies, we significantly enhance the llm's capability to detect code vulnerabilities. this combination not only improves detection accuracy but also enriches the model's understanding of the context and rationale behind vulnerabilities. extensive evaluations conducted on six diverse and comprehensive datasets demonstrate that vulllm surpasses existing approaches in terms of effectiveness, generalization, and robustness. further validation through ablation study confirms the critical role of multitask learning and data augmentation in boosting vulllm's performance.