## 3 Methodology
### Data Preprocessing  
To ensure a fair and reliable evaluation of LLMs in vulnerability detection, a comprehensive data preprocessing pipeline is applied prior to inference. This pipeline is designed to eliminate potential biases and ensure that models analyze only the intrinsic characteristics of source code, without relying on extraneous hints. The main actions of Data Preprocessing are presented using Algorithm 1. ```
Input:  SARD--Source code dataset with SARIF metadata Output:  CleanedDataset--Clean dataset without biases begin  1. NonEssentialFiles in SARD: io.c, std_testcase.h, and std_testcase_io.h  2. TargetFiles = FilterNonEssentialFiles(SARD)  3. LabeledFiles = FilterNonAnnotatedFiles(TargetFiles)  4. ProcessedFiles = RemoveComments(LabeledFiles)  5. CleanedDataset = AnonymizeFilenames(ProcessedFiles) return CleanedDataset (*.code) end
```  
**Algorithm 1**Pseudocode for Data Preprocessing Stage Algorithm  
Algorithm 1 receives the SARD dataset with SARIF metadata as input. The output algorithm returns the CleanedDataset without biases. Each test case retrieved from SARD includes multiple C/C++ source files with _.c_ or _.cpp_ extensions. However, not all of these files contain meaningful vulnerability information. In particular, many files serve as auxiliary components, such as utility functions or standard testing interfaces, which are reused across multiple test cases. A typical example is the _testcasesupport_ directory, which includes files like _io.c_, _std_testcase.h_, and _std_testcase_io.h_. These files are defined as _NonEssentialFiles_ in Line 1 of Algorithm 1. They are shared across different CWE categories and do not contribute to the unique vulnerability logic of individual test cases. In Line 2, all non-essential files are programmatically filtered out and return _TargetFiles_. Next, only source files that are directly annotated in the SARIF metadata as containing vulnerabilities are retained. This avoids sample contamination and reduces noise in model evaluation. So in Line 3, _LabeledFiles_ is returned by filtering non-annotated files. Then, the issue of code comments is addressed. Comments, while sometimes beneficial, can also introduce biases. They may provide direct cues about the presence of vulnerabilities or even describe the bug itself. Additionally, in practice, high-quality comments are often missing, especially in buggy or legacy code, which further supports the rationale for excluding them to better simulate such real-world conditions. These clues can inadvertently \begin{table}
\begin{tabular}{c c c} \hline \hline
**Total Samples** & **Vulnerable Samples (Ratio)** & **Non-Vulnerable Samples (Ratio)** \\ \hline
1346 & 963 (71.55\%) & 383 (28.45\%) \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Evaluation Sample Distribution. assist LLMs in identifying vulnerabilities, thus skewing performance evaluations. All inline (//) and block (/* */) comments are removed using custom scripts. In addition, unnecessary line breaks and formatting are normalized to present the LLMs with pure source code only, thereby enforcing an equal basis for model comparison. After this step in Line 4, _ProcessedFiles_ is returned after removing unnecessary items. In Line 5, another subtle yet critical preprocessing step involves filename anonymization. To evaluate the intrinsic code comprehension capabilities of each model, any auxiliary signals that might artificially inflate performance are anonymized. In the SARD dataset, filenames often embed semantic information, such as the CWE identifier (e.g., _CWE119_BufferOverflow_bad.cpp_) or the vulnerability state (e.g., _goodB2G_, _bad_, _goodG2B_). Prior studies have shown that such metadata can strongly influence the predictions of LLMs, especially those with prompt-sensitivity or contextual learning capabilities. All filenames are replaced with randomized hash-based identifiers, ensuring that no external semantic hints (such as CWE type or vulnerability state) are leaked into the inference process.