Paper: Just-in-Time Security Patch Detection - LLM At the Rescue for Data Augmentation (ICLR-2024)

Reviewer comment (Reproducibility):
The model configuration of the proposed method and baselines used in the experiments are not mentioned in the paper. Without these, it is hard to justify the performance of the used models.

Author response:
**A2:** Thanks for your kind reminding, we address your concern by listing up configuration of our model and GraphSPD

We conducted our experiments on 2 pieces of Tesla V100 DGXS 32GB GPUs, ubuntu Ubuntu 20.04.6 LTS, Python 3.7.6, CUDA Toolkit 11.2 and cuDNN v8.1.

Configurations in Experimental Details

For our work: CodeT5+ 6B version; LLaMa 7b version; Batch size: 32; Dimension size: 256; Optimizer: Adam/AdamW; Drop rate: 0.5; Learning rate: 1e-4.

GraphSPD: Dimension of node embeddings after the 1st, 2nd, and 3th convolution is 50, 25, and 12, respectively; Drop rate: 0.5; Learning rate: 0.01

Reviewer comment (Experiments):
Some results seem suspicious, e.g., in the ablation study, removing the patch altogether from the model's input barely causes any performance drop. A deeper ablation study might be needed.

Author response:
**A5**: In response to the minimal performance impact noted in the ablation study when removing the patch, we did additional experiments to evaluate impact of individual components on the model's performance. The result show that performance of different unique part.

**Generated Data Comparison of Various LLMDA Methods With and Without PBCL on PatchDB and SPI-DB**

* Generated Data Comparison of Various Methods With and Without PBCL on PatchDB and SPI-DB

| Method      | Dataset | AUC   | F1    | +Recall | -Recall | TPR   | Dataset | AUC   | F1    | +Recall | -Recall | TPR   |
|-------------|---------|-------|-------|---------|---------|-------|---------|-------|-------|---------|---------|-------|
| LLMDA   | PatchDB | 84.49 | 78.19 | 80.22   | 87.33   | 80.12 | PatchDB | 82.93 | 76.45 | 78.72   | 85.81   | 78.60 |
|             | SPI-DB  | 68.98 | 58.13 | 70.94   | 80.62   | 73.95 | SPI-DB  | 67.43 | 56.61 | 69.45   | 79.10   | 72.91 |
| LLMDA   | PatchDB | 82.35 | 76.31 | 77.24   | 86.13   | 78.70 | PatchDB | 80.25 | 74.73 | 73.12   | 82.91   | 73.50 |
| _{DP}_      | SPI-DB  | 67.66 | 55.82 | 69.43   | 78.69   | 72.46 | SPI-DB  | 65.37 | 52.10 | 64.37   | 74.55   | 69.89 |
| LLMDA   | PatchDB | 79.21 | 73.73 | 73.61   | 83.38   | 75.77 | PatchDB | 75.07 | 71.37 | 71.54   | 80.77   | 73.24 |
| _{EX}_      | SPI-DB  | 65.25 | 52.72 | 67.22   | 75.55   | 70.25 | SPI-DB  | 62.81 | 50.45 | 63.44   | 70.34   | 65.93 |
| LLMDA   | PatchDB | 77.90 | 72.32 | 72.55   | 81.69   | 74.20 | PatchDB | 75.86 | 69.57 | 69.67   | 78.59   | 71.35 |
| _{PT}_      | SPI-DB  | 63.98 | 51.20 | 66.12   | 73.98   | 68.32 | SPI-DB  | 61.27 | 49.34 | 63.44   | 70.00   | 65.29 |

Reviewer comment (Writing):
There are many head-scratchers in this paper after Section 3 in terms of writing. For example, weird phrases like "Dominance Demonstrated" clearly indicate that something is off. The method PBCL is referred to as SBCL, which I'm assuming is because the word Probabilistic and Stochastic are synonyms. Moreover, the name of the technique (LLMDA) is written as "Low-Level Malware Detection Algorithm" in the appendix, and I can't see how this is a proper name for this method. I would like to hear the author's justifications for this situation. Unfortunately, without a significant rewrite, this paper is not up to the standards we would expect from ICLR.

Author response:
**A1:** Thanks to the reviewer for pointing out the weird phrase like "Dominance Demonstrated". What we want to express here is that our model outperforms the state-of-the-art works and we are sorry to make it confusing.

**A2:** In our work, SBCL is one component of PBCL. As shown in Sec 2.3, PBCL represents a whole pipeline for batch embedding calculation. In other words, SBCL is one part of PBCL.

**A3:** Actually, as shown in the paper title, LLMDA is a short name of "LLM data augmentation" instead of "Low-Level Malware Detection Algorithm". In our appendix, we mention the word "LLMDA" to notice the readers the name of our approach in the appendix.

Reviewer comment (Evaluation):
There is a lack of intuition for some of the design choices (especially PBCL). It provides some performance improvements in the ablation study but it is unclear what it actually achieves. A case study (e.g., analyzing some representations with and without this loss) is recommended to provide a better intuition for it.

Author response:
**A4:** In response to feedback, we expanded our case study on models with and without PBCL (Probabilistic Batch Embedding Calculation) loss to provide clearer insights. This is exemplified by comparing PBCL-enhanced models in scenarios like sgminer.c and krb5/auth_context.c patches, demonstrating their superior performance in detecting security patches through more sensitive and nuanced analysis.

With PBCL

| Patch | Explanation | Description |
| --- | --- | --- |
| ... if (!p->authenticator) return ENOMEM; + ``memset (p->authenticator, 0, sizeof(*p->authenticator))``;**(p=0.75)** p->flags = KRB5_AUTH_CONTEXT_DO_TIME; | Added memset to initialize ```authenticator' memory in krb5_auth_con_init function`` **(p=0.84)**. | ``zero authenticator`` **(p=0.77)** |

Without PBCL

| Patch | Explanation | Description |
| --- | --- | --- |
| ... if (!p->authenticator) return ENOMEM; + ``memset (p->authenticator, 0, sizeof(*p->authenticator));``**(p=0.59)** p->flags is KRB5_AUTH_CONTEXT_DO_TIME; | Added memset to initialize ```authenticator' memory in krb5_auth_con_init function`` **(p=0.68)**. | ``zero authenticator`` **(p=0.65)** |

Reviewer comment (Evaluation):
There is a red flag in the ablation study that removing the patch itself from the input barely hurts the performance. Possible explanations include a bug in evaluation (e.g., testing on the train set, training on the test set), issues with the LLM-provided patch explanations, or the Code-LLM suffering from test set leakage (since these models are trained on everything). With pre-trained LLMs, it is impossible to ensure they have not seen the testing samples or related data before. It is unclear how to confirm or refute this, but currently it is concerning that the patch code itself seems to have little impact on the results. The reviewer requests ideas to address this concern.

Author response:
**A6**: To address concerns of data leakage due to the involvement of Large Language Models (LLMs) in our expanded dataset, we conducted empirical validations including tests on an external dataset not used in LLM training, ensuring no prior exposure. The consistent model performance, improved results with LLM-generated explanations, and stable k-fold cross-validation across data splits indicate that improvements are due to the integration of LLM explanations, not data leakage.

Reviewer comment (Experiments):
The datasets used are fairly small, with PatchDB having 36K samples and SPI-DB only 25K. For deep learning, these sizes are quite modest. Training and evaluating on larger corpora could lend more statistical power.

Author response:
**"Dataset size"**: SPI-DB and PatchDB are now two widely used datasets for security patch detection, including 36k and 25k items inside, respectively. Due to the huge workload by manual way of collecting and labeling the dataset is greatly time-consuming, we have been still building a bigger security patch dataset in our other project. Thanks for the reviewer's suggestion, we will evaluate our method in future datasets.

Reviewer comment (Reproducibility):
Could you provide the exact prompts used to generate the explanations and instructions? This context would help with reproducibility.

Author response:
**"Prompts used to generate the explanations and instructions?"**:  
In our work, we designed several prompts to drive LLM to generate explanations of the given patch. They are as follows:  
"Can you compare these and explain the key differences and implications of these changes?"  
"Can you explain what happened in the following code changes"  
"Can you explain it to me about the good or bad effect of the following changed codes"  

Finally, the prompt "Can you compare these and explain the key differences and implications of these changes?" works much better than other prompts.

Reviewer comment (Experiments):
The ablation study removes one component at a time. How does performance degrade when ablating multiple components together?

Author response:
**More ablation study**: We have conducted more experiments for ablation study. Here, in LLMDA_{DP}, we only use the description as the model's input, the same as  LLMDA_{EX}  and  LLMDA_{PT}

**Table: Generated Data Comparison of Various LLMDA Methods With and Without PBCL on PatchDB and SPI-DB**

| Method      | Dataset | AUC   | F1    | +Recall | -Recall | TPR   | Dataset | AUC   | F1    | +Recall | -Recall | TPR   |
|-------------|---------|-------|-------|---------|---------|-------|---------|-------|-------|---------|---------|-------|
| LLMDA   | PatchDB | 84.49 | 78.19 | 80.22   | 87.33   | 80.12 | PatchDB | 82.93 | 76.45 | 78.72   | 85.81   | 78.60 |
|             | SPI-DB  | 68.98 | 58.13 | 70.94   | 80.62   | 73.95 | SPI-DB  | 67.43 | 56.61 | 69.45   | 79.10   | 72.91 |
| LLMDA   | PatchDB | 82.35 | 76.31 | 77.24   | 86.13   | 78.70 | PatchDB | 80.25 | 74.73 | 73.12   | 82.91   | 73.50 |
| _{DP}_      | SPI-DB  | 67.66 | 55.82 | 69.43   | 78.69   | 72.46 | SPI-DB  | 65.37 | 52.10 | 64.37   | 74.55   | 69.89 |
| LLMDA   | PatchDB | 79.21 | 73.73 | 73.61   | 83.38   | 75.77 | PatchDB | 75.07 | 71.37 | 71.54   | 80.77   | 73.24 |
| _{EX}_      | SPI-DB  | 65.25 | 52.72 | 67.22   | 75.55   | 70.25 | SPI-DB  | 62.81 | 50.45 | 63.44   | 70.34   | 65.93 |
| LLMDA   | PatchDB | 77.90 | 72.32 | 72.55   | 81.69   | 74.20 | PatchDB | 75.86 | 69.57 | 69.67   | 78.59   | 71.35 |
| _{PT}_      | SPI-DB  | 63.98 | 51.20 | 66.12   | 73.98   | 68.32 | SPI-DB  | 61.27 | 49.34 | 63.44   | 70.00   | 65.29 |

This table presents a comprehensive comparison of various configurations of a tool, denoted as LLMDA, under different conditions and datasets. The performance metrics for these configurations are evaluated both with and without the application of PBCL, across two datasets: PatchDB and SPI-DB.

The configurations being compared include LLMDA, LLMDA_{DP},  LLMDA_{EX}, and LLMDA_{PT}. Each of these is assessed using the following metrics: AUC (Area Under Curve), F1 Score, Positive Recall (+Recall), Negative Recall (-Recall), and TPR (True Positive Rate). The results are organized into two sections, representing the scenarios with and without PBCL.

Key observations from the analysis are:
- The standard LLMDA configuration shows the highest performance across all metrics in both datasets, irrespective of PBCL usage.
- LLMDA_{DP} generally follows LLMDA in terms of performance, indicating it is the second most effective configuration.
- There is a progressive decrease in performance in the order of LLMDA_{DP}, LLMDA_{EX}, and LLMDA_{PT}.
- The inclusion of PBCL enhances the performance of each configuration, as evident from the improved metrics across both datasets.
- The impact and effectiveness of PBCL, along with the relative performance of each configuration, remain consistent in both the PatchDB and SPI-DB datasets.  
We find that out of all components, descriptions matter more.

Reviewer comment (Experiments):
Can you apply the case study analysis to a larger and more diverse sample of patches? Any insights on patterns?

Author response:
**"case study"**: We show diverse samples for the case study in our appendix part in 11 categories.