# Improving VulRepair's Perfect Prediction by Leveraging the LION Optimizer
## 1 Introduction
It is used to repair the vulnerability of the function, or, as they say, to suggest repairs to vulnerable lines [7]. Perfect prediction refers to an outcome that is 100% accurate and not a random guess. The perfect prediction percentage provides the percentage of correct outcomes. Regarding VulRepair, it indicates the percentage of cases where a correct repair is matched, or perfectly predicted, to a vulnerable function. In their study, [PERSON] et al. showed how VulRepair, with a 44% perfect prediction in fixing vulnerabilities, outperforms CodeBERT at 31% and VRepair at 23%. They further demonstrated the importance of using pretraining with natural and programming languages and how a subword tokenizer is better than a word-level tokenizer in the VulRepair model. [PERSON] et al. used the Big-Vul dataset, which is a C/C++ code vulnerability dataset, to evaluate vulnerable functions [7]. The dataset plays an important role in vulnerability research and was introduced by [PERSON] et al. [8]. It contains 3754 code vulnerabilities and includes CVE IDs and CVE severity scores. Its use applies to file, function, and line fixes. More information is available on GitHub at [9]. [PERSON] et al. also used the CVEfixes dataset, which is used in software security research for vulnerability prediction, classification, severity prediction, and automated vulnerability repair, as provided by [PERSON] et al. [10]. The initial release of CVEfixes contains 5495 vulnerability-fixing commits. VulRepair was evaluated on both Big-Vul and CVEfixes, which together contain 8482 vulnerability repairs across over 180 different CWE types. From [7], we note several points. First, VulRepair achieves optimal performance when the vulnerable functions contain fewer than 500 tokens and the repair tokens are fewer than 20. The T5 architecture has a token limit of 512, with anything more becoming truncated. Additionally, VulRepair is more effective when vulnerable repairs have fewer than 10 tokens. Second, the provided settings are intended to replicate their experiment, and researchers are encouraged to find methods to deal with larger functions and repair tokens. Third, they discuss threats to their validity, one of which is hyperparameter optimization, which involves finding the optimal hyperparameters for the best results. However, determining the best hyperparameters is costly and time-consuming, as the models require hours for training and testing. Our study asserts that research is continually evolving with ongoing advances. Computer systems, including software, hardware, and algorithms, are constantly improving, which requires continuous testing to evaluate model performance. We address their validity threats and utilize updated libraries to replicate their research. For a comparison of software versions from 2022 and 2024, see Table 1.  
We also use a 2023 optimizer to help improve the accuracy of their results. Furthermore, [PERSON] et al. reported that the goal of their research was not to find the optimal hyperparameter settings but to compare their model to other models. They suggested that the model could be improved by optimizing the hyperparameters [7]. We also note that the MickeyMike tokenizer is not working. We switch to SalesForce. The results of our experiment will address the following research question:  
Can we improve the deep learning model's training, enhance vulnerable function detection, and increase the perfect prediction by adjusting VulRepair's hyperparameters and using its current libraries?