# Improving VulRepair's Perfect Prediction by Leveraging the LION Optimizer
## 5 Discussion  
Using current libraries, a recently developed optimizer (LION) over AdamW, and hyperparameter adjustment, we created ImpVulRepair. It not only relies on updated libraries and software such as Python version 3.10.12 instead of version 3.9.7 but also uses Google Colab with a V100 rather than an NVIDIA 3090 as the graphics processing unit. Our ImpVulRepair can repair vulnerable functions with patches, achieving a perfect prediction of 56%, whereas VulRepair, developed in 2022, repairs vulnerable C/C++ functions with a PP of 44%. ImpVulRepair includes the Google Chrome browser, Google Colab, Google Drive, the LION optimizer created by Google Brain, and the T5 architecture introduced by Google in 2019. There are many browsers we could have selected, but we leaned toward Google Chrome since we have used the Google Cloud Platform in the past, which recommends using Google Chrome. We used Google Colab as it provides access to GPUs, does not require software to install, and runs in a browser. In addition, Google Colab integrates with Google Drive, where we can easily save our work. We selected the LION optimizer after conducting research on optimizers and finding one that was recent as of 2023 and that works well with T5 models. Coincidentally, it was created by Google Brain. Our intention was not to show that Google products are in tune with each other but rather to try to resolve the issues of GPU and attached file disconnections. However, at the end of our testing, Google Colab started suggesting new GPUs that may perform better. They also now offer Tensor Processing Units (TPUs) version 2, developed by Google, which are suitable for neural networks handling large datasets and may outperform GPUs in some situations [28]. As of April 2024, the Colab TPU version 1 and the V100 GPU we used are being deprecated. ImpVulRepair could attain consistent results in areas where previous tests failed. Exploring and evaluating newly available Colab GPUs and TPUs might achieve improvements in patching vulnerabilities. With regard to best practices, datasets should be prepared for training machine learning models. This includes cleaning the data by removing duplicates, that is, deduplication of data [41]. If duplicates are not removed, the model's efficiency may be inflated [42]. The datasets used for training and testing VulRepair contained duplicates. Although it may not be ideal, this should be documented and, at best, justified. We acknowledged it but did not justify it. According to [43], 60% of the pairs are duplicates. If the datasets are deduplicated, the performance decreases from 44.2% to 10.2%. Another study found that 40% of the data overlapped between the training set and the test set [44]. Their Mistral model used 7 billion parameters and 5 epochs, whereas VulRepair uses 220 million parameters and 75 epochs. Both use the AdamW optimizer. If overlap is used, a 57% performance prediction with a beam size of 5 could be achieved, but this PP reduces to 26% with deduplication. These studies indicate that our results would be lower with the deduplication of data. Regarding performance, in one study, [PERSON] et al. claimed that their TSBPORT, or type-sensitive patch BackPORTing, improved the success rate of VulRepair by 63.9% [45]. Learning models need sufficient data to train. An external threat to VulRepair is that it may not generalize well to other CWEs and datasets. Other datasets could be explored [7]. In one study, [PERSON] et al. asserted that the lack of datasets for vulnerable programs prevents them from progressing. They used a new technique called VGX to generate \"high-quality\" vulnerability datasets that can improve VulRepair by 85% [46]. Because our ImpVulRepair is an extension of VulRepair, we note that these datasets could also be used to improve model training. Like [PERSON] et al., we did not conduct an exhaustive search to refine all hyperparameters, nor did we attempt to increase memory by adding extra equipment.