# Collecting Vulnerable Source Code from Open-Source Repositories for Dataset Generation
## 3 Discussion
### Future Enhancements  
The performance of SVCP4C could be improved in several aspects. First, SVCP4C does not parallelize HTTP requests. There are several existing solutions for parallelizing HTTP requests in Python, and implementing one of them is crucial to reduce download time. However, with parallel requests several difficulties may arise, for example, with parallel requests come parallel responses; therefore, persistence becomes a critical operation which shall involve synchronization mechanisms. Moreover, asynchronous requests imply receiving responses in no particular order.  
Regarding the 10,000 query result limit, the restriction itself cannot be eliminated because that is the way SonarCloud's web API is implemented. We could, however, surpass it. To do so, and assuming no project has more than 10,000 issues, the issues must be requested on a per-project basis. As of right now, SVCP4C retrieves all project _ids_ that meet our filtering criteria and requests the issues of all the _ids_ altogether. This change would affect the performance as SVCP4C would go from a single HTTP GET request specifying \(N\) project _ids_ to an HTTP GET request per project _id_ (\(N\) HTTP GET requests).  
Complementing the detection with dictionaries of vulnerable functions could improve the accuracy ratio when tagging a specific code line as vulnerable. This improvement would greatly reduce the number of vulnerabilities unreported by SonarCloud. To illustrate a case where SonarCloud fails to detect the vulnerability, imagine a buffer 16 bytes long. If the programmer uses the scanf function to fill it up, SonarCloud successfully detects the possible buffer overflow by reporting the use of unsafe functions. As a fix, SonarCloud wisely recommends the use of a width specifier for the corresponding placeholder. It is also stated in the Common Weakness Enumeration [60]. However, as soon as the developer places the width specifier, SonarCloud assumes it is a correct one. We consider this assumption both critical and harmful as a self-induced buffer overflow may arise. The programmer could use a width specifier bigger than the actual buffer to which the data will be copied to. Assuming the previous 16-byt buffer, a program could fill it with up to 30 bytes of data if the programmer used the \"%30s\" width specifier for the function scanf. In this case, the difference in size is evident but a more subtle off-by-one buffer overflow could remain undetected. With the help of dictionaries, functions could be specified in order to flag them and always check their parameters.  
As SonarCloud's responses consist of starting and ending line and starting and ending offset (column) of the vulnerability, highlighting the piece of code if the request is made using GUI, future research should consider expanding this information. From TOOBAD4 ML's perspective, it is much more useful to retrieve the start and end positions of full vulnerable function signature instead of simply its invocation keyword (Figure 2) or a single parameter. A function may have a variable number of arguments or spread across multiple lines, among others, which complicates its static analysis. Knowing beforehand its starting and finishing positions eases its parsing and thus its analysis.