# An Improved Reference Paper Collection System Using Web Scraping with Three Enhancements
The second enhancement is the implementation of the program for multi-threading to run the multiple paper searches simultaneously on a multi-core CPU [11; 12]. It can reduce the response time drastically. The third enhancement is the implementations of _Dynamic URL Redirection_, _Regex-based URL Parsing_, and _HTML Scraping with URL Extraction_. The extracting sentence embedding and their semantic information using _Sentence Transformers models_ (_SBERT_), along with text similarity score using _Cosine Similarity_ is adopted. This approach can offer fast checking of PDF file accessibility and improve accuracy relevance. To increase availability and usability, this system is implemented as a web application system. On the client-side, _Streamlit_ is adopted to rapidly build interactive web apps, and display the user interface [13]. On the server-side, the main program has been implemented using _Python_ and supporting libraries. By utilizing _endpoints_ from external web APIs [14], it retrieves article metadata in a JSON format using the _requests_ library [15] and in an _XML_ format using _ElementTree_ library [16; 17]. The program employs the _concurrent_.____futures_ library to run multiple search site accesses on multi-threading and asynchronous execution to enhance processing speed by handling multiple requests concurrently [18]. The program employs _BeautifulSoup_ and _Regular Expression_ libraries [19] to parse HTML contents from article URLs and check accessible PDF files. Furthermore, the program employs _PyPDF2_ for extracting keyword from PDF files [20], and _sentence-transformers_ library for extracting semantic representation of a sentence and check their similarity [21]. For evaluations of the proposal, we implemented the prototype environment, and selected five reference paper queries covering different topics. Then, we compared the number of available relevant references with full-text PDF access by the proposal with our previous system [22], and examined the impact of multi-threading on the response time [23]. The proposal increases the number of relevant references with full-text PDF access by 64.38% on average, and reduces the response time by 59.78% on average. After them, we compared the number of references by the proposal with common literature search tools such as _Publish or Perish_[24], _OpenAIRE_[25], _OpenAlex_[26], and _Google Scholar_[27]. The results show that the proposal outperforms them in providing the number of relevant references and full-text PDF access. Finally, we discussed the proper external web APIs source for reference papers based on the perspective of the proposed system and the limitations. The paper is structured as follows: Section 2 introduces related works in the literature. Section 3 presents the adopted tools. Section 4 presents three enhancements of the _reference paper collection system_. Section 5 presents multiple web API integration. Section 6 presents multi-thread implementation. Section 7 presents fast checking of PDF file accessibility. Section 8 presents keyword matching and text similarity. Section 9 evaluates the performance of the proposal. Section 10 concludes this paper with brief future works.