# An Improved Reference Paper Collection System Using Web Scraping with Three Enhancements  
[PERSON]  
1 Department of Information and Communication Systems, Okayama University, Okayama 700-8530, Japan; [EMAIL_ADDRESS] (T.M.F. ); [EMAIL_ADDRESS] (K.C.B. ); [EMAIL_ADDRESS] (I.N. ); [EMAIL_ADDRESS] (S.T.A.) [PERSON]  
1 Department of Information and Communication Systems, Okayama University, Okayama 700-8530, Japan; [EMAIL_ADDRESS] (T.M.F. ); [EMAIL_ADDRESS] (K.C.B. ); [EMAIL_ADDRESS] (I.N. ); [EMAIL_ADDRESS] (S.T.A.) [PERSON]  
1 Department of Information and Communication Systems, Okayama University, Okayama 700-8530, Japan; [EMAIL_ADDRESS] (T.M.F. ); [EMAIL_ADDRESS] (K.C.B. ); [EMAIL_ADDRESS] (I.N. ); [EMAIL_ADDRESS] (S.T.A.) [PERSON]  
1 Department of Information and Communication Systems, Okayama University, Okayama 700-8530, Japan; [EMAIL_ADDRESS] (T.M.F. ); [EMAIL_ADDRESS] (K.C.B. ); [EMAIL_ADDRESS] (I.N. ); [EMAIL_ADDRESS] (S.T.A.) [PERSON]  
1 Department of Information and Communication Systems, Okayama University, Okayama 700-8530, Japan; [EMAIL_ADDRESS] (T.M.F. ); [EMAIL_ADDRESS] (K.C.B. ); [EMAIL_ADDRESS] (I.N. ); [EMAIL_ADDRESS] (S.T.A.) [PERSON]  
3 Department of Data Science, Universitas Pembangun National Veteran Jawa Timur, Surabaya 60294, Indonesia; [EMAIL_ADDRESS] (A.M.); [EMAIL_ADDRESS] (D.A.P.) [PERSON]  
3 Department of Data Science, Universitas Pembangun National Veteran Jawa Timur, Surabaya 60294, Indonesia; [EMAIL_ADDRESS] (A.M.); [EMAIL_ADDRESS] (D.A.P.) Correspondence: [EMAIL_ADDRESS] (T.M.F. ); [EMAIL_ADDRESS] (K.C.B. ); [EMAIL_ADDRESS] (I.N. ); [EMAIL_ADDRESS] (I.N. ); soethandra@s.  
the other hand, _open-access_ publications offer ethical solutions for researchers to access needed articles with low costs. Platforms like _Google Scholar_, _Publish or Perish, OpenAlex_, and _OpenAIRE_ have been widely recognized as search tools that facilitate access to academic articles [6]. Previously, we have proposed a _reference paper collection system_ using a _web scraping technology_ and AI models for natural languages. This system collects the site information for reference papers through _Google Scholar_ by running a script program to control the search process with _Selenium_. _Selenium_ is a popular open source web scraping software. The system provides PDF files of related papers by downloading them from the collected websites and analysing the similarity of their contents to the user request using open source natural language models [7]. However, this system can often find a limited number of relevant reference papers, even after taking long time, since it relies on a single paper search site and runs on a single thread at a multi-core CPU. The system retrieves around 25 to 50 papers per request, with an average relevance accuracy of 53.29% and a response time ranging from 41 to 507 s. The reliance on a single search site may limit the diversity of references obtained and can trigger suspicion because repetitive requests are sent without a time delay. Furthermore, it can increase the risk of violating the terms of service and prohibiting automated scraping, which can lead to _Internet protocol blocking_, and make the site inaccessible and the system unusable. To overcome these limitations, in this paper, we present an improved _reference paper collection system_ with three enhancements. For its efficient implementation and program management, we newly designed and implemented the whole system from scratch using proper frameworks and libraries. The first enhancement is the integration of the APIs from three paper search sites. They are the _bulk search endpoint_ in the _Semantic Scholar_ API, the _article search endpoint_ in the _DOAJ_ API, and the _search and fetch endpoint_ in the _PubMed_ API to retrieve article metadata. _Semantic Scholar_ covers computer science, geoscience, neuroscience, and others [8]. _DOAJ_ covers a broader scope [9]. _PubMed_ focuses on biotechnology, medical, and health domains [10].