# An Approach to Trustworthy Article Ranking by NLP and Multi-Layered Analysis and Optimization
## 2 Related Work
### Ranking Systems in Academic Article Repositories  
Major article repositories like Google Scholar and Web of Science use various ranking mechanisms to organize search results. Google Scholar combines citations and relevance to match a user's query, where citations serve as a proxy for an article's impact, and relevance ensures the search results pertain to the user's needs [20]. However, Google Scholar operates as a \"black box\" with a proprietary ranking algorithm that is not disclosed to the public, making it difficult for users to understand the criteria influencing the ranking of search results. In comparison, Web of Science offers single criteria ranking systems such as relevance, citation count, and date posted [21]. While these methods effectively highlight certain aspects of an article, they often overlook other factors contributing to the overall trustworthiness. For instance, a highly cited article may be influential but not necessarily the most current or accurate. Other repositories follow similar approaches. PubMed,for example, primarily ranks articles by relevance and date [22], which can sometimes prioritize newer articles over those that are more impactful or thoroughly reviewed.  
While these single-criteria and opaque ranking methods have their merits, they fall short in providing a comprehensive measure of an article's trustworthiness. Users are often left without clear insights into why certain articles are ranked higher than others, which can lead to confusion and potential misjudgment of the quality of the information. To address these limitations, there is a growing need for a transparent, multi-criteria ranking system that integrates various trust indicators. Such a system would consider not only citations and relevance but also factors like the authors' credentials, the reputation of the publication source, the thoroughness of the peer review process, and the article's content quality and recency. We integrated all those considerations into the design of our system, which makes the following contributions:  
* Integrated Semantic Relevance and Trust Assessment: Combines BERT-based semantic matching with trustworthiness evaluation to retrieve articles that are both topically relevant and scientifically reliable.
* Robust Problematic Article Filtering: Applies a Random Forest classifier trained on retracted and non-retracted articles to effectively eliminate unreliable publications with 90% overall accuracy.
* Transparent Multi-factor Ranking Strategy: Introduces a scoring model that merges citation, Altmetric, and impact factor data to produce an interpretable and adjustable trustworthiness ranking.
* Validated on Large Cross-domain Dataset: Demonstrates a consistent performance using over 16,000 articles across diverse scientific fields, supporting generalizability and applicability to real-world literature discovery tasks.  
The proposed three-layer system will be explained in detail in Section 4. Before illustrating the workflow of that system, we will first introduce the data resources used in this work.