# An Approach to Trustworthy Article Ranking by NLP and Multi-Layered Analysis and Optimization
## 5 Expanded Experiments, Results, and Discussion
### Exemplar Result and Discussion
Third, we saw a gap between how well the model performed on training data versus testing data, suggesting some overfitting. Though this  
Figure 8: An exemplar result following the three-layer architecture of our proposed system. The result shows not only the top recommended articles with additional metadata but also some eliminated articles. It is noteworthy that the eliminated articles shown in this result are all retracted in the real world. did not greatly harm the model's usefulness, it indicates that some patterns the model learned might be specific to our dataset and might not work as well with new data. In Layer 3, our ranking algorithm remains in early experimental stages and requires further validation and refinement with reference data from experts, to determine its effectiveness in real-world applications. This study holds both theoretical and practical significance for the fields of information retrieval, scholarly communication, and research integrity. Theoretically, it contributes to the advancement of trust-aware search systems by demonstrating how semantic similarity, classification models, and composite ranking functions can be joined in a multi-layer structure to address the challenges of relevance and trustworthiness in academic article discovery. Unlike traditional search systems that rely on single-criterion ranking or opaque algorithms, our method introduces a more interpretable and systematic approach grounded in modern NLP and machine learning. It provides a framework that can be extended to evaluate not only articles but also other scholarly objects such as datasets and software packages. Practically, the system addresses a growing need in the research community for tools that help users confidently identify high-quality literature amid rising publication volumes and concerns about misinformation and retractions. The modular design allows for easy adaptation across disciplines and use cases, while the use of open data sources and transparent scoring functions supports reproducibility and user trust. By placing indicators of scientific reliability alongside topical relevance, this system empowers users to make more informed decisions about what research to read, cite, and build upon. Furthermore, researchers, librarians, and research administrators can use the system to streamline literature reviews, examine citations, and support quality assurance processes in their work. Based on these findings, we see several ways to improve our work in the future. A fundamental opportunity is to enhance the feature dataset by collecting and developing additional indicators that capture new dimensions of article quality, such as the h-index indicating the impact and productivity of authors, author network analysis, institutional affiliations, and other publication metadata. We will also address the dataset imbalance more fundamentally by expanding our dataset through collaboration with additional journals and academic institutions, with a particular focus on increasing the representation of non-retracted papers. This approach will naturally improve class balance while maintaining data authenticity and ensuring that the model reflects genuine publication patterns rather than artificially modified distributions. For the final ranking (Layer 3), we plan to obtain expert validation through structured surveys to determine the effectiveness of the ranking system across diverse research domains and publication types. The other way to address this problem is to use Large Language Models [25] to determine the effectiveness of the ranking models. To improve accessibility, we will develop an intuitive web interface where researchers can easily find trustworthy articles and understand the process of why those articles are chosen. For example, the trust ranking scores will be shown alongside their search results. Finally, a list of user studies can be conducted with researchers from various disciplines to gather feedback on the system's usability.