## 3 The Proposed System
### Sentence Classification
#### 3.2.1 Target Sentence Labeling
The system employs cosine similarity based on Sentence-BERT embeddings to classify each target sentence as similar, different, or very different, with respect to sentences in the reference article. We implemented a sequence of steps to extract the most relevant sentences (i.e., the different and very different ones) from the target articles. Then, we employed a Transformer-based abstractive summarization model to summarize the sentences in an easy-to-read format. We conducted experiments using crowdsourced human judgments as the ground truth to assess the classifier's effectiveness and the summary's ability to represent information in the target articles that differs from the reference article. We obtained an average F1 score of 0.772 against average F1 scores of 0.797 and 0.676 achieved by, respectively, a fine-tuned BERT classifier and a prompt-tuned BERT classifier. By computing the Wilcoxon signed-rank test, our classifier was statistically equivalent to the first BERT classifier and statistically outperformed the second one. This is a remarkable result considering that our classifier requires less computational effort. Regarding the summaries--based on the human judgments collected through crowdsourcing, the summaries have been deemed to accurately represent the information contained in the selected sentences of the target articles

We strongly believe that the large availability of data prevents today's users from finding the information necessary to create their own opinions. The proposed system can be considered a first step toward the end goal of empowering users to discover novel and relevant information, and will be further improved in future work. In particular, we intend to investigate other similarity metrics and classification algorithms to better address the problem of semantic similarity between sentences. Furthermore, we plan to improve the quality of summaries, especially concerning their novelty, by leveraging different methods. Finally, our ultimate goal is to build a platform that allows users to submit a reference article and receive a summary with information different from the reference article. The following supporting information can be downloaded at: [[https://www.mdpi.com/article/10.3390/a16110513/s1](https://www.mdpi.com/article/10.3390/a16110513/s1)]([https://www.mdpi.com/article/10.3390/a16110513/s1](https://www.mdpi.com/article/10.3390/a16110513/s1)). Figure 14: Interface of the application. **Author Contributions:** Conceptualization, P.D., A.B., and F.M. ; methodology, P.D., A.B., and F.M. ; software development, P.D. ; validation, P.D. ; data preparation, P.D. ; writing--original draft preparation, P.D. ; writing--review and editing, A.B. and F.M. ; visualization, P.D. ; supervision, F.M. ; project administration, F.M. ; funding acquisition, F.M. All authors have read and agreed to the published version of the manuscript. **Funding:** This work has been partially funded by the PNRR-M4C2-Investimento 1.3, Partenariato Esteso PE0000013--\"FAIR-Future Artificial Intelligence Research\"--Spoke 1 \"Human-centered AI\" as part of the NextGenerationEU program, and the Italian Ministry of University and Research (MUR) under the framework of the PRIN 2022 JLB83Z \"Psychologically-tailored approaches to Debunk Fake News detected automatically by an innovative artificial intelligence approach\", the FoReLab and CrossLab projects (Departments of Excellence). **Data Availability Statement:** The data used in this study are available on GitHub at [[https://github.com/pietrodelloglio/dataset-for-system-acquiring-content-differences](https://github.com/pietrodelloglio/dataset-for-system-acquiring-content-differences)]([https://github.com/pietrodelloglio/dataset-for-system-acquiring-content-differences](https://github.com/pietrodelloglio/dataset-for-system-acquiring-content-differences)) (accessed on 3 November 2023). **Conflicts of Interest:** The authors declare no conflict of interest.