## 3 The Proposed System
### Sentence Classification
#### 3.2.1 Target Sentence Labeling
Second, we compared the proposed system with two BERT classifiers, applying model tuning and prompt tuning respectively, in order to obtain a reliable comparison with state-of-the-art approaches. Section 4.2 provides insights into the experiment for parameter tuning and a comparison with the two BERT-based classifiers. Concerning the evaluation of the summaries, we again performed a crowdsourcing experiment, asking participants to, given a reference article and the target sentences, rate the quality of the summary in terms of completeness, factual correctness, and novelty. We detail this experiment in Section 4.3. Finally, in Section 4.4, we present some examples of how our system works in practice. ### Dataset

We compiled a dataset that includes 43 articles, each relating to one of 8 different news events, with at least 3 articles for each news event. We assume that articles pertaining to the same event and published on the same date (or similar) are comparable. For each news event, we consider one article as the reference article, and the others as target articles. For each article, we record the news it is sharing, the main topic of that news event (e.g., politics, health, etc. ), the newspaper source, the date of publication, and the URL. To simplify the collection process, we exploited CrowdTangle [[https://apps.crowdtangle.com/](https://apps.crowdtangle.com/)]([https://apps.crowdtangle.com/](https://apps.crowdtangle.com/)) (accessed on 3 November 2023). We collected a list of events that occurred between 2020 and 2021 and obtained social media posts from Facebook pages of international newspapers (e.g., CNN, The Guardian, Al Jazeera English) with links to articles about the news. We then collected the news content directly from the newspaper website. In order to obtain ground-truth labels for sentences in the target articles, we performed a crowdsourcing experiment using Prolific [[https://www.prolific.co/](https://www.prolific.co/)]([https://www.prolific.co/](https://www.prolific.co/)) (accessed on 3 November 2023). The annotators were presented with a list of \(\langle\)target sentence--reference sentence\(\rangle\) pairs. Annotators were asked to decide whether the target sentence was similar, different, or very different from the reference sentence. Each pair was labeled by seven different annotators, and the majority vote determined the gold standard label. If the majority vote was uncertain, the pair was discarded. The dataset is available on GitHub [[https://github.com/pietrodelloglio/dataset-for-system-acquiring-content-differences](https://github.com/pietrodelloglio/dataset-for-system-acquiring-content-differences)]([https://github.com/pietrodelloglio/dataset-for-system-acquiring-content-differences](https://github.com/pietrodelloglio/dataset-for-system-acquiring-content-differences)) (accessed on 3 November 2023). ### Target Sentence Classification Evaluation

First, we aimed to identify the optimal threshold parameters for our proposed system. To achieve this, we conducted a series of experiments involving different valuesfor LS and HS, and compared the automatic classifications generated by our system with human labeling. We employed a leave-one-out cross-validation approach, where one news article was held out as a test set, while the remaining articles were used to determine the threshold values that best aligned with human labeling. For each fold, we obtained eight different LS and HS pairs that achieved the best performance according to the highest number of news articles. In case of a tie, we randomly selected one pair. We obtained the same LS and HS pairs for all folds. This suggests that the choice of LS and HS thresholds was not particularly sensitive. Table 6 shows an example of F1 scores obtained in an example fold. The table refers to the following events: \"Mars Landing\" (M.L. ), \"Italy Lockdown\" (I.L. ), \"Italy Euro 2020\" (I.E. ), \"Global Warming\" (G.W. ), \"[PERSON] Death\" (G.F.), \"[PERSON] coming out\" (E.P. ), and \"China Landed on Mars\" (C.L.).