## 3 The Proposed System
### Sentence Classification
#### 3.2.1 Target Sentence Labeling
\begin{table}
\begin{tabular}{p{142.3 pt} p{142.3 pt}} \hline \hline
**Reference Article (NBC News)** & **Summary of Target Articles (USA Today \& Al Jazeera English)** \\ \hline ‘Some of the most prized, centuries-old relics of France and Christiany survived the devastating Notre Dame Cathedral fire that almost wiped out the cherished Paris landmark, authorities said Tuesday. Culture Minister [PERSON] breathed a huge sigh of relief, telling reporters outside Notre Dame that the nation’s “most precious treasures” were largely spared. Saved treasures from the Notre Dame Cathedral are temporarily stored in city hall after a massive fire devastated large parts of the Gothic cathedral in Paris. [PERSON]/ Reuters Many of the works will be stored at Paris’ City Hall and the Louvre, where they will be examined, treated for damage, and protected, officials said. [PERSON], director-general of UNESCO, the U.N. culture agency, said Notre Dame has “a particular place in the world’s collective imagination” and has plededed her agency’s help to rebuild. Monday’s fire almost destroyed the entire cathedral, which has stood in Paris and survived nearly 900 years of tumultuous French history. Paris prosecutor [PERSON] has launched an investigation, which he said would be “long and complex.” \\ \hline \hline \end{tabular} The huge fire that tore through Notre Dame Cathedral in Paris last night has left the famous landmark in ruins, but not as badly as many had feared. The famous spire was reduced to ash, but the building’s stone face and bell towers were saved, the BBC reports. The main altar has been badly damaged, but the famous white marble [PERSON] sculpture by French sculptor [PERSON] is still standing, seemingly unscathed, at its place before the main altar. “The great tragedy is the structure itself and its fittings”, a Brown University art professor tells the Los Angeles Times. “Many people don’t know that the roofs are timber, largely replaced in the 19 th century, and they have been totally consumed. Masonry will burn and degrade under intense heat, which is clearly what is happening, but we can only wait and see what kind of damage has spread to the church within.” A full investigation is underway.’

\end{table} 
Table 5: Reference article and summary extracted from VDS sentences and some of the DS sentences collected from two target articles about the Notre Dame fire in 2019. ## 4 Experimental Results

The effectiveness of the proposed approach was evaluated along two dimensions. First, we assessed the performance of the 'Sentence Classification' module in determining a class of similarity between sentences. In particular, we compared the output of the module with human judgment collected by crowdsourcing and with two BERT classifiers. Second, we evaluated how much the summary produced by our approach represents a coherent, factually correct, and complete summarization of the target sentences different from the reference article. Again, we used human judgment collected by crowdsourcing. For the experiments, we first collected and annotated a dataset of online newspaper articles spanning various topics that serve both as ground truth for performance evaluation and as a training dataset for setting the optimal parameters, especially concerning the thresholds. The dataset simulates a scenario with a reference article and at least two other target articles for a given news item. Target sentences are labeled through crowdsourcing with three classes--SS, DS, and VDS--for their similarity with the sentences in the reference article. We detail the creation of the dataset in Section 4.1. We tuned the model's parameters (i.e., the similarity thresholds) and applied the sentence classification to target articles. We evaluated the performances of the 'Sentence Classification' module in two ways. First, we measured the F1 score with respect to human judgment.