# Similarity Calculation via Passage-Level Event Connection Graph
## 5 Experiments and Analyses
### Experimental Results
#### 5.2.4 Comparison of Different Algorithms  
The following table shows the results of comparing our similarity calculation with the supervised and unsupervised baseline algorithms. The supervised baseline algorithms include textCNN, LSTM, LSTM+Bidirectional attention (abbreviated as LSTM+BIA), and BERT-base. They are conducted only on the large-scale testing corpora including short sentences, since these corpora have enough data to form training set. The unsupervised baseline algorithms include Average (abbreviated as AVE), TextRank+Average (abbreviated as TR+AVE), and TextRank+Concatenation (abbreviated as TR+CON). The details of the baseline algorithms are already told at the beginning of Section 5. For unsupervised similarity calculations, to test their similarity calculating results, we first represent input text as vector. This vector is averaged from all the word vectors through input text, or some chosen word vectors by TextRank algorithm. Then, Cosine similarity is used to decide whether two texts are similar or not. For supervised similarity calculations, we also represent input text as vector, while this vector is formed by different encoder models like LSTM, Bi-LSTM, or pretrained models (BERT or RoBERTa). Then, two \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Methods** & **TextRank** & **LDA** & **Ours** \\ \hline \multicolumn{4}{c}{Long Texts} \\ \hline A1 & ZecOps, Apple, mail, leak, hacker & Apple, mobile, ZecOps, bug, hacker & Apple, mail, software, leak, \({}^{*}\) \\ \hline A2 & [PERSON], soccer, Bayern Munich, & soccer, Bayern Munich, beat, Man & Bayem Munich, beats, Man Utd, \({}^{*}\) \\  & injury, champion & Utd, [PERSON] & Bayern Munich, beats, Man Utd, \({}^{*}\) \\ \hline A3 & Barclay, bank, economic, & Barclay, coronavirus, bank, & coronavirus, pandemic, costs, \\  & coronavirus, profit & pandemic, work & \(\epsilon\)2.1 bn, \({}^{*}\) \\ \hline \multicolumn{4}{c}{Short Texts} \\ \hline S1 & Carmaker, Tesla, build, factory, & Carmaker, Tesla, build, factory, & Tesla, build, factory, Shanghai, \({}^{*}\) \\  & Shanghai & Shanghai & Tesla, build, factory, Shanghai, \({}^{*}\) \\ \hline S2 & [PERSON], death, BBC, TV & BBC, apologize, footage, mistake, \({}^{*}\) & BBC, apologize, footage, mistake, \({}^{*}\) \\ \hline S3 & Coronavirus, economy, sink, & Coronavirus, economy, sink, & Economy, sink, pandemic, \\  & pandemic, shutdown & shutdown, \({}^{*}\) & shutdown, \({}^{*}\) \\ \hline \hline \end{tabular}
\end{table} 
Table 3: The extracted words from the sampled texts (* marks that less than five words can be extracted). vectors obtained by different encoders are sent to MLP layer to output a probability in terms of softmax to indicate similarity results. All the testing algorithms output a value to measure the similarity between two texts. We record the similarity values of all the text pairs in the testing corpora via the given algorithms and take the mean of all the values as the threshold to decide whether two texts are similar or not. To make the obtained results more persuasive, we add significant test. We separate each testing corpus into ten parts, and record calculating results in each part. Two-tail paired t-test is applied to determine whether the results obtained by different algorithms over the ten times' calculations are significantly different or not. We set three significant levels as 0.01, 0.05, and 0.1 (labelled as ***, **, and *). As shown in Table 4, we list the results obtained in different languages and in different tasks. It can be found that supervised algorithms overwhelm unsupervised algorithms by a large margin in all the testing corpora. The reason is straightforward, since supervised algorithms can utilize training data to obtain a reasonable hyperplane to separate similar text pairs from dissimilar ones. Correspondingly, unsupervised algorithms cannot acquire any transcendental guidance to help model the discrimination between similar and dissimilar text pairs. They only depend on data's natural distribution, thus, obtain lower performance.