# Similarity Calculation via Passage-Level Event Connection Graph
## 5 Experiments and Analyses
### Experimental Results
#### 5.2.4 Comparison of Different Algorithms
Anyway, unsupervised algorithms only choose some words with prominent distribution or aggregate all the words in the text to generate text representation, whereas long text has many words which are little related to the main content. This situation causes unsupervised algorithms obtain extremely lower performances on the manually annotated corpora which include only long texts. It can be found that our calculation obtains comparable results with supervised baseline algorithms and performs much better than unsupervised baseline algorithms especially on the corpora including long texts. The reason is totally due to our event connection graph. Based on this graph, the nodes (or words), which can represent the core event mentioned in one text, can be finally extracted. The unrelated noisy words are ignored when calculating text similarity. For this reason, we can acquire accurate similarity results on both long and short texts. Besides, to further improve performance, graph embedding is used to encode the information carried by the entire graph into the chosen nodes to make the chosen nodes carry the global information expressed by entire text. The significant test results also prove the reliability of the high performance of our calculation. In the experiments, there are also two SOTA text similarity calculations, Con-SIM and RCMD. They both obtain extraordinary results. Moreover, it can be observed that RCMD even obtains over 90% F1 value across all the testing corpora. These two methods both have two layers. The lower layer is text encoder, where Con-SIM takes BERT as the encoder and RCMD takes a more powerful model (RoBERTa) as encoder. The difference is that RoBERTa has more parameters; thus, RCMD obtains higher performance. The upper level is the interaction layer, where Con-SIM takes hierarchical interactive attention and \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multicolumn{2}{c}{**Methods**} & \multicolumn{2}{c}{**English**} & \multicolumn{2}{c}{**Chinese**} & \multicolumn{2}{c}{**Spanish**} \\ \cline{3-8} \multicolumn{2}{c}{} & **Para** & **Q\&Q** & **MA** & **Q\&Q** & **MA** & **MA** \\ \hline \multirow{5}{*}{supervised} & TextCNN & 0.84 *** & 0.82 ** & — & 0.86 *** & — & — \\  & LSTM & 0.83 ** & 0.84 ** & — & 0.83 ** & — & — \\  & LSTM+BIA & 0.83 *** & 0.87 ** & — & 0.89 *** & — & — \\  & BERT-base & 0.85 *** & 0.89 *** & — & 0.91 *** & — & — \\  & Con-SIM & 0.87 *** & 0.90 *** & — & 0.92 *** & — & — \\  & RCMD & **0.90 *** & **0.92 *** & — & **0.93 *** & — & — \\ \hline \multirow{5}{*}{unsupervised} & AVE & 0.58 * & 0.61 ** & 0.42 * & 0.63 ** & 0.49 * & 0.48 ** \\  & TR+AVE & 0.65 ** & 0.66 ** & 0.49 ** & 0.68 ** & 0.55 ** & 0.59 ** \\ \cline{1-1}  & TR+CON & 0.69 * & 0.71 *** & 0.53 ** & 0.66 ** & 0.53 ** & 0.57 ** \\ \cline{1-1}  & Ours & 0.82 *** & 0.86 *** & **0.68 *** & 0.89 *** & **0.67 *** & **0.71 *** \\ \hline \hline \end{tabular}
\end{table} 
Table 4: The comparison of our calculation with the baseline algorithms (highest values in bold. ***, **, and * indicate three significant levels as 0.01, 0.05, and 0.1.). RCMD takes two matrixes to model local interaction (inner sentence) and global interaction (outer sentence) to obtain sentence matching results. As indicated in the experiments, our proposed method obtains lower performance than those SOTA methods. However, it is easy to be explained. Those SOTA methods all take pretrained models to encode input texts with massive parameters. Anyway, those methods need to consume training data to adjust model to deal with domain-specific data. As domain changed, these methods are easy to be distorted as shown in the experiments. Compared with those methods, our proposed method is unsupervised based. Thus, it keeps its performance across domains. Anyway, with the help of passage-level document representation, our proposed method can obtain high performance. Though the performance is lower than the ones with pretrained models, its performance is much higher than the baseline unsupervised methods.