SlowGuess/ACL_2020_OCR huggingface dataset:
Semantic Similarity BERT $_{finetune}$  performs better than other neural models, including In-fersent (Conneau et al., 2017), ESIM (Chen et al., 2017), BERTScore (Zhang et al., 2020) and pretrained BERT embedding (Devlin et al., 2019). For BERTScore, we useidf weighting, and treat simple sentence as reference.
5 Experiments on Automatic Sentence Simplification
In this section, we compare different automatic text simplification models trained on our new parallel corpora, NEWSELA-AUTO and WIKI-AUTO, with their counterparts trained on the existing datasets. We establish a new state-of-the-art for sentence simplification by training a Transformer model with initialization from pre-trained BERT checkpoints.
5.1 Comparison with existing datasets
Existing datasets of complex-simple sentences, NEWSELA (Xu et al., 2015) and WIKILARGE (Zhang and Lapata, 2017), were aligned using lexical similarity metrics. NEWSELA dataset (Xu et al.,
2015) was aligned using JaccardAlign (§4.1). WIK-ILARGE is a concatenation of three early datasets (Zhu et al., 2010; Woodsend and Lapata, 2011; Coster and Kauchak, 2011) where sentences in Simple/Normal English Wikipedia and editing history were aligned by TF-IDF cosine similarity.
For our new NEWSELA-AUTO, we partitioned the article sets such that there is no overlap between the new train set and the old test set, and vice-versa. Following Zhang and Lapata (2017), we also excluded sentence pairs corresponding to the levels 0-1, 1-2 and 2-3. For our WIKIAUTO dataset, we eliminated sentence pairs with high  $(>0.9)$  or low  $(<0.1)$  lexical overlap based on BLEU scores (Papineni et al., 2002), following Šajner et al. (2015). We observed that sentence pairs with low BLEU are often inaccurate paraphrases with only shared named entities and the pairs with high BLEU are dominated by sentences merely copied without simplification. We used the benchmark TURK corpus (Xu et al., 2016) for evaluation on Wikipedia, which consists of 8 human-written references for sentences in the validation and test sets. We discarded sentences in TURK corpus from WIKIAUTO. Table 4 shows the statistics of the existing and our new datasets.
5.2 Baselines and Simplification Models
We compare the following seq2seq models trained using our new datasets versus the existing datasets:
1. A BERT-initialized Transformer, where the encoder and decoder follow the  $\mathrm{BERT}_{\mathrm{base}}$  architecture. The encoder is initialized with the same checkpoint and the decoder is randomly initialized (Rothe et al., 2020).
2. A randomly initialized Transformer with the same  $\mathrm{BERT}_{\mathrm{base}}$  architecture as above.
3. A BiLSTM-based encoder-decoder model used in Zhang and Lapata (2017).
4. EditNTS (Dong et al., 2019), a state-of-the-art neural programmer-interpreter (Reed and de Freitas, 2016) approach that predicts explicit edit operations sequentially.
In addition, we compared our BERT-initialized Transformer model with the released system outputs from Kriz et al. (2019) and EditNTS (Dong et al., 2019). We implemented our LSTM and Transformer models using Fairseq. We provide the model and training details in Appendix D.1.
<sup>7</sup>https://github.com/yuedongP/EditNTS
<sup>8</sup>https://github.com/pytorch/fairseq
7949
Evaluation on our new test set	Evaluation on old test set
SARI	add	keep	del	FK	Len	SARI	add	keep	del	FK	Len
Complex (input)	11.9	0.0	35.5	0.0	12	24.3	12.5	0.0	37.7	0.0	11	22.9
Models trained on old dataset (original NEWSELA corpus released in (Xu et al., 2015))
\(Transformer_{rand}\)	33.1	1.8	22.1	75.4	6.8	14.2	34.1	2.0	25.5	74.8	6.7	14.2
LSTM	35.6	2.8	32.1	72.0	8.2	16.9	36.2	2.5	34.9	71.3	7.7	16.3
EditNTS	35.5	1.8	30.0	75.4	7.1	14.1	36.1	1.7	32.8	73.8	7.0	14.1
\(Transformer_{bert}\)	34.4	2.4	25.2	75.8	7.0	14.5	35.1	2.7	27.8	74.8	6.8	14.3
Models trained on our new dataset (NEWSELA-AUTO)
\(Transformer_{rand}\)	35.6	3.2	28.4	75.0	7.1	14.4	35.2	2.5	29.7	73.5	7.0	14.2
LSTM	35.8	3.9	30.5	73.1	7.0	14.3	36.4	3.3	33.0	72.9	6.6	14.0
EditNTS	35.8	2.4	29.4	75.6	6.3	11.6	35.7	1.8	31.1	74.2	6.1	11.5
\(Transformer_{bert}\)	36.6	4.5	31.0	74.3	6.8	13.3	36.8	3.8	33.1	73.4	6.8	13.5
Simple (reference)	-	-	-	-	6.6	13.2	-	-	-	-	6.2	12.6
Table 5: Automatic evaluation results on NEWSELA test sets comparing models trained on our dataset NEWSELA-AUTO against the existing dataset (Xu et al., 2015). We report SARI, the main automatic metric for simplification, precision for deletion and F1 scores for adding and keeping operations. Add scores are low partially because we are using one reference. Bold typeface and underline denote the best and the second best performances respectively. For Flesch-Kincaid (FK) grade level and average sentence length (Len), we consider the values closest to reference as the best.
Model	F	A	S	Avg.
LSTM	3.44	2.86	3.31	3.20
EditNTS (Dong et al., 2019)†	3.32	2.79	3.48	3.20
Rerank (Kriz et al., 2019)†	3.50	2.80	3.46	3.25
TransformerBERT (this work)	3.64	3.12	3.45	3.40
Simple (reference)	3.98	3.23	3.70	3.64
Table 6: Human evaluation of fluency (F), adequacy (A) and simplicity (S) on the old NEWSELA test set. †We used the system outputs shared by the authors.
Model	Train	F	A	S	Avg.
LSTM	old	3.57	3.27	3.11	3.31
LSTM	new	3.55	2.98	3.12	3.22
TransformerBERT	old	2.91	2.56	2.67	2.70
TransformerBERT	new	3.76	3.21	3.18	3.39
Simple (reference)	—	4.34	3.34	3.37	3.69
Table 7: Human evaluation of fluency (F), adequacy (A) and simplicity (S) on NEWSELA-AUTO test set.
In this section, we evaluate different simplification models trained on our new datasets versus on the old existing datasets using both automatic and human evaluation.
5.3.1 Automatic Evaluation
We report SARI (Xu et al., 2016), Flesch-Kincaid (FK) grade level readability (Kincaid and Chissom, 1975), and average sentence length (Len). While SARI compares the generated sentence to a set of reference sentences in terms of correctly inserted, kept and deleted n-grams  $(n\in \{1,2,3,4\})$ , FK measures the readability of the generated sentence. We also report the three rewrite operation scores used in SARI: the precision of delete (del), the F1-scores of add (add), and keep (keep) operations.
Tables 5 and 8 show the results on Newsela and
Figure 3: Manual inspection of 100 random sentences generated by Transformer<sub>bert</sub> trained on NEWSELA-AUTO and existing NEWSELA datasets, respectively.
Wikipedia datasets respectively. Systems trained on our datasets outperform their equivalents trained on the existing datasets according to SARI. The difference is notable for  $\mathrm{Transformer}_{bert}$  with a  $6.4\%$  and  $3.7\%$  increase in SARI on NewSELA-AUTO test set and TURK corpus, respectively. Larger size and improved quality of our datasets enable the training of complex Transformer models. In fact,  $\mathrm{Transformer}_{bert}$  trained on our new datasets outperforms the existing state-of-the-art systems for automatic text simplification. Although improvement in SARI is modest for LSTM-based models (LSTM and EditNTS), the increase in F1 scores for addition and deletion operations indicate that the models trained on our datasets make more meaningful changes to the input sentence.
5.3.2 Human Evaluation
We also performed human evaluation by asking five Amazon Mechanical Turk workers to rate fluency, adequacy and simplicity (detailed instructions in Appendix D.2) of 100 random sentences generated by different simplification models trained on NewSELA-AUTO and the existing dataset. Each
SARI	add	keep	del	FK	Len
Complex (input)	25.9	0.0	77.8	0.0	13.6	22.4
Models trained on old dataset (WIKILARGE)
LSTM	33.8	2.5	65.6	33.4	11.6	20.6
Transformerrand	33.5	3.2	64.1	33.2	11.1	17.7
EditNTS	35.3	3.0	63.9	38.9	11.1	18.5
Transformerbert	35.3	4.4	66.0	35.6	10.9	17.9
Models trained on our new dataset (WIKI-AUTO)
LSTM	34.0	2.8	64.0	35.2	11.0	19.3
Transformerrand	34.7	3.3	68.8	31.9	11.7	18.7
EditNTS	36.4	3.6	66.1	39.5	11.6	20.2
Transformerbert	36.6	5.0	67.6	37.2	11.4	18.7
Simple (reference)	-	-	-	-	11.7	20.2
Table 8: Automatic evaluation results on Wikipedia TURK corpus comparing models trained on WIKI-AUTO and WIKILARGE (Zhang and Lapata, 2017).
worker evaluated these aspects on a 5-point Likert scale. We averaged the ratings from five workers. Table 7 demonstrates that  $\mathrm{Transformer}_{bert}$  trained on NEWSELA-AUTO greatly outperforms the one trained on the old dataset. Even with shorter sentence outputs, our  $\mathrm{Transformer}_{bert}$  retained similar adequacy as the LSTM-based models. Our  $\mathrm{Transformer}_{bert}$  model also achieves better fluency, adequacy, and overall ratings compared to the SOTA systems (Table 6). We provide examples of system outputs in Appendix D.3. Our manual inspection (Figure 3) also shows that  $\mathrm{Transformer}_{bert}$  trained on NEWSELA-AUTO performs  $25\%$  more paraphrasing and deletions than its variant trained on the previous NEWSELA (Xu et al., 2015) dataset.
Text simplification is considered as a text-to-text generation task where the system learns how to simplify from complex-simple sentence pairs. There is a long line of research using methods based on hand-crafted rules (Siddharthan, 2006; Niklaus et al., 2019), statistical machine translation (Narayan and Gardent, 2014; Xu et al., 2016; Wubben et al., 2012), or neural seq2seq models (Zhang and Lapata, 2017; Zhao et al., 2018; Nisioi et al., 2017). As the existing datasets were built using lexical similarity metrics, they frequently omit paraphrases and sentence splits. While training on such datasets creates conservative systems that rarely paraphrase, evaluation on these datasets exhibits an unfair preference for deletion-based simplification over paraphrasing.
Sentence alignment has been widely used to extract complex-simple sentence pairs from parallel articles for training text simplification systems. Previous work used surface-level similarity metrics,
such as TF-IDF cosine similarity (Zhu et al., 2010; Woodsend and Lapata, 2011; Coster and Kauchak, 2011; Paetzold et al., 2017), Jaccard-similarity (Xuet al., 2015), and other lexical features (Hwang et al., 2015; Stajner et al., 2018). Then, a greedy (Stajner et al., 2018) or dynamic programming (Barzilay and Elhadad, 2003; Paetzold et al., 2017) algorithm was used to search for the optimal alignment. Another related line of research (Smith et al., 2010; Tufis et al., 2013; Tsai and Roth, 2016; Gottschalk and Demidova, 2017; Aghaebrahimian, 2018; Thompson and Koehn, 2019) aligns parallel sentences in bilingual corpora for machine translation.
In this paper, we proposed a novel neural CRF model for sentence alignment, which substantially outperformed the existing approaches. We created two high-quality manually annotated datasets (NEWSELA-MANUAL and WIKI-MANUAL) for training and evaluation. Using the neural CRF sentence aligner, we constructed two largest sentence-aligned datasets to date (NEWSELA-AUTO and WIKI-AUTO) for text simplification. We showed that a BERT-initialized Transformer trained on our new datasets establishes new state-of-the-art performance for automatic sentence simplification.
We thank three anonymous reviewers for their helpful comments, Newsela for sharing the data, Ohio Supercomputer Center (Center, 2012) and NVIDIA for providing GPU computing resources. We also thank Sarah Flanagan, Bohan Zhang, Raleigh Potluri, and Alex Wing for help with data annotation. This research is supported in part by the NSF awards IIS-1755898 and IIS-1822754, ODNI and IARPA via the BETTER program contract 19051600004, ARO and DARPA via the Social-Sim program contract W911NF-17-C-0095, Figure Eight AI for Everyone Award, and Criteo Faculty Research Award to Wei Xu. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies, either expressed or implied, of NSF, ODNI, IARPA, ARO, DARPA or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for governmental purposes notwithstanding any copyright annotation therein.
7951
Ahmad Aghaebrahimian. 2018. Deep neural networks at the service of multilingual parallel sentence extraction. In Proceedings of the 27th International Conference on Computational Linguistics.
Fernando Alva-Manchego, Joachim Bingel, Gustavo Paetzold, Carolina Scarton, and Lucia Specia. 2017. Learning how to simplify from explicit labeling of complex-simplified text pairs. In Proceedings of the Eighth International Joint Conference on Natural Language Processing.
Ron Artstein and Massimo Poesio. 2008. Survey article: Inter-coder agreement for computational linguistics. Computational Linguistics.
Regina Barzilay and Noemie Elhadad. 2003. Sentence alignment for monolingual comparable corpora. In Proceedings of the 2003 Conference on Empirical Methods in Natural Language Processing.
Ohio Supercomputer Center. 2012. Oakley supercomputer. http://osc.edu/ark:/19495/hpc0cvqn.
R. Chandrasekar, Christine Doran, and B. Srinivas. 1996. Motivations and methods for text simplification. In The 16th International Conference on Computational Linguistics.
Han-Bin Chen, Hen-Hsen Huang, Hsin-Hsi Chen, and Ching-Ting Tan. 2012. A simplification-translation-restoration framework for cross-domain SMT applications. In Proceedings of the 24th International Conference on Computational Linguistics.
Qian Chen, Xiaodan Zhu, Zhen-Hua Ling, Si Wei, Hui Jiang, and Diana Inkpen. 2017. Enhanced LSTM for natural language inference. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics.
Alexis Conneau, Douwe Kiela, Holger Schwenk, Loic Barrault, and Antoine Bordes. 2017. Supervised learning of universal sentence representations from natural language inference data. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing.
William Coster and David Kauchak. 2011. Simple English Wikipedia: A new text simplification task. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics.
Yue Dong, Zichao Li, Mehdi Rezagholizadeh, and Jackie Chi Kit Cheung. 2019. EditNTS: An neural
programmer-Interpreter model for sentence simplification through explicit editing. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.
Noemie Elhadad and Komal Sutaria. 2007. Mining a lexicon of technical terms and lay equivalents. In Biological, translational, and clinical language processing.
Simon Gottschalk and Elena Demidova. 2017. Multiwiki: interlingual text passage alignment in wikipedia. ACM Transactions on the Web.
Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation.
William Hwang, Hannaneh Hajishirzi, Mari Ostendorf, and Wei Wu. 2015. Aligning sentences from standard Wikipedia to simple Wikipedia. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics.
Sebastien Jean, Orhan Firat, Kyunghyun Cho, Roland Memisevic, and Yoshua Bengio. 2015. Montreal neural machine translation systems for WMT'15. In Proceedings of the Tenth Workshop on Statistical Machine Translation.
Tomoyuki Kajiwara, Hiroshi Matsumoto, and Kazuhide Yamamoto. 2013. Selecting proper lexical paraphrase for children. In Proceedings of the 25th Conference on Computational Linguistics and Speech Processing.
Robert P. Jr.; Rogers Richard L.; Kincaid, J. Peter; Fishburne and Brad S. Chissom. 1975. Derivation of new readability formulas (automated readability index, fog count and flesch reading ease formula) for navy enlisted personnel. research branch report.
Reno Kriz, João Sedoc, Marianna Apidianaki, Carolina Zheng, Gaurav Kumar, Eleni Miltsakaki, and Chris Callison-Burch. 2019. Complexity-weighted loss and diverse reranking for sentence simplification. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics.
Makoto Miwa, Rune Sætre, Yusuke Miyao, and Jun'ichi Tsujii. 2010. Entity-focused sentence simplification for relation extraction. In Proceedings of the 23rd International Conference on Computational Linguistics.
Shashi Narayan and Claire Gardent. 2014. Hybrid simplification using deep semantics and machine translation. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics.
Christina Niklaus, Matthias Cetto, André Freitas, and Siegfried Handschuh. 2019. Transforming complex sentences into a semantic hierarchy. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.
7952
Sergiu Nisioi, Sanja Štajner, Simone Paolo Ponzetto, and Liviu P. Dinu. 2017. Exploring neural text simplification models. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics.
Gustavo Paetzold, Fernando Alva-Manchego, and Lucia Specia. 2017. MASSAlign: Alignment and annotation of comparable documents. In Proceedings of the IJCNLP 2017, System Demonstrations.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: A method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics.
David Pellow and Maxine Eskenazi. 2014. An open corpus of everyday documents for simplification tasks. In Proceedings of the 3rd Workshop on Predicting and Improving Text Readability for Target Reader Populations.
Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. GloVe: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing.
Sarah E Petersen and Mari Ostendorf. 2007. Text simplification for language learners: A corpus analysis. In Proceedings of Workshop on Speech and Language Technology for Education.
Scott E. Reed and Nando de Freitas. 2016. Neural programmer-interpreters. In 4th International Conference on Learning Representations.
Luz Rello, Ricardo Baeza-Yates, and Horacio Saggion. 2013. The impact of lexical simplification by verbal paraphrases for people with and without dyslexia. In Proceedings of the 14th International Conference on Computational Linguistics and Intelligent Text Processing.
Sascha Rothe, Shashi Narayan, and Aliaksei Severyn. 2020. Leveraging pre-trained checkpoints for sequence generation tasks. Transactions of the Association for Computational Linguistics.
Horacio Saggion. 2017. Automatic text simplification. Synthesis Lectures on Human Language Technologies.
Advaith Siddharthan. 2006. Syntactic simplification and text cohesion. Research on Language and Computation.
Advaith Siddharthan and Napoleon Katsos. 2010. Reformulating discourse connectives for non-expert readers. In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics.
Jason R. Smith, Chris Quirk, and Kristina Toutanova. 2010. Extracting parallel sentences from comparable corpora using document level alignment. In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics.
Sanja Šajner, Hannah Béchara, and Horacio Saggion. 2015. A deeper exploration of the standard PB-SMT approach to text simplification and its evaluation. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing.
Sanja Štajner, Marc Franco-Salvador, Paolo Rosso, and Simone Paolo Ponzetto. 2018. CATS: A tool for customized alignment of text simplification corpora. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation.
Sanja Štajner and Maja Popovic. 2016. Can text simplification help machine translation? In Proceedings of the 19th Annual Conference of the European Association for Machine Translation.
Brian Thompson and Philipp Koehn. 2019. Vecalign: Improved sentence alignment in linear time and space. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing.
Chen-Tse Tsai and Dan Roth. 2016. Cross-lingual wikification using multilingual embeddings. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics.
Dan Tufis, Radu Ion, Stefan Dumitrescu, and Dan Stefanescu. 2013. Wikipedia as an SMT training corpus. In Proceedings of the International Conference on Recent Advances in Natural Language Processing.
Lucy Vanderwende, Hisami Suzuki, Chris Brockett, and Ani Nenkova. 2007. Beyond subbasic: Task-focused summarization with sentence simplification and lexical expansion. Inf. Process. Manage.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems.
David Vickrey and Daphne Koller. 2008. Sentence simplification for semantic role labeling. In Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumont, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, R'emi Louf, Morgan Funtowicz, and Jamie Brew. 2019. Huggingface's transformers: State-of-the-art natural language processing. ArXiv, abs/1910.03771.
7953
Kristian Woodsend and Mirella Lapata. 2011. Learning to simplify sentences with quasi-synchronous grammar and integer programming. In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing.
Sander Wubben, Antal van den Bosch, and Emiel Krahmer. 2012. Sentence simplification by monolingual machine translation. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics.
Wei Xu, Chris Callison-Burch, and Courtney Naples. 2015. Problems in current text simplification research: New data can help. Transactions of the Association for Computational Linguistics.
Wei Xu and Ralph Grishman. 2009. A parse-and-trim approach with information significance for Chinese sentence compression. In Proceedings of the 2009 Workshop on Language Generation and Summarisation.
Wei Xu, Courtney Naples, Ellie Pavlick, Quanze Chen, and Chris Callison-Burch. 2016. Optimizing statistical machine translation for text simplification. Transactions of the Association for Computational Linguistics.
Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2020. BERTScore: Evaluating text generation with BERT. In International Conference on Learning Representations.
Xingxing Zhang and Mirella Lapata. 2017. Sentence simplification with deep reinforcement learning. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing.
Sanqiang Zhao, Rui Meng, Daqing He, Andi Saptono, and Bambang Parmanto. 2018. Integrating transformer and paraphrase rules for sentence simplification. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing.
Zhemin Zhu, Delphine Bernhard, and Iryna Gurevych. 2010. A monolingual tree-based translation model for sentence simplification. In Proceedings of the 23rd International Conference on Computational Linguistics.
7954
A Neural CRF Alignment Model
We used PyTorch to implement our neural CRF alignment model. For the sentence encoder, we used Huggingface implementation (Wolf et al., 2019) of BERT<sup>10</sup> architecture with 12 layers of Transformers. When fine-tuning the BERT model, we use the representation of [CLS] token for classification. We use cross entropy loss and update the weights in all layers. Table 9 summarizes the hyperparameters of our model. Table 10 provides the thresholds for our paragraph alignment Algorithm 2, which were chosen based on NEWSELA-MANUAL dev data.
Parameter	Value	Parameter	Value
hidden units	768	# of layers	12
learning rate	0.00002	# of heads	12
max sequence length	128	batch size	8
Table 9: Parameters of our neural CRF sentence alignment model.
Threshold	Value
τ1	0.1
τ2	0.34
τ3	0.9998861788416304
τ4	0.998915818299745
τ5	0.5