SlowGuess/NAACL_2021_OCR huggingface dataset:
Linh The Nguyen, Linh Van Ngo, Khoat Than, and Thien Huu Nguyen. 2019. Employing the correspondence of relations and connectives to identify implicit discourse relations via label embeddings. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4201-4207, Florence, Italy. Association for Computational Linguistics.
Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1532-1543, Doha, Qatar. Association for Computational Linguistics.
Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 2227-2237, New Orleans, Louisiana. Association for Computational Linguistics.
Emily Pittler, Annie Louis, and Ani Nenkova. 2009. Automatic sense prediction for implicit discourse relations in text. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, pages 683-691, Suntec, Singapore. Association for Computational Linguistics.
Rashmi Prasad, Nikhil Dinesh, Alan Lee, Eleni Miltsakaki, Livio Robaldo, Aravind Joshi, and Bonnie Webber. 2008. The Penn discourse TreeBank 2.0. In Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC'08), Marrakech, Morocco. European Language Resources Association (ELRA).
Lianhui Qin, Zhisong Zhang, Hai Zhao, Zhiting Hu, and Eric Xing. 2017. Adversarial connective-exploiting networks for implicit discourse relation classification. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1006-1017, Vancouver, Canada. Association for Computational Linguistics.
Samuel Ronqvist, Niko Schenk, and Christian Chiarcos. 2017. A recurrent neural model with attention for the recognition of Chinese implicit discourse relations. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 256-262, Vancouver, Canada. Association for Computational Linguistics.
Michael Schlichtkrull, Thomas N Kipf, Peter Bloem, Rianne Van Den Berg, Ivan Titov, and Max Welling. 2018. Modeling relational data with graph convolutional networks. In European Semantic Web Conference, pages 593-607. Springer.
Wei Shi and Vera Demberg. 2019. Next sentence prediction helps implicit discourse relation classification within and across domains. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 5790-5796, Hong Kong, China. Association for Computational Linguistics.
Sheng Xu, Peifeng Li, Fang Kong, Qiaoming Zhu, and Guodong Zhou. 2019. Topic tensor network for implicit discourse relation recognition in Chinese. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 608-618, Florence, Italy. Association for Computational Linguistics.
Yue Zhang, Qi Liu, and Linfeng Song. 2018. Sentence-state LSTM for text representation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 317-327, Melbourne, Australia. Association for Computational Linguistics.
A Sentence-state LSTM
Sentence-state LSTM (SLSTM) (Zhang et al., 2018) is a novel graph RNN. We denote the process of updating sentence states as  $\mathrm{SLSTM}_{h\to g}$ , the process of updating token states as  $\mathrm{SLSTM}_{g\to h}$ .
$\mathbf{SLSTM}_{\mathbf{h}\to \mathbf{g}}$  . At the  $t$  -th round, the hidden state of  $i$  -th sentence  $g_{i}^{t}$  is computed based on the values  $h_{i,j}^{t - 1}$  for all  $j\in [0,\dots ,|S_i|]$  ..
$$
\overline {{h}} _ {i} = a v g (h _ {i, 0} ^ {t - 1}, h _ {i, 1} ^ {t - 1}, \dots , h _ {i, | S _ {i})} ^ {t - 1})
$$
$$
\widehat {f} _ {g _ {i}} ^ {t} = \sigma (W _ {g} g _ {i} ^ {t - 1} + U _ {g} \overline {{h}} _ {i} + b _ {g})
$$
$$
\widehat {f} _ {i, j} ^ {t} = \sigma (W _ {f} g _ {i} ^ {t - 1} + U _ {f} h _ {i, j} ^ {t - 1} + b _ {f})
$$
$$
o _ {i} ^ {t} = \sigma \left(W _ {o} g _ {i} ^ {t - 1} + U _ {o} \bar {h} _ {i} + b _ {o}\right) \tag {7}
$$
$$
f _ {i, 0} ^ {t}, \dots f _ {i, | S _ {i} |} ^ {t}, f _ {g _ {i}} ^ {t} = F _ {s} (\widehat {f} _ {i, 0} ^ {t}, \dots , \widehat {f} _ {i, | S _ {i} |} ^ {t}, \widehat {f} _ {g _ {i}} ^ {t})
$$
$$
v _ {g _ {i}} ^ {t} = f _ {g _ {i}} ^ {t} \odot v _ {g _ {i}} ^ {t - 1} + \sum_ {j} f _ {i, j} ^ {t} \odot v _ {i, j} ^ {t - 1}
$$
$$
g _ {i} ^ {t} = o _ {i} ^ {t} \odot \tanh \left(v _ {g _ {i}} ^ {t}\right)
$$
where  $W_{*}, U_{*}$  and  $b_{*}$  are model parameters, here,  $* \in \{g, f, o\}$ .  $|S_i|$  is the number of tokens of the i-th sentence.  $f_{i,0}^{t}, \ldots, f_{i,|S_i|}$  and  $f_{g_i}^t$  are gates controlling information from  $v_{i,0}^{t-1}, \ldots, v_{i,|S_i|}^{t-1}, v_{g_i}^{t-1}$ , respectively.  $o_i^t$  is an output gate from the recurrent cell  $v_{g_i}^t$  to  $g_i^t$ .  $F_s$  represents the softmax function.
$\mathbf{SLSTM}_{\mathbf{g}\rightarrow \mathbf{h}}$  . At the  $t$  -th round, the hidden state of each token  $h_{i,j}^{t}$  is computed based on the initial input  $x_{i,j}$  , its hidden state of last round  $h_{i,j}^{t - 1}$  , the hidden states of its neighbors of last round  $h_{i,j - 1}^{t - 1}$ $h_{i,j + 1}^{t - 1}$  and the contextual representation  $c_{i}^{t - 1}$
$$
\varepsilon_ {i, j} ^ {t} = [ h _ {i, j - 1} ^ {t - 1}, h _ {i, j} ^ {t - 1}, h _ {i, j + 1} ^ {t - 1} ],
$$
$$
\widehat {i} _ {i, j} ^ {t} = \sigma \left(W _ {i} \varepsilon_ {i, j} ^ {t} + U _ {i} x _ {i, j} + V _ {i} c _ {i} ^ {t - 1} + b _ {i}\right)
$$
$$
\widehat {l} _ {i, j} ^ {t} = \sigma (W _ {l} \varepsilon_ {i, j} ^ {t} + U _ {l} x _ {i, j} + V _ {l} c _ {i} ^ {t - 1} + b _ {l})
$$
$$
\widehat {r} _ {i, j} ^ {t} = \sigma (W _ {r} \varepsilon_ {i, j} ^ {t} + U _ {r} x _ {i, j} + V _ {r} c _ {i} ^ {t - 1} + b _ {r})
$$
$$
\widehat {f} _ {i, j} ^ {t} = \sigma \left(W _ {f} \varepsilon_ {i, j} ^ {t} + U _ {f} x _ {i, j} + V _ {f} c _ {i} ^ {t - 1} + b _ {f}\right)
$$
$$
\widehat {s} _ {i, j} ^ {t} = \sigma (W _ {s} \varepsilon_ {i, j} ^ {t} + U _ {s} x _ {i, j} + V _ {s} c _ {i} ^ {t - 1} + b _ {s})
$$
$$
o _ {i, j} ^ {t} = \sigma (W _ {o} \varepsilon_ {i, j} ^ {t} + U _ {o} x _ {i, j} + V _ {o} c _ {i} ^ {t - 1} + b _ {o})
$$
$$
u _ {i, j} ^ {t} = \tanh (W _ {u} \varepsilon_ {i, j} ^ {t} + U _ {u} x _ {i, j} + V _ {u} c _ {i} ^ {t - 1} + b _ {u})
$$
$$
i _ {i, j} ^ {t}, l _ {i, j} ^ {t}, r _ {i, j} ^ {t}, f _ {i, j} ^ {t}, s _ {i, j} ^ {t} = F _ {s} (\widehat {i} _ {i, j} ^ {t}, \widehat {l} _ {i, j} ^ {t}, \widehat {r} _ {i, j} ^ {t}, \widehat {f} _ {i, j} ^ {t}, \widehat {s} _ {i, j} ^ {t})
$$
$$
\begin{array}{l} v _ {i, j} ^ {t} = l _ {i, j} ^ {t} \odot v _ {i, j - 1} ^ {t - 1} + f _ {i, j} ^ {t} \odot v _ {i, j} ^ {t - 1} + r _ {i, j} ^ {t} \odot v _ {i, j + 1} ^ {t - 1} \\ + s _ {i, j} ^ {t} \odot v _ {g _ {i}} ^ {t - 1} + i _ {i, j} ^ {t} \odot u _ {i, j} ^ {t} \\ \end{array}
$$
$$
h _ {i, j} ^ {t} = o _ {i, j} ^ {t} \odot \tanh  \left(v _ {i, j} ^ {t}\right) \tag {8}
$$
where  $W_{*}, U_{*}$  and  $b_{*}$  are model parameters, here,  $* \in \{i, l, r, f, s, o\}$ .  $F_{s}$  represents the softmax
function, and  $\sigma$  represents the sigmoid function.  $\widehat{i}_{i,j}^{t},\widehat{l}_{i,j}^{t},\widehat{r}_{i,j}^{t},\widehat{r}_{i,j}^{t},f_{i,j}^{t}$  are gates conveying information from the  $\varepsilon_{i,j}^{t}$  and  $x_{i,j}$  to the cell state  $v_{i,j}^{t}$  which are normalised.  $o_t^i$  is an output gate from the cell  $v_{i,j}^{t}$  to the hidden state  $h_{i,j}^{t}$ .