SlowGuess/NAACL_2021_OCR huggingface dataset:
Sho Takase, Jun Suzuki, Naoaki Okazaki, Tsutomu Hirao, and Masaaki Nagata. 2016. Neural headline generation on Abstract Meaning Representation. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1054-1059, Austin, Texas. Association for Computational Linguistics.
Rob A. Van der Sandt. 1992. Presupposition projection as anaphora resolution. Journal of Semantics, 9(4):333-377.
Rik van Noord, Lasha Abzianidze, Antonio Toral, and Johan Bos. 2018. Exploring neural methods for parsing discourse representation structures. Transactions of the Association for Computational Linguistics, 6:619-633.
Rik van Noord, Antonio Toral, and Johan Bos. 2019. Linguistic information in neural semantic parsing with multiple encoders. In Proceedings of the 13th International Conference on Computational Semantics (IWCS), pages 24-31, Gothenburg, Sweden.
Noortje J. Venhuizen, Johan Bos, Petra Hendriks, and Harm Brouwer. 2018. Discourse semantics with information structure. Journal of Semantics, 35(1):127-169.
Oriol Vinyals, Meire Fortunato, and Navdeep Jaitly. 2015. Pointer networks. In Neural Information Processing Systems (NeurIPS), pages 2692-2700, Montréal, Canada.
Juen-tin Wang. 1980. On computational sentence generation from logical form. In Proceedings of the 8th International Conference on Computational Linguistics (COLING), pages 405-411, Tokyo, Japan.
Tianming Wang, Xiaojun Wan, and Hanqi Jin. 2020. AMR-to-text generation with graph transformer. Transactions of the Association for Computational Linguistics, 8:19-33.
Yaushian Wang, Hung-Yi Lee, and Yun-Nung Chen. 2019. Tree transformer: Integrating tree structures into self-attention. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 1060–1070, Hong Kong, China.
Michael White, Rajakrishnan Rajkumar, and Scott Martin. 2007. Towards broad coverage surface realization with CCG. In Proceedings of the Workshop on Using Corpora for NLG: Language Generation and Machine Translation (UCNLG+MT), pages 267-276, Copenhagen, Denmark.
Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and S Yu Philip. 2020. A comprehensive survey on graph neural networks. IEEE Transactions on Neural Networks and Learning Systems.
Kun Xu, Lingfei Wu, Zhiguo Wang, Yansong Feng, and Vadim Sheinin. 2018. SQL-to-text generation with graph-to-sequence model. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 931-936, Brussels, Belgium.
Xuchen Yao, Gosse Bouma, and Yi Zhang. 2012. Semantics-based question generation and implementation. Dialogue and Discourse, 2(3):11-42.
Yongjing Yin, Linfeng Song, Jinsong Su, Jiali Zeng, Chulun Zhou, and Jiebo Luo. 2019. Graph-based neural sentence ordering. In Proceedings of the 28th International Joint Conference on Artificial Intelligence (IJCAI), pages 5387-5393, Macao, China.
Jie Zhu, Junhui Li, Muhua Zhu, Longhua Qian, Min Zhang, and Guodong Zhou. 2019. Modeling graph structure in transformer for better AMR-to-text generation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 5462-5471, Hong Kong, China.
A Counting Method
We count how frequently condition  $r_i$  appears before  $r_j$  with type  $t$ . Type  $t$  is identified according to the overlap between the arguments of the two conditions<sup>6</sup>. For example,  $r_i = (\text{Named}, x_3, \text{"tom"})$  and  $r_j = (\text{Agent}, e_1, x_3)$  has the type “ $a_0 \to a_1$ ”, showing that the first argument in  $r_i$  equals to the second argument in  $r_j$ . We score the order of two conditions using the following function:
$$
\begin{array}{l} \operatorname {S C O R E} _ {\mathbb {K}} (R \mid R _ {s e t}) = \sum_ {i} \left(\sum_ {i <   j} \operatorname {C o u n t} _ {\mathbb {K}} \left(r _ {i}, r _ {j}\right)\right) \tag {16} \\ - \sum_ {j <   i} \operatorname {C o u n t} _ {\mathbb {K}} \left(r _ {i}, r _ {j}\right) \\ \end{array}
$$
where COUNT returns the frequency of a pair of conditions subject to a dataset or corpus  $\mathbb{K}$ ; the score increases with  $r_j$  following  $r_i$  more frequently than preceding it.
A.1 Types
We define different types of relations between two conditions based on argument overlap (i.e., two simple conditions, a simple and a complex condition, and two complex conditions).
Simple and Simple Given two simple conditions,  $r_i = (l_i, a_{0i}, a_{1i})$  and  $r_j = (l_j, a_{0j}, a_{1j})$  (with different arguments), we define their types as:
-  $t = a_0 \to a_0$  if  $a_{0i} = a_{0j}$  and  $a_{1i} \neq a_{1j}$ .
-  $t = a_1 \rightarrow a_1$  if  $a_{0i} \neq a_{0j}$  and  $a_{1i} = a_{1j}$ .
-  $t = a_0 \to a_0, a_1 \to a_1$  if  $a_{0i} = a_{0j}$  and  $a_{1i} = a_{1j}$ .
-  $t = a_0 \to a_1$  if  $a_{0i} = a_{1j}$  and  $a_{1i} \neq a_{0j}$ .
-  $t = a_{1} \to a_{0}$  if  $a_{1i} = a_{0j}$  and  $a_{0i} \neq a_{1j}$ .
-  $t = a_0 \to a_1, a_1 \to a_0$  if  $a_{1i} = a_{0j}$  and  $a_{0i} = a_{1j}$ .
-  $t =$  None if others
Simple and Complex Given a simple conditions  $r_i = (l_i, a_{0i}, a_{1i})$  and a complex condition  $r_j = (l_j, V_j)$ , the types are defined as:
-  $t = 1$  if  $a_{0i} \in V_j$  and  $a_{1i} \notin V_j$ .
-  $t = 1$  if  $a_{0i} \notin V_j$  and  $a_{1i} \in V_j$ .
-  $t = 2$  if  $a_{0i} \in V_j$  and  $a_{1i} \in V_j$ .
-  $t =$  None if others
Algorithm 1 Greedy Ordering Algorithm
Input:  $R_{\text{set}}$ , set of conditions
Output:  $R^{*}$ , list of ordered conditions
1:  $R^{*} = \prod$
2: while  $\bar{R}_{set}$  is not empty do
3:  $r^{*} = \arg \max_{r\in R_{set}}\mathrm{PARTIAL}(r)$
4:  $R_{set}$  removes  $r^i$
5:  $R^{*}$  appends  $r^{*}$
6: end while
Complex and Complex Given two complex conditions,  $r_i = (l_i,V_i)$  and  $r_j = (l_j,V_j)$ , the types are defined as:
-  $t = \text{intersection}$  if  $V_{i} \cap V_{j} \neq \emptyset$ .
-  $t =$  None otherwise
A.2 Greedy Algorithm
We generate an ordering of conditions following greedy Algorithm 1 which identifies the highest scoring pair in  $R_{set}$ , appends it to partial ordering  $R^{*}$ , and keeps going until  $R_{set}$  is empty. The score is given by the function:
$$
\begin{array}{l} \operatorname {P A R T I A L} _ {\mathbb {K}} (r) = \sum_ {r ^ {\prime} \in R _ {s e t}} \operatorname {C O u n t} \left(r, r ^ {\prime}\right) _ {\mathbb {K}} \tag {17} \\ - \sum_ {r ^ {\prime \prime} \in R} \operatorname {C O u n t} (r, r ^ {\prime \prime}) _ {\mathbb {K}}, \\ \end{array}
$$
B Edge Construction Algorithm
Algorithm 2 shows how edges are created for graph ordering.
C Model Settings
In the following, we report the best experimental settings for our condition ordering and text generation models.
C.1 Condition Ordering Models
Model hyperparameters are shown in Table 6. The size of the token embeddings (in the graph nodes) and edge embeddings is 100. The node hidden dimension is the same as the hidden dimension of the BILSTM which is used to encode a sequence of input words for each node and the hidden dimension of PointerNet. The BILSTM and PointerNet have one layer. Training hyperparameters are shown in Table 7.
<sup>6</sup>We only consider (and count) conditions with overlapping arguments, i.e., their type  $t$  is not None.
hyperparameters	value
embedding dim	50
hidden dim	300
LSTM layer	1
graph step	2
beam size	64
C.2 Text Generation Models
Model hyperparameters are shown in Table 8. The size of the input embeddings in the encoder and the decoder is 300. The hidden dimensions of the encoder and decoder are 512. Both the encoder and decoder have two layers. The hyperparameters of the training are shown in Table 9.
We show below model performance with various hyperparameters. Best hyperparameters were manually chosen after monitoring model accuracy on the development set.
We take BiGraph as our final condition ordering model (see Table 10) and Sibling with BiGraph as the DRS-to-text generation model (see Table 11).
Table 6: Hyperparameters of condition ordering model.
hyperparameters	value
optimizer	adam
learning rate	0.001
β	(0.9, 0.999)
dropout rate	0.5
batch size	32
max gradient norm	5
training steps	400,000
learning rate decay	0.3
start decay steps	200,000
decay steps	20,000
warmup steps	40,000
Table 7: Training hyperparameters for the condition ordering models.
hyperparameters	value
embedding dim	300
hidden dim	512
layer	2
attention	general
beam size	5
Table 8: Hyperparameters of text generation model.
hyperparameters	value
optimizer	adam
learning rate	0.001
β	(0.9, 0.999)
dropout rate	0.5
batch size	30000
batch type	tokens
max gradient norm	5
training steps	30,000
learning rate decay	0.5
start decay steps	8,000
decay steps	1,000
warmup steps	4,000
Table 9: Training hyperparameters for text generation model.
embedding	hidden	step	dropout	beam	Acc
50	300	2	0.5	64	69.10
100	—	—	—	—	68.61
150	—	—	—	—	67.76
—	100	—	—	—	67.33
—	200	—	—	—	68.77
—	—	1	—	—	55.71
—	—	3	—	—	69.12
—	—	—	0.2	—	68.02
—	—	—	0.4	—	68.52
—	—	—	0.6	—	67.83
—	—	—	—	16	69.03
—	—	—	—	32	69.07
—	—	—	—	96	69.10
Table 10: Performance of condition ordering model (development set; various hyperparameters). Increasing model size leads to out-of-memory problems when training on a single GPU.
We provide example output of our final model (Sibling+GraphOrder) on the GMB test dataset. The DRS in tree format with condition ordering given by GraphOrder is shown in Figures 9-11. Figure 10 expands nonterminal  $b_{33}$  in Figure 9, and Figure 11 expands nonterminal  $b_{28}$  in Figure 10. The corresponding document generated by Sibling is:
the u.s. dollar hit a record low against the euro tuesday . it took a dollar , but 48 cents to buy one euro , and a series of problems including the key of the u.s. housing sector in which has been battered by the slowing n.01 , including continuing the u.s. economy , the dollar . speculator are selling dollars because they expect that the u.s. central bank will try to stimulate the economy by cutting interest rates soon . u.s. lower interest rates can cut the return on investments . the falling dollar is prompting oil-rich nations around the persian gulf to consider ending the practice of linking the value of its currency to those of the dollar and instead supplement the u.s. currency . such a move would reduce demand for dollars and weaken the u.s. currency .
embedding	hidden	layer	dropout	beam	BLEU
300	512	2	0.5	5	58.97
100	—	—	—	—	58.19
200	—	—	—	—	58.81
—	128	—	—	—	43.36
—	256	—	—	—	55.06
—	—	1	—	—	52.58
—	—	3	—	—	58.94
—	—	—	0.1	—	47.43
—	—	—	0.2	—	58.24
—	—	—	0.3	—	58.97
—	—	—	0.4	—	58.96
—	—	—	0.6	—	58.22
—	—	—	—	1	58.73
—	—	—	—	3	58.91
—	—	—	—	10	58.97
Table 11: Performance of text generation model (development set; various hyperparameters). Increasing model size leads to out-of-memory problems when training on a single GPU.
Figure 9: A partial DRS in tree format with condition ordering recovered by GraphOrder.
Figure 10: A partial DRS in tree format with condition ordering recovered by GraphOrder.
Figure 11: A partial DRS in tree format with condition ordering recovered by GraphOrder.