SlowGuess/ACL_2020_OCR huggingface dataset:
Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2023-2035, Florence, Italy. Association for Computational Linguistics.
Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. 2014. Stochastic backpropagation and approximate inference in deep generative models. Proceedings of ICML.
Sunita Sarawagi and William W Cohen. 2005. Semimarkov conditional random fields for information extraction. In L. K. Saul, Y. Weiss, and L. Bottou, editors, Advances in Neural Information Processing Systems 17, pages 1185-1192. MIT Press.
Sheng Shen, Daniel Fried, Jacob Andreas, and Dan Klein. 2019. Pragmatically informative text generation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4060-4067, Minneapolis, Minnesota. Association for Computational Linguistics.
Tianxiao Shen, Tao Lei, Regina Barzilay, and Tommi Jaakkola. 2017. Style transfer from non-parallel text by cross-alignment. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems 30, pages 6830-6841. Curran Associates, Inc.
Mitchell Stern, Jacob Andreas, and Dan Klein. 2017. A minimal span-based neural constituency parser. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 818-827, Vancouver, Canada. Association for Computational Linguistics.
Ramakrishna Vedantam, C. Lawrence Zitnick, and Devi Parikh. 2015. Cider: Consensus-based image description evaluation. In CVPR, pages 4566-4575. IEEE Computer Society.
Wenhui Wang and Baobao Chang. 2016. Graph-based dependency parsing with bidirectional LSTM. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2306-2315, Berlin, Germany. Association for Computational Linguistics.
Sam Wiseman, Stuart Shieber, and Alexander Rush. 2017. Challenges in data-to-document generation. Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing.
Sam Wiseman, Stuart Shieber, and Alexander Rush. 2018. Learning neural templates for text generation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3174-3187, Brussels, Belgium. Association for Computational Linguistics.
Kun Xu, Chongxuan Li, Jun Zhu, and Bo Zhang. 2019. Multi-objects generation with amortized structural regularization. arXiv preprint arXiv:1906.03923.
Pengcheng Yin, Chunting Zhou, Junxian He, and Graham Neubig. 2018. Structvae: Tree-structured latent variable models for semi-supervised semantic parsing. CoRR, abs/1806.07832.
Ruqing Zhang, Jiafeng Guo, Yixing Fan, Yanyan Lan, Jun Xu, and Xueqi Cheng. 2018. Learning to control the specificity in neural response generation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1108-1117, Melbourne, Australia. Association for Computational Linguistics.
The generative model is an LSTM with two layers with hidden dimension equals 500, input dimension equals 400, and dropout of 0.2. The inference network uses a one-layer Bi-LSTM with hidden size of 500 and input size of 400 to encode the sentence. We use large max segment length,  $L = 8$  (segmental for data-to-text) and  $L = 1$  (linear chain for POS induction) and 0.2 dropout in the inference network. The Bi-LSTM used for encoding the source table is has hidden dimension of 300. Both the generative model and the inference network share word embeddings.
The batch size is 10 for WikiBio and 20 for PTB and E2E. The generative model and the inference network are optimized by Adam (Kingma and Ba, 2014) gradient clipping at 1, with learning rate of 0.002 and 0.001 respectively. Parameters are all initialized from a standard Gaussian distribution. The learning rate decays by a factor of two for any epoch without improvement of loss function on validation set, and this decay condition is not triggered until the eighth epoch for sufficient training. Training is done for max of 30 epochs and allows for early stopping.
For data-to-text problem, we need to encode the data table. We encode the E2E source table by directly concatenating word embeddings and field embeddings and indices for each token, for example, if the word  $w$  is the  $i$ th token from left and  $j$ th token from right under field type  $f$ , then we represent the token using a concatenation  $[\mathrm{emb}(w) \cdot \mathrm{emb}(f) \cdot \mathrm{emb}(i) \cdot \mathrm{emb}(j)]$ . We encode the WikiBio table by passing a bidirectional-LSTM through the tokens in the table, where each token has similar embedding by concatenation as above. The encoding of the table is denoted as  $c$ . We use copy attention (Gu et al., 2016; Gulcehre et al., 2016) in the generative model, and the attention vector  $\alpha$  at a time step is parametrized by the class label  $z$  at that time step. Recall the contextual representation is  $\sum_{i} \alpha_{i} \cdot c_{i}$ , where  $\alpha_{i} = \text{softmax}(\text{score}(h_{t}, c_{i}))$  and  $\text{score}(h_{t}, c_{i}) = (W_{z}(h_{t}) + b_{z}) \cdot (W_{2}(c_{i}) + b_{2})$ , the parametrization from  $z$  happens during the feedforward network indexed by  $z$ . For the WikiBio data, we use a dual attention mechanism described in (Liu et al., 2018), where the first attention is the same as above and the second attention uses a different encoder context  $c_{i}'$ , the  $c_{i}'$  only looks at the concatenation of field type and field index, but not the field value
itself, i.e.  $[\mathrm{emb}(f)\cdot \mathrm{emb}(i)\cdot \mathrm{emb}(j)]$ . Then the two attention forms two different sets of  $\alpha_{i}$  and they are multiplied together and renormalized to form an attention.