SlowGuess/ACL_2020_OCR huggingface dataset:
Tomáš Mikolov, Martin Karafiát, Lukáš Burget, Jan Černocký, and Sanjeev Khudanpur. 2010. Recurrent neural network based language model. In Eleventh annual conference of the international speech communication association.
Ishan Misra, C. Zitnick, Margaret Mitchell, and Ross Girshick. 2016. Seeing through the human reporting bias: Visual classifiers from noisy human-centric labels. pages 2930-2939.
Will Monroe and Christopher Potts. 2015. Learning in the rational speech acts model. arXiv preprint arXiv:1510.06807.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI Blog, 1(8).
Ehud Reiter and Robert Dale. 1997. Building applied natural language generation systems. Natural Language Engineering, 3(1):57-87.
Jost Schatzmann, Karl Weilhammer, Matt Stuttle, and Steve Young. 2006. A survey of statistical user simulation techniques for reinforcement-learning of dialogue management strategies. The knowledge engineering review, 21(2):97-126.
Pararth Shah, Dilek Hakkani-Tür, Bing Liu, and Gokhan Tur. 2008. Bootstrapping a neural conversational agent with dialogue self-play, crowdsourcing and on-line reinforcement learning. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 3 (Industry Papers), pages 41-51.
Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks. In Advances in neural information processing systems, pages 3104-3112.
Bart Thomee, David A. Shamma, Gerald Friedland, Benjamin Elizalde, Karl Ni, Douglas Poland, Damian Borth, and Li-Jia Li. 2016. Yfcc100m: the new data in multimedia research. Commun. ACM, 59(2):64-73.
Ramakrishna Vedantam, Samy Bengio, Kevin Murphy, Devi Parikh, and Gal Chechik. 2017. Context-aware captions from context-agnostic supervision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 251-260.
Oriol Vinyals and Quoc Le. 2015. A neural conversational model. arXiv preprint arXiv:1506.05869.
Ronald J Williams. 1992. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning, 8(3-4):229-256.
Ludwig Wittgenstein. 1953. Philosophical investigations.
Daniel M Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B Brown, Alec Radford, Dario Amodei, Paul Christiano, and Geoffrey Irving. 2019. Fine-tuning language models from human preferences. arXiv preprint arXiv:1909.08593.
C Lawrence Zitnick and Devi Parikh. 2013. Bringing semantics into focus using visual abstraction. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3009-3016.
7673
A Hyperparameters
The following tables represent our choice of hyperparameters in the speaker and listener agents. Hyperparameters in Table 7 where chosen in the image captioning task using the validation set. Hyperparameters in Table 8 where chosen in the referential task using the validation set.
Agent	Hyperparameter	Value
listener	LSTM hidden size	512
speaker	LSTM hidden size	512
listener	visual embeddings size	512
speaker	visual embeddings size	1024
Table 7: Settings shared across all experiments.
Model	Hyperparameter	Value
fine-tuning++	KL regularization	0.1
multi-task	structural weight	0.1
PoE	structural weight	0
noisy channel / PoE	number of samples	20
noisy channel / PoE	message embedding size	1024
noisy channel / PoE	entropy regularization	0.1
Table 8: Settings for particular speakers.
B ResNet module
We use ResNet-50 (He et al., 2016) pre-trained on ImageNet. For image captioning and also for models that use the pre-trained captioning model (i.e. reward finetuning, PoE and noisy channel) we backpropagate gradients into the ResNet module. However, in all rerankers we freeze the ResNet during reward optimization. Moreover, we also keep the ResNet fixed in the jointly learned listener to prevent additional drift, however we back-propagate when we pre-train the fixed listener, grounded though the discriminative caption speaker.
7674