Hallucination Detection for Grounded Instruction Generation
Authors: Lingjun Zhao
Khanh Xuan Nguyen
Hal Daumé III
Venue: EMNLP 2023 Findings (2023)
Keywords: Hallucination detection
multimodality
natural language generation

Abstract: We investigate the problem of generating instructions to guide humans to navigate in simulated residential environments. A major issue with current models is hallucination: they generate references to actions or objects that are inconsistent with what a human follower would perform or encounter along the described path. We develop a model that detects these hallucinated references by adopting a model pre-trained on a large corpus of image-text pairs, and fine-tuning it with a contrastive loss that separates correct instructions from instructions containing synthesized hallucinations. Our final model outperforms several baselines, including using word probability estimated by the instruction-generation model, and supervised models based on LSTM and Transformer.

Hallucination Detection for Grounded Instruction Generation
Lingjun Zhao and + Khanh Nguyen and o Hal Daumé III
University of Maryland, College Park University of California, Berkeley Microsoft Research
1zhao 23@umd. edu
Abstract
reduces the necessary supervision in two ways. First, we leverage pre-trained vision-language model (Guhur et al., 2021) that has Iearned transferable representations of path-instruction pairs through self-supervised learning. Second, we introduce data-augmentation strategies to create synthetic data with "free" hallucination labels. We fine-tune the pre-trained model with the synthesized data using contrastive learning objective learn representations that separate positive examples (hallucinationss) from negative examples (non-balluciiaations) Our model outperforms various baselines in terms of F-1 scores on human-annotated evaluation data, beating an LSTM- and a Transformer-base models by 6.2 and 10.0 points, respectively, Ablation studies demonstrate the effectiveness of the proposed self-supervised pre-training and contrastive fine-tuning approach. We release the code, models, and data at ht github.
We investigate the problem of generating in- structions to guide humans to navigate in simu- lated residential environments. A major issue with current models is hallucination: they gen- erate references to actions or objects that are inconsistent with what a human follower would perform or encounter along the described path. We develop model that detects these hallu- cinated references by adopting a model pre- trained on large corpus image-text pairs, and
it with contrastive loss that separates correct instructions from instructions containing synthesized hallucinations. Our nal model outperforms several baselines. in- cluding using word probability estimated by the instructtinnggeneaatoo model, and supervised models based on LSTM and Transformer.
1 Introduction
Performance of neural-network-brrre based models generating navigation instructions is substantially inferior that of humans (Zhao et al., 2023). These models often hallucinate, generating references to objects or actions that do not exist or are impossi- ble to execute in the environment. Similar behavior has been observed in language models in other domains of text generation (Raunak et al., 2021; Ji et al., 2023; Xiao and Wang. 2021; Lee et al.. 2018; Guerreiro al., 2022; Rawte et al., 2023). Instructions containing hallucinations can con- fuse or misdirect humans, leading to frustration and sometimes even catastrophic mistakes. Detect- ing hallucinations is therefore essential to improve instruction generation models and inform risk to human users. Nevertheless, ground-truth word- level hallucination labels are typically not readily available in this domain. Mcanwhile, hiring crowd- workers annotate instructions can be very costly (Anderson al., 2018b; He al., 2021: Wang et al., 2022; Gao et al., 2022).
html.
Related Work
Hallucination detection. Neural sequence to se- quence models are prone generate hallucinations, where the outputs are inconsistent with the inputs or the environments (Müller al., 2019; Maynez et al., 2020; Wiseman al., 2017: Martindale al., 2019; Durmus et al., 2020; Ji et al., 2023). Recent work largely focuses on text-only domains (Wang and Sennrich, 2020: Zhou et al., 2020; Chen et al., 2021; Dale et al., 2022; Xu et al., 2023; Nie et al., 2019; Falke et al., 2019; Kryšciński et al., 2019; Rebuffel et al., 2022; Liu al., 2021; van der Poel al., 2022) and image captioning (Rohrbach al., 2018: Dai et al., 2022: Biten al., 2022: Li et al., 2023; Gunjal al., 2023). To the best of our knowl- edge, our work is the first study of hallucination grounded instruction generation.
We propose a data-cfficient weakly supervised approach to hallucination detection. Our approach

---

Grounded Instruction Generation. Instruction generation has been commonly studied in naviga- tion settings (Anderson et al., 1991; Byron al., 2010; Koller et al., 2010; Striegmitz et al., 2011; Goeddel and Olson, 2012; Fried et al., 2017, 2018; Wang et al., 2022; Kamath et al., 2022). Recent work by Zhao et al. (2023) reveals a significant gap between the performance of models and humans. Our work constructs model that can be useful for evaluating and enhancing instruction -eeerratonnnn models. Huang et al. (2019) and Zhao et al. (2021) train LSTM-based discriminative models with contrastive learning to score instructions. We follow a similar approach but focus on identifying werd-level hallucinations, and effectively leverage a large pre-trained Transformer model.
be replaced because it inaccurately describes an observation or action. For example, an in-
struction says "Walk past the reception desk and out the door on the right," " but in the de- scribed trajectory, the door is on the left:
Extrinsic hallucination is word that needs to be removed because it has no correspondence in the input trajectory. Our model typically hibits this type of hallucination by repeatedly generating the same sentence, e.g.. "Walk out of the office. Walk into the hallway and turn left. Walk into the hallway and turn left."
We formulate hallucination detection as binary classification: given an input I (r, u, i) consist- ing of trajectory i E {1. |u|), decide whether the word uy is hallucination, i.e. whether it should be replaced removed make consistent with
instruction u, and an index
3 Problem Setting
Grounded instruction generation. Our task takes place in an environment, where a speaker model S(u r) composes an instruction u to com- municate an imaginary trajectory r to follower so that the latter can generate the same trajectory in the environment, An instruction is sequence of words t4, whereas trajectory is sequence of observations O₂ and actions az. We employ the Matterport3D simulator for experiments (Ander- son et al., 2018b) which embeds follower in a 3D model real-world residential building. The ob- servation Of of the follower comprises of an RGB image representing the panoramic view at loca- tion in a building, and orientation features encoding the follower's gaze direction. Each action a, moves the follower new location close where is standing and changes its observation.
Candidate selection. For each instruction, we identify set of candidate words for classification. which (a) directional words like left, right, etc. (see AA.2 for full list) well (b) nouns identi- fied by the SpaCy part-of- tagger (Honnibal and Montani, 2017).
4 Hallucination Detection Model
4.1 Architecture
We learn classifier Cly 1 to decide whether word u, is hallucinated. Our model is based on the Airbert model (Guhur al., 2021), which inherits the ViLBERT architecture (Lu et al., 2019). An overview of the model is given in Figure one encodes the instruction u and the other encodes the trajectory We wrap the word to be classified us between pair of special tokens ([BH] and [EH]). Let hunn be the output of the language-encoding Transformer, and hvision be that of the vision-encoding Transformer. The model computes score function s(x) s(r, u, i) w (hrang @ hivision,, where w a learnable vector. and @ denotes element-wise multiplication. More details about the model are given in $A.1.
x= (r, i))
implements two Transformers:
Speaker model. We follow Zhao et al. (2023) to train TS-based (Raffel et al., 2020) speaker model. This model encodes a trajectory into sequence of hidden vectors and applies multi-header attention on those vectors to generate an instruction auto- regressively. It is trained on the Room-to-Room (R2R) dataset provided by the Matterport3D simu- lator. Detail about the model is provided in $A.1.
Hallucination in grounded instruction. Instruc- tions generated by our speaker model often contain words that are inconsistent with the input trajectory. We refer to those words as hallucinations. Similar to prior work (Zhou et al., 2020), observe two types of hallucinations:
4.2 Learning approach
Self-supervised pre-training. Instead of learn- ing from scratch, we fine-tune a pre-trained check- point of the Airbert model. The checkpoint was first trained on a large collection of 1.4M images
Intrinsic hallucination is a word that needs to

---

[CLS] Walk [BH] down [EH] flight of stairs stop the landing.
Positive:
Embeddings
Transformer
Cross-modal
Similar
{
[IMG]
[MMG]- [IMG]
Embeddings
Transformer
Cross-modal attention
Dissimilar
[CLS] [BH] up [EH] one flight of and stop the landing
Negative:
Embeddings
Transformer
Figure I: Our hallucination detection model, which takes input instruction with target word and determines whether it should be replaced or removed to be consistent with visual trajectory. To build this model, we fine-tune pre-trained Airbert (Guhur et al., 2021) with a contrastive learning objective.
and 0.7M captions collected from AirBnB. It was subsequently adapted for a trajectory-instructio compatibility estimation task using the Room-to- Room dataset. The objective in each phase com- bines stRT-style pre-training (mask and pair pre- diction) with contrastive learning. We refer the readers to the original paper for an elaborate de- scription of the pre-training phase.
ing set. Thus, we propose a more cost-effective methodology for generating training data.
Synthetic negative examples. We start with a training example (u+,r) in the Room-to-Room training set and modify the human-writter instruc- tion u to create instructions with hallucinations. We first extract the candidate words in the instruc- tion ($3). To create an intrinsic hallucination, we choose candidate word and apply the following procedure:
Contrastive fine-tuning. We assume dataset of contrastive pairs (=*, The positive and neg- ative examples of pair have the same trajectory
If the word is direction, we replace it with alternative direction. E.g., "Walk downup one flight of stairs and stop on the landing.": If it is room, we substitute it with another room randomly selected from a pre-composed list. E.g., "Exil the bedreom balcony via the farthest left. Walk toward the couch. Stop there.";
r and word index i, but differ in the instruction hallucination, whereas that in æt not. For each pair, we com- pute the model scores s(x+) and )) and con- struct the softmax distribution p = Softmax(s) where
u. The classified word x
(((*)),((=)) We then train the model to recognize the positive example by min- imizing the cross entropy between P and p* O). This objective effectively forces the repre- sentation of the trajectory to be similar to that of the positive instruction and dissimilar to that of the negative instruction. At inference time, we define the hallucination detection classifier as C(x) o (s(x:)) where a is the sigmoid function.
Otherwise, we swap it for another word in the instruction that is neither direction nor room. E.g., "Exit the bedroom using the door step on the left then go straight until you get the stairs and wait on the second step door." Using this procedure, we first generate an intrin- sic hallucination in u+ to synthesize u Then, with probability of 0.5,
synthesize another intrinsic hallucination in each of u+ and u- This step makes the training instructions more similar to the test-time inputs, which may contain multiple intrinsic hallucinations as they are generated by imperfect speaker models.
4.3 Synthesizing data creation
Even for fine-tuning, acquiring human-labeled data can be prohibitively expensive. For evaluation, we manually annotated a small sample of labels ($5).
The annotation process laborious, with erage time of 30 minutes required annotate just 10 instructions. Based on our calculations, with compensation of 15 USD per hour, would cost approximately 9,000 USD to hire crowd workers to annotate all instances (~~12, 12,000) in the R2R train-
To create an instruction with extrinsic hallu- cinations, we append sentence, taken from u- or another instruction, to the end of random sentence in ut For example: "Walk out of the office. Walk into the hallway and uum left. Walk

---

Model
F-1
Precision
Recall
Random
16.6
13.4
21.7
Speaker model probability
29.5
20.9
50.0
LSTM-based encoder-decoder
38.7
37.4
40.2
T5-small (Transformer-hased oncoder-decoder)
33.9
26.5
46.7
T5-base (Transformer-basee encoder-decoder)
34.9
25.2
56.5
Fine-tuned Airbert (ours)
44.9
42.3
47.8
Table I: Performance the of proposed hallucination detection model and various baselines. The decision threshold of each model is selected to maximize F-1 score of hallucination labels on the development set.
ing environments. We use the speaker model generate instructions from these trajectories. The first two authors then manually annotate word-level hallucinations, creating 209 development examples and 632 test examples. The final labels are de- cided by mutual agreement. We choose the deci- sion threshold of model to maximize its F-1 score on the development set.
.49
Final Model
Pre-training
MLE Fine-tuning
The effectiveness of self-supervised pre- training and contrastive fine-tuning. Results are F-1 scores of hallucination labels the test set.
Figure 2:
(i) random classifier assigns label chosen uniformly random, (ii) speaker model probability defines the hallucination probability C(x) = S(u, r;; u,z;) where x = (r. u, i), S the speaker model ($ 3), and ucz is the instruction generated up to step - 1 for the input r; (iii) LSTM and (iv) TS are binary classifiers learned under standard maximum-likelihode objective. They implement encoder-decoder architecture based on LSTM and Transformer, respectively, and are trained using the same synthetic dataset as our proposed model. These models are initialized with random parameters. The detailed implementations and hyperparameters of all models are given in $A.1.
Baselines.
into the hallway and turn lef."" Every word in the added sentence considered an extrinsic hallucination. We do not create additional intrinsic hallucinations in the instruction.
Alleviating input-distribution shift. Model trained only on human-written instruction may perform poorly on model-generated instructions. Therefore, we also include "high-quality" model- generated instructions the R2R training set as positive examples and apply the same strategies generate negative examples. The quality of an instruction is measured by the success rate of an ensemble VLN O BERT instruction-folloiing agents (Hong et al., 2021) in recreating the de- scribed trajectory. We consider model-generated instruction to be of high quality if at least 80% of the ensemble agents can successfully reach the final location in the deseribed trajectory.
Main results (Table 1). The speaker-model- probability is remarkably strong baseline, despite not trained for hallucination detection. Its performance is on par with that of T5, which is the same model but trained specifically for hallucination detection. The LSTM-based model outperforms the basba models. Scaling up the size of the T5 model improves the recall score by 10 points. Our proposed model (fine-tuned Airbert) beats all baselines by wide margins in terms of F-1 score for hallucination labels. (+10.0 versus T5-base, +6.2 versus LSTM). It excels in precision compared to the baselines. We also include results on the development set in AA.3.
5 Experiments
Data. Following the procedure described in $4.3, we generate a training set of 325,346 contrastive pairs. For evaluation, uation trajectories in (Zhao al., 2023) form the set. We randomly select another set of 20 trajectories the R2R validation seen set for devel- opment. The environments in which the evaluation trajectories are generated are subset of the train-
use the same 75 eval-

---

highlight:
up the steps and turn right, Walk the steps right
Model highlight: exit the bedroom Gold highlight:: Walk past bed and exit the bedroom
Gold highlight:: Walk up the steps and right Walk up the steps and turn right
Success detecting extrinsic hallucination: the second should be removed entirely; the model marks all the candidate words the sentence.
(b) Success detecting intrinsic hallucination: the correct dircction go the left side of the bedroom, exiting
Model highlight: Walk hallway stop in the first doonway your
highlight:
past the couch and stop
Gold highllghtt: Walk past the couch and stop
Gold highlight: Walk down the hallwaky and first
your left
(c) Model misidentifies the stopping location due to lacking depth information: the TV in the far left corner looks to be close to the true stopping location.
(d) Ambiguous direction: a slight left turn that appears like a straight walk in this viewpoint.
Figure 3: Some successful and failure cases of the fine tne-tuned Airbert model. The blue arrow indicates the described path, and the green represents the next location.
Ablation studies (Figure 2). Our results confirm that self-supervised pre-training and contrastive fine-tuning are requisite to the performance of our model. Without pre-training, our model is just as bad as the LSTM-based model. We also compare fine-tuning via contrastive learning with fine-tuning via maximum-likelihon learning. In the latter ap- proach, the model simply takes as input an example (r, u. i) and learns directly predict the true label. The approach underperforms contrastive learning by 4.9 F-1 points. Our finding aligns with previous work (Gunel al., 2021; Zhang al., 2021; Goyal et al., 2023), suggesting that contrastive learning is effective not only as a representation learning objective, but also as classification objective.
Type
F1
Precision
Recall
Direction
48.1
41.9
56.4
Room
38.9
38.9
38.9
Object 38.7
50.0
31.6
Table 2: Fine-tuned Airbert performance broken down by word type. Results are on test set.
pre-training multimodal data and contrastive fine-tuning synthetic data promising scal- able approaches. We hope that these directions can be further developed in future work.
Limitations
Error and Qualitative Analysis. In Table 2, break down the performance of our model by word type. Our model struggles with detecting room and object hallucinations, indicating that its un- derstanding of visually grounded words lacking. Especially, it has relatively low recall on object hal- lucinations, potentially due to lack of diversity of this word type in the training data. Figure shows few successful and failure examples of our model.
Despite the effectiveness of the data generation method, this approach requires substantial domain- specific knowledge. Our method, particularly to generate directional hallucinations, is based on heuristics and does not take into account the ac- tual environment. Another limitation is the small size of the evaluation datasets due to the expensive cost of annotation.
Acknowledgments
6 Conclusion
We thank the CLIP Laboratory Maryland and our reviewers for providing helpful feedback improve the manuscript.
This work an early attempt to address the hallu- cination issue in grounded instruction generation. We have shown that techniques like self-supervised

---

References
Tobias Falke, Leonardo FR Ribeiro, Prasetya Ajie Utama, Ido Dagan, and Iyyna Gurevych. 2019. Rank- ing generated summaries by correctness: An interest- ing but challenging application for natural language inference. In Proceedings of the 57th Annual Meet- ing of the Association for Computational Linguistics, pages 2214-2220.
Anne H Anderson, Miles Bader, Ellen Gurman Bard, Elizabeth Boyle, Gwyneth Doherty, Simon Garrod, Stephen Isard, Jacqueline Kowtko, Jan McAllister, Jim Miller, et al. 1991. The here map task corpus. Language and speech, 34(4):351-36.
Peter Anderson, Xiaodong He, Chris Buehler, Damien Teney, Mark Johnson, Stephen Gould, and Lei 7hang. 2018a. Bottom-up up and top-down attention for image captioning and visual question answering. In Pro- ceedings of the IEEE conference on computer vision and pattern recognition, pages 6077-6086.
Daniel Fried, Jacob Andreas, and Dan Klein, 2017. Uni- fied pragmatic models for generating and following instructions. arXiv preprint arXiv:
Daniel Fried, Ronghang Hu, Volkan Cirik, Anna Rohrbach, Jacob Andreas, Louis-Philippe Morency, Taylor Berg- Kirkpatrick, Kate Saenko, Dan Klein, and Trevor Darrell. 2018. Speaker-follower mod- els for vision-anddlanguage navigation. Advances in Neural Information Processing Systems, 31.
Peter Anderson, Qi Wu, Damien Teney, Jake Bruce, Mark Johnson, Niko Sünderhauf, Ian Reid, Stephen Gould, and Anton Van Den Hengel. 2018h, 8b. Vision- and-language navigation: Interpreting visually- grounded navigation instructions in real environ- ments, In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3674- 3683.
Xiaofeng Gao, Qiaozi Gao, Ran Gong, Kaixiang Lin, Govind Thattai, and Gaurav S Sukhatme. 2022. Di- alfred: Dialogue-enabled agents for embodied in- struction following, IEEE Robotics and Automation Letters, 1(4):10099-1005.
Ali Furkan Biten, Lluis Gómez, and Dimosthenis Karatzas. 2022. Let there be a clock on the beach: Re- ducing object hallucination in image captioning. In Proceedings of the IEEE/CVF Winter Conference on Applications Computer Vision, pages 1381-1390.
Robert Goeddel and Edwin Olson. 2012. Dart: A particle-hased method for generating easy-to-follow directions. In 2012 IEEE/RS! International Confer- ence on Intelligent Robots and Systems, pages 1213- 1219. IEEE
Donna Byron, Alexander Koller, Kristina Striegnitz, Jus- tine Cassell, Robert Dale, Johanna D Moore, and Jon Oberlander. 2010. Report on the first nlg challenge on generating instructions in virtual environments (give).
Sachin Goyal, Ananya Kumar, Sankalp Garg, Zico Kolter, and Aditi Raghunathan. 2023. Finetune like you pretrain: Improved finetuning of zero-shot vision models. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Puttern Recognition, pages 19338-19347.
Angel Chang, Angela Dai, Thomas Funkhouser, Maciej Halber, Matthias Niessner, Manolis Savva, Shuran Song, Andy Zeng, and Yinda Zhang. 2017. Matter- port3D: Learning from RGB-D data in indoor envi- ronments. International Conference on 3D Vision (3DV).
Nuno M Guerreiro, Elena Voita, and André FT Martins. 2022. Looking for a needle in haystack: A com- prehensive study of hallucinations in neural machine translation. arXiv preprint arXiv:2208.0528
Pierre-Louis Guhur, Makarand Tapaswi, Shizhe Chen, Ivan Laptev, and Cordelia Schmid. 2021. Airbert: In- domain pretraining for vision- aan-language naviga- tion. In Proceedings of the IEEF/CVV International Conference on Computer Vision, pages 1634-1643.
Sihao Chen, Fan Zhang, Kazoo Sone, and Dan Roth. 2021. Improving faithfulness in abstractive sum- marization with contrast candidate generation and selection. arXiv preprint arXiv:2704.0990
Wenliang Dai, Zihan Liu, Ziwei Ji, Dan Su, and Pascale Fung. 2022. Plausible may not be faithful: Probing object hallucination in laniunnl pre-training. arXiv preprint 1r.06222.0.06688
Beliz Gunel, Jingfei Du, Alexis Conneau, and Ves Stoy- anov. 2021. Supervised contrastive learning for pre- trained language model fine-tuning, In Proceedings of the International Conference on Learning Repre- sentations.
David Dale, Elena Voita, Loic Barrault, and Marta Costa-jussà. 2022. Detecting and mitigating halluci- nations in machine translation: Model internal work- ings alone do well, sentence similarity even better. arXiv preprint arXiv:2212.08597.
Anisha Gunjal. Jihan Yin, and Erhan Bas. 2023. De- tecting and preventing hallucinations in large vision language models. arXiv preprint 1rXii:2308.06388
Keji He, Yan Huang, Qi Wu, Jianhua Yang. Dong An. Shuanglin Sima, and Liang Wang. 2021. Landmark- exr: Solving vision-and lad-uuuu navigation with fine-grained alignment supervision. Advances in Neu- ral Information Processing Systems, 34:652-663.
Esin Durmus, He He, and Mona Diah. 2020. Feqa: A question answering evaluation framework for faithful- ness assessment abstractive summarization. arXiv preprint arXiv:2005.037.

---

Yicong Hong, Wu, Yuankai Qi, Cristian Rodriguez- Opazo, and Stephen Gould. 2021. VIn bert: A recur- rent vision-and-language bert for navigation. In Pro- ceedings of the IEEE/CVF conference on Computer Vision and Paltern Recognition, pages 1643-1653.
Marianna Martindale, Marine Carpuat, Kevin Duh, and Paul McNamee. 2019. Identifying fluently inade- quate oulput in neural and statistical machine transla- tion. In Proceedings of Machine Translation Summit XVII: Research Track, pages 233-243.
Matthew Honnibal and Ines Montani. 2017. spaCy 2: Natural language understanding with Bloom embed- dings, convolutional neural networks and incremental parsing. To appear.
Joshua Maynez, Shashi Narayan, Bernd Bohnet, and Ryan McDonald. 2020. On faithfulness and factu- ality in abstractive summarization. arXiv preprint arXiv:2005.000 00661.
Haoshuo Huang, Vihan Jain, Harsh Mehta, Jason Baldridge, and Eugene le. 2019. Multi-modal dis- criminative model for aiiion-and-lnguuage naviga- tion. arXiv preprint arXiv: /905. 13358.
Mathias Müller, Annette Rios, and Rico Sennrich. 2019. Domain robustness in neural machine translation. arXiv preprint 1911::991..03109.
Feng Nie, Jin-Ge Yao, Jinpeng Wang, Rong Pan, and Chin- Yew Lin. 2019. simple recipe towards re- ducing hallucination in neural surface realisation. In Proceedings of the 57th Annual Meeting of the Asso- ciation for Computational Linguistics, pages 2673- 2679.
Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023. Survey of halluci- nation natural language generation. ACM Comput- ing Surveys. 55(12):
Aishwarya Kamath, Peter Anderson, Wang, Jing Yu Koh, Alexander Ku, Austin Waters, Yinfci Yang. Ja- Baldridge, and Zarana Parckh. 2022. A new path: Scaling vision-an anda-and--aa lan-uan navigation with synthetic instructions and imitation learning. arXiv preprint arxiv: 100.03112.
OpenAL. 2023. Gpt-4 technical report.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sas- try, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In International confer- ence on machine learning, pages 8748-8763. PMLR.
Alexander Koller, Kristina Striegnitz, Andrew Gar- gett, Donna Byron, Justine Cassell, Robert Dale, Jo- hanna D Moore, and Jon Oberlander. 2010. Report on the second nlg challenge on generating instruc- tions in virtual environments (2i In Proceed- ings of the 6th international natural language gen- eration conference. The Association for Computer Linguistics.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the limits of transfer learning with unified text-to-text trans- former. The Journal of Machine Learning Research, 21(1):5485-551.
Wojciech Kryšciñski. Bryan McCann, Caiming Xiong, and Richard Socher. 2019. Evaluating the factual consistency of abstractive text summarization. arXiv preprint arXiv: 12840.
Vikas Raunak, Arul Menezes, and Marcin Junczys- Dowmunt. 2021. The curious case of hallucina- tions in neural machine translation. arXiv preprint arXiv:2104.0664
Katherine Lee, Orhan Firat, Ashish Agarwal, Clara Fan- njiang, and David Sussillo. 2018. Hallucinations in neural machine translation. In Interpretability and Robustness in Audio, Speech, and Language Work- shop (NeurIPS).
Vipula Rawte, Amit Sheth, and Amitava Das. 2023. A survey of hallucination in large foundation models. arXiv preprint arXiv:2309.0522
Clément Rebuffel, Marco Roberti, Laure Soulier, Geof- frey Scoutheeten, Rossella Cancellicre, and Patrick Gallinari. 2022. Controlling hallucinations at word level in data-to-text generation. Data Mining and Knowledge Discovery, pages 1-37.
Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji-Rong Wen. 2023. Eval- uating object ballucination in large vision-language models. arXiv preprini 10X
Tianyu Liu, Yizhe Zhang, Chris Brockett, Yi Mao, Zhifang Sui, Weizhu Chen, and Bill Dolan. 2021. token-level reference-fiee free hallucination detection benchmark for free-form text generation. arXiv preprint 2rXiv:2104.088
Anna Rohrbach, Lisa Anne Hendricks, Kaylee Burns, Trevor Darrell, and Kate Saenko. 2018. Object hallucination in image captioning. arXiv preprint 1rXiv:1889.027556
Sheng Shen, Liunian Harold Li. Hao Tan, Mohit Bansal, Anna Rohrbach, Kai-Wei Chang, Zhewei Yao, and Kurt Keutzer. 2021. How much can clip benefit vision-and lan-aaauagge tasks? arXiv preprint 11Xiv:2107.06383.
Jiasen Lu, Dhruv Batra, Devi Parikh, and Stefan Lee. 2019. Vilbert: Pretraining task-agnostic visiolinguis- tie representations assoo-and-llaggun tasks. Ad- vances in neural information processing systems, 32.

---

Kristina Striegnitz, Alexandre Denis, Andrew Gar- gett, Konstantina Garoufi, Alexander Koller, and Mar- iët Theune. 2011. Report on the second second chal- lenge generating instructions virtual environ- ments (give-2.5). In 13th European workshop on natural language generation.
Liam van der Poel, Ryan Cotterell, and Clara Meis- ter. 2022. Mutual information alleviates hallucina- tions in abstractive summarization. arXiv preprint 0rXiv:2210.132100
Chaojun Wang and Rico Sennrich. 2020. On exposure bias, hallucination and domain shift in neural ma- chine translation. arXiv preprint arXiv:2005003642
Su Wang, Ceslee Montgomery, Jordi Orbay, Vighnesh Birodkar, Aleksandra Faust, Izzeddin Gur, Natasha Jaques, Austin Waters, Jason Baldridge, and Peter Anderson. 2022. Less is more: Generating grounded navigation instructions from landmarks. In Proceed- ings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 15428-15438.
Sam Wiseman, Stuart M Shieber, and Alexander M Rush. 2017. Challenges in data-to-document genera- tion. arXiv preprint arXiv: 1707.08052
Yijun Xiao William Yang Wang. 2021. On halluci- nation and predictive uncertainty in conditional lan- guage generation. arXiv preprint arXiv:2103.15022
Weijia Xu, Sweta Agrawal, Eleftheria Briakou, Mari- amna Martindale, and Marine Carpuat. 2023. Un- derstanding and detecting hallucinations in neural machine translation via model introspection. arXiv preprint arXiv:2301.077
Jianguo Zhang, Trung Bui, Seunghyun Yoon, Xiang Chen, Zhiwei Liu, Congying Xia, Quan Hung Tran, Walter Chang, and Philip Yu. 2021. Few-shot intent detection via contrastive pre- pre-training and fine-tuning In Proceedings of Empirical Methods in Natural Lan- guage Processing.
Lingjun Zhao, Khanh Nguyen, and Hal Daumé III. 2023. Define, evaluate, and improve task-oriented cognitive capabilities for instruction generation models. In Findings of ACL.
Ming Zhao, Peter Anderson, Vihan Jain, Su Wang. Alexander Ku, Jason Baldridge, and Eugene le. 2021. On the evaluation of ssioo-and-languuge navigation instructions. arXiv preprint arXiv:2101.100 10504.
Chunting Zhou, Graham Neubig. Jiatao Gu, Mona Diab, Paco Guzman, Luke Zettlemoyer, and Marjan Ghazvininejad. 2020. Detecting hallucinated con- tent conditional neural sequence generation. arXiv preprint arX.v:2011.02593.

---

A Appendices
Let Manng be the output the position of the [CLS] token of the language-encoding Trans- former, and Avision the output the position of the first [IMG] token of the vision-encoding Trans- former. The score function s(x) is defined as:
A.1 Models
Speaker. The speaker model takes as input a tra- jectory and computes a distribution over instruc- tions. To encode a trajectory r, following prior work (Shen et al., 2021; Zhao et al., 2023), we convert each panoramic observation On into a col- lection of 36 images that represent the first-person views obtained from 36 gaze directions. We feed these into pre-trained vision model (Radford et al., 2021) to obtain a of view vectors { For each action at, which corresponds to an adjacent location, let E {I. .,6,,, 36) the direction to- wards that location and let = ( ((er) be the horizomtal and vertical angles of that direction. We represent 0, by concatenating the visual features of with the directional features [cos 0, sin 0). The sequence of observation and action representations is fed into a Transformer encoder to produce se- quence of hidden vectors. A transformer decoder then applies multi-headed attention those vectors and generates an instruction u regreseygess.vvly.
s(x) s(r, u. w (hiang @ hvision)
where is a learnable vector, and C denotes element-wise multiplication.
T5. This model is the same as the speaker model. However, instead of generating an instruction, it computes a score s(x) like the Airbert model. The input æ is also tuple (r, u, 2). The instruction u has the same format in the case of the Airbert model, with the word to be classified surrounded by
two special tokens. Let (hr))= be the sequence of hidden vectors obtain after decoding the input instruction. We compute the mean vector h | hy. The score is computed as s(x:) w h, where w a learnable vector.
LSTM. This model is similar to the T5 model except that the encoder and decoder are LSTMs.
Airbert model. The input of the classifier is a trajectory r and an instruction u. The instruction has the following format:
Hyperparamters and Computation. The hyper- parameters and computation cost of all models are listed in Table 3.
[ [CLS], .1...., [BH], ue [EH]. ..., u [SEP]
A.2 Word replacement
where the word to be classified u, is enclosed by special tokens [BH] and [EH]. and the [CLS] and [SEP] tokens mark the beginning and the end. Each token are replaced by a sum token em- bedding and a positional embedding. We pass this sequence embeddings into a Transformer.
We compiled list of direction words and divided them into groups (Table 4). When constructing a negative example, if word is selected, a replace- ment randomly selected among the remaining words in the same group.
Our compiled list of rooms to generate synthetic examples are: "laundry room", "mudroom" "family room", "balcony", "utility room". "tool room", "entryway", "foyer" " "lobby", "library", "bathroom", "bar", " "spa", "sauna", "living room", "other room", "staircase", "garage", "hallway", "office", "classroom", "outdoor areas", "meeting room", "conference room" "dining room", "lounge", "bedroom" "porch", "terrace", "deck" "driveway", "kitchen", "toilet", "workout room" "exercise room", "gym", "tv room" "recreation room "closet", "junk", "study", "guest room", "music room", "home theater", "sunroom", "conservatory", "playroom", "pantry",
For the trajectory, the model extracts from panoramic view O: set of image regions Uhi J==1 and represents the sequence of obser- vations as:
{o
(1) [ [[IMG], 01 ...... [K) [IMG], 02 , [[MG]] of
(K)
(K)
o
where [IMG] the embedding of an observation- separating token. Each image region of") is con- verted into visual embedding, which is addi- tion of three embeddings: visual embedding (com- puted by Faster R-CNN model (Anderson al.. 2018 (018,)), directional embedding, and region-index embedding. We feed the sequence visual embed- dings into a second Transformer.
"game room"

---

Hyperparameterss
Fine-tuned Airbert
LSTM
Speaker Model
Learning rate
10-6
10
10
10
Batch
128
64
32
Optimizer
AdamW
AdamW
Adam
Adam
Num. training iterations
x 10*
x 10
3 x
10
16 x 10*
Max. instruction length
60
80
80
80
Image feature size
2048
512
512
512
Embedding dropout
0.1
0.3
0.3
0.3
Hidden size
768
512
512
512
Num. of Transformer//1. STM layers
SMM
12
2
4
Transformer/1 .STM dropout rate
0.1
0.2
0.5
0.3
Num. of parameters (million) Computation and training time
250M
57M (small), 120M (base)
8M
S7M
RTX A4000: 72h
RTX A6000: 72h
RTX A6000: 48h
RTX A6000: 48h
Hallucination Threshold
0.92
0.98
0.98
0.42
Table 3: The hyperparameters of all models. For the T5 models, we use decoders with two layers, which improve the performance compared to the original decoders.
Direction Type
Candidate Words
Horizontal
left
right
front
back
forward
backward
towards away from
through
past
leftmost rightmost
Vertical
bottom middle
top
up down
ahove under
Location
enter exit
into outside
of
inside first second
third
Table Directional word list. Each row shows a group of words.
"storage room", "attic", "basement", "gallery", "greenhouse", "yoga studio", "meditattoon room", "stairs", "staircase", "floor" ). These are based on room labels in the Matterport3D dataset (Chang et al., 2017) and suggestions of GPT-4 (OpenAI, 2023).
A.3 Results on development set (Table 5)
Model
F-1
Precision
Recall
Random
20.4
20.0
20.8
Speaker probability
45.5
37.3
58.3
LSTM
34.0
32.7
35.4
T5-small
44.9
34.4
64.6
T5-base
40.0
28.6
66.7
Fine-tuned Airbert
57.1
50.0
66.7
Table 5:
Performance the development set of all
models.


Reviews:
[{"review_id": "FxzkBEVWIf", "reviewer": "Reviewer_ieLz", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank you for your thorough review and suggestions.\n\n**Only two kinds of hallucination are studied**: We create synthetic extrinsic hallucinations by not only repeating sentences, but also randomly sample a sentence from the instruction dataset to add to the ground truth instruction, which is to ask the model to do a subtask that most likely does not exist in the trajectory. We agree that it would be interesting to investigate going a detour in the middle, and will incorporate the suggestions in future work.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. **Importance of the problem**: A system that is describe data in text form often exhibits *hallucination* problem. Despite that this is a huge problem in the general domain, a focus study on *Grounded Instruction Following* is still useful to Embodied AI and pragmatics community. \n2. **The innovation of the approach**: Creating synthetic data and training with contrastive objective are not new techniques, but afaik, they are used in this problem for the first time. \n3. **Experiments**: The results show that their model and training method is better than the baselines.", "reasons_to_reject": "My major concern is that **Only two kinds of hallucination are studied**: using wrong word or repetitive sentences according to their data generation method. In practice the hallucination could be more complicated, e.g. instruction for going a detour in the middle (which could be part of the trajectory [1]) or asking the model to do a subtask that doesn't exist in the trajectory. More insights should be given regarding approaches and empirical results regarding finding this kind of hallucination. \n\n------\nEDIT after rebuttal: I have read the author's rebuttal and other reviews. Reviewer CKdk raised a similar concern regarding synthetic data creation method in this paper. I think this paper will be much stronger if the authors include other kinds of hallucination in both training and test data. The author's rebuttal was insufficient to resolve this concern. \n\nI will lower my excitement score to 3 (*it can significantly benefit from another round of revision*), but keep the soundness score (*This study provides sufficient support for all of its claims/arguments*, although the claims are weak). \n\n------\n[1]: Don't Copy the Teacher: Data and Model Challenges in Embodied Dialogue. EMNLP 2022. So Yeon Min, Hao Zhu, Ruslan Salakhutdinov, Yonatan Bisk", "questions_to_authors": "", "paper_topic_and_main_contributions": "### A summary of the paper\n- The paper investigates detecting hallucinations (inconsistent references to objects or actions) in instructions generated by models to navigate simulated environments (Section 1).\n\n- The authors categorize the hallucinations into *intrinsic hallucinations* -- inaccurate description of the trajectory that needs to be replaced by a correct word and *extrinsic hallucinations* -- extra unnecessary words that should be removed as they are unrelated to the trajectory. (Section 3)\n\n- The paper proposes a weakly-supervised approach using a pre-trained vision-language model fine-tuned with synthetic data to detect hallucinations (Sections 3-4). \n\n- The approach uses data augmentation strategies to create synthetic training data with free hallucination labels (Section 4.3).\n\n- A contrastive learning objective is used to fine-tune the pre-trained model to distinguish hallucinations from non-hallucinations (Section 4.2). \n\n- The proposed model outperforms LSTM and Transformer baselines and ablation studies show benefits of pre-training and contrastive fine-tuning (Section 5). \n\n- The model struggles with some types of hallucinations like detecting rooms and objects, due to lack of diversity in training data (Section 5).\n\n### Contributions\nThe core contributions are developing a weakly-supervised and data-efficient approach for hallucination detection for grounded instruction following, and showing its effectiveness compared to supervised baselines. The pre-training plus contrastive fine-tuning method helps reduce the need for expensive human annotations.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "ZYRoxzlmlu", "reviewer": "Reviewer_CKdk", "summary": "", "questions": "- A. Why would repeating a sentence lead to external hallucination?\n- B. Line 201: \"we manually annotated a small sample of labels.\" How many? Was this data used in the synthetic data creation, or was it done just to see how time-consuming it would be?\n- C. §5: Are the LSTM and T5 baselines trained on the same synthetic dataset?\n- D. §5: Based on my understanding, since the task is to evaluate whether a given word in an instruction is hallucinated, a different index for the same instruction would count as a distinct example. If there are 632 test examples, how many instructions do they come from? Did you use every word in each instruction as an example, or only words susceptible to hallucination with the same method in §3? How many authors helped to annotate this data, and what was inter-annotator agreement?\n- E. §5: Why was the dev set collected? It's not clear how it was used.\n- F. Do you plan to release the dev and test data that you created?", "limitations": "", "rating": 2, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank you for your thorough assessment and suggestions. To answer the questions,\n\n**Question A**: For example, if we add a repeating sentence to ground truth instruction A “Walk out of the office. Walk into the hallway and turn left”, the new instruction B becomes “Walk out of the office. Walk into the hallway and turn left. **Walk into the hallway and turn left**”. The last sentence either does not make sense in the environment after completing the second sentence, or the last sentence would lead to a wrong destination. Each word in the last sentence from instruction B needs to be removed in order to be consistent with the input trajectory, thus repeating a sentence most likely will lead to external hallucination during the synthetic dataset creation.\n\n**Question B**: We annotated 209 development examples and 632 test examples (mentioned in Section 5 line 255), in order to evaluate our proposed model and baseline models. The datasets were created for evaluation, instead of training. \n\n**Question C**: Yes, the baselines were trained on the same synthetic dataset as our proposed model.\n\n**Question D**: The 632 test examples come from 75 instructions. We use only words that are susceptible to hallucination with the same method in §3. Two authors annotated the datasets, and the labels are determined by mutual agreement.\n\n**Question E**: The dev set was collected for determining the hallucination decision threshold of each model by maximizing its F-1 score (Table 1 caption). If a word has a hallucination probability greater than the selected threshold, the word is considered hallucination.\n\n**Question F**: Yes, we plan to release the datasets.\n\n**“Results vary greatly between the dev and test set in Table 2”**: The dev set is small, and this dataset was created for determining the hallucination decision threshold of each model by maximizing its F-1 score. We consider the results on the test set to be more reliable as this dataset is larger.\n\n**“The evaluation setting is limited in terms of impact”**: Other evaluation settings to compare with existing benchmarks, e.g.discarding hallucinated instructions to improve performance, can be an interesting follow-up. Human evaluation by navigation with the generated instruction is necessary in those settings. We experimented with the evaluation setting of communicating hallucination to human users, however, it is a non-trivial problem. Our preliminary results show that humans are susceptible to the text instructions and the display of the instruction. We will investigate these problems in future work.\n\nFor your other suggestions, we will incorporate them in the next version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper tackles an interesting problem. Hallucination is one of the most prominent issues in NLP today, and it is interesting to see discussion of it in the multimodal space.", "reasons_to_reject": "- The method is evaluated on one test set, created by the authors manually annotating a small subset (75 trajectories) of an existing dataset (R2R). They produce 632 test examples this way, but it is not clear how many unique instructions this corresponds to. As seen in Table 2, results vary greatly between the dev and test set, and there is no reporting of variance across different random seeds for training.\n- The gold labels in the test set are annotated by the authors themselves. There is no description of how many authors did annotation, or what inter-annotator agreement was. \n- The evaluation setting is limited in terms of impact. The paper would be much stronger if the authors showed that if they use their method to discard hallucinated instructions, then they can improve performance on an existing benchmark.\n- The synthetic data creation method is relatively limited, with three word-swap rules for creating intrinsic hallucinations, and appending a random sentence for creating extrinsic hallucinations.", "questions_to_authors": "- A. Why would repeating a sentence lead to external hallucination?\n- B. Line 201: \"we manually annotated a small sample of labels.\" How many? Was this data used in the synthetic data creation, or was it done just to see how time-consuming it would be?\n- C. §5: Are the LSTM and T5 baselines trained on the same synthetic dataset?\n- D. §5: Based on my understanding, since the task is to evaluate whether a given word in an instruction is hallucinated, a different index for the same instruction would count as a distinct example. If there are 632 test examples, how many instructions do they come from? Did you use every word in each instruction as an example, or only words susceptible to hallucination with the same method in §3? How many authors helped to annotate this data, and what was inter-annotator agreement?\n- E. §5: Why was the dev set collected? It's not clear how it was used.\n- F. Do you plan to release the dev and test data that you created?", "paper_topic_and_main_contributions": "The paper presents a model for detecting hallucinated words in navigation instructions. They present a formulation of the hallucination detection problem at the word-level, create new evaluation data, and establish baselines.", "excitement": 2, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "- I wonder if the bigram \"navigation instructions\" should be in the title, to distinguish from instructions for models, as in the instruction-tuning literature.\n- Lines 147-148: \"For each instruction, we pre-select a subset of words as candidates for classification.\" This makes it sound like *for each instruction*, you manually identified candidate words. I would suggest \"We develop heuristics to identify candidate words, which we apply to all instructions\" (if it is true)."}}, {"review_id": "t7QUzteZVc", "reviewer": "Reviewer_Kkpf", "summary": "", "questions": "- Could the authors clarify if pre-training is a novelty of this paper? If so, in what way is it novel?\n- What does the performance for the baselines look like once they are parameter-matched? (say, by using a T5-base instead of T5-small)", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank you for your thorough review and questions.\n\n**“Is pre-training a novelty of this paper?”**: Pre-training method is not novel, but applying it to this problem is novel. While pre-training has been shown to help on a variety of classification tasks, any claim that it would help on a new task needs to be empirically verified. This task is especially challenging than standard text or image classification tasks: it involves understanding the correspondence between a **sequence of photos** and a **sequence of words**. The novelty of our work also lies in contrastively fine-tuning a pre-trained model using a synthetically created dataset.\n\n**“Comparison with parameter-matched baselines”**: We trained a hallucination detection model using T5-base, to have another row of experimental results for Table 1. On the Test set, T5-base hallucination detection model underperforms our model by 32.4% while having a comparable number of parameters (F1=34.9). On the Dev set, T5-base model underperforms our model by 42.8% (F1=40.0). As the training data is not very large, using a larger model might not be too helpful here. The T5-base hallucination detection model still underperforms our proposed model.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The approach of constructing synthetic negatives by swapping rooms/directions is interesting, particularly based on the amount of improvement observed from the addition of such negatives.", "reasons_to_reject": "- Pre-training a transferred model on the new task is claimed to be a novelty here. However, task-adaptive pre-training has been long known to help improve any classification task (Gururangan et al. 2020). This limits the novelty of the proposed pre-training strategy in this work. From the result in Table 2, without pre-training, the fine-tuned Airbert model performs almost as well as the LSTM model (worse than the Speaker and the T5 models)\n- The baselines in Table 1 are significantly weaker than the proposed model in terms of model parameters (57 million for speaker/T5 vs 250 million for Airbert from Appendix A.1; lines 649-657) which may skew the results in favor of the model shown here. Without a clear comparison with baselines that are nearly parameter-matched, these numbers do not add much value to the analysis.\n\n**Reference**:\n\nGururangan et al. 2020: Don't Stop Pretraining: Adapt Language Models to Domains and Tasks, ACL 2020", "questions_to_authors": "- Could the authors clarify if pre-training is a novelty of this paper? If so, in what way is it novel?\n- What does the performance for the baselines look like once they are parameter-matched? (say, by using a T5-base instead of T5-small)", "paper_topic_and_main_contributions": "Instruction generation models in embodied navigation tasks tend to hallucinate by generating inconsistent actions or repeating themselves. This paper proposes to detect such hallucinations using a model trained in a joint pre-training + contrastive-finetuning fashion. To improve contrastive learning, the author(s) propose using synthetic negatives generated by altering the rooms/directions or swapping words in generated instructions. On the R2R dataset, the author(s) demonstrate their objectives applied to a pre-trained Airbert model detects hallucinations better than models trained using the standard likelihood objective.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: Reviewers agreed that the paper was sufficiently sound, but generally agreed that the scope of the paper was limited. Reviewers raised concerns about the evaluation task (only hallucination detection, rather than downstream performance on VLN after filtering augmented data, or instruction generation), small size of the test dataset, and limited type of hallucinations detected. However, reviewers also identified that the problem is important, and the method is simple (uses a small number of word-swapping rules, and random sentence appending, in combination with pre-training) but effective. This method, or extensions of it, may prove helpful in some downstream tasks. 

One additional paper on hallucination detection for generation in VLN settings is Huang et al., Multi-Modal Discriminative Model for Vision-and-Language Navigation. SpLU-RoboNLP workshop, 2019, although it doesn't diminish the novelty of this paper as it keeps the language the same and corrupts paths, and doesn't use pre-training.
Decision: Accept-Findings