SlowGuess/NAACL_2022_OCR huggingface dataset:
Moreover, we further investigate the performance of the different components of the two proposed reconstruction strategies, respectively. As shown in Table 11, encoder-decoder-reconstructor module for back reconstruction strategy significantly outperforms encoder-reconstructor module by 2.2 combined scores because dialog state decoder could achieve more dialog context information for encoder-decoder-reconstructor. In addition, regarding two denoising reconstruction modules, dialog state denoising and response denoising have achieved similar performance. These two modules could improve the denoising capability of the task-oriented dialog system.
Table 10: Comparison of policy optimization models evaluated on MultiWOZ 2.0.
Model	Inform	Success	BLEU	Combined
Back reconstruction	92.9	84.0	18.8	107.3
w/o enc-rec	92.2	83.5	19.0	106.9
w/o enc-dec-rec	92.1	81.2	18.0	104.7
Denoising reconstruction	92.0	84.4	18.1	106.3
w/o dialog state denoising	91.7	83.0	17.9	105.3
w/o response denoising	92.8	81.2	18.6	105.6
Table 11: The performance of the different components of the two proposed reconstruction strategies. enc-dec denotes encoder-reconstructor module, enc-dec-rec denotes encoder-decoder-reconstructor module.
A.5 More examples
Figures 7 - 11 show several examples generated by MinTL and BORT, respectively. As shown in Figure 7, MinTL generates the inadequate dialog state, which may provide the hotel without internet. Our proposed BORT reconstructs the generated dialog state back to the original input context to ensure the information in the input side is completely transformed to the output side to achieve an adequate dialog state via a back reconstruction strategy. Figure 8 shows that our proposed BORT generated the correct slot value 'european' rather than the corrupted one 'europeon' from the corrupted dialog context, indicating the robustness of the denoising reconstruction strategy. As shown in Figures 9 - 11, MinTL generates the inaccurate dialog state, leading to the inaccurate response. The results are consistent with our opinion that the generated dialog state, which is crucial for task completion of a task-oriented dialog system, has always been inaccurate across the end-to-end task-oriented dialog system training. Moreover, Figure 11 shows that MinTL faces the problem of error propagation from both previously generated inaccurate dialog states and responses. Our proposed BORT can alleviate these issues via reconstruction strategies, further demonstrating the effectiveness of BORT.
Figure 7: An example of the task-oriented dialog systems in dialog session MUL1139.
Figure 8: An example of the task-oriented dialog systems in dialog session PMUL0095.
Figure 9: An example of the task-oriented dialog systems in dialog session PMUL3868.
Figure 10: An example of the task-oriented dialog systems in dialog session MUL0286.
Figure 11: An example of the task-oriented dialog systems in dialog session PMUL3875.