SlowGuess/NAACL_2022_OCR huggingface dataset:
Haipeng Sun, Junwei Bao,\* Youzheng Wu, Xiaodong He
JD AI Research, Beijing, China
{sunhaipeng6, baojunwei, wuyouzheng1, hexiaodong}@jd.com
A typical end-to-end task-oriented dialog system transfers context into dialog state, and upon which generates a response, which usually faces the problem of error propagation from both previously generated inaccurate dialog states and responses, especially in low-resource scenarios. To alleviate these issues, we propose BORT, a back and denoising reconstruction approach for end-to-end task-oriented dialog system. Squarely, to improve the accuracy of dialog states, back reconstruction is used to reconstruct the original input context from the generated dialog states since inaccurate dialog states cannot recover the corresponding input context. To enhance the denoising capability of the model to reduce the impact of error propagation, denoising reconstruction is used to reconstruct the corrupted dialog state and response. Extensive experiments conducted on MultiWOZ 2.0 and CamRest676 show the effectiveness of BORT. Furthermore, BORT demonstrates its advanced capabilities in the zero-shot domain and low-resource scenarios<sup>1</sup>.
Recently, task-oriented dialog systems, which aim to assist users to complete some booking tasks, have attracted great interest in the research community and the industry (Zhang et al., 2020c). Task-oriented dialog systems have been usually established via a pipeline system, including several modules such as natural language understanding, dialog state tracking, dialog policy, and natural language generation. The natural language understanding module converts user utterance into the structured semantic representation. The dialog state generated by the dialog state tracking module is used to query the database to achieve matched entities. The natural language generation module converts the action
Figure 1: Illustration of different error propagation problem types, denoted by arrows in different colors, along the multi-turn task-oriented dialog flow. For example, the orange arrow indicates that the error in the previously generated response would affect the response generation in the current dialog turn.
state estimated by the dialog policy module to the natural language response. This modular-based architecture is highly interpretable and easy to implement, used in most practical task-oriented dialog systems in the industry. However, every module is optimized individually and doesn't consider the entire dialog history, which affects the performance of the dialog system. Therefore, many researchers focus on end-to-end task-oriented dialog systems to train an overall mapping from user natural language input to system natural language output (Lei et al., 2018; Zhang et al., 2020b; Hosseini-Asl et al., 2020; Lin et al., 2020; Yang et al., 2021).
However, all existing task-oriented dialog systems still suffer from one or more types of error propagation problems from both previously generated inaccurate dialog states and responses, as is illustrated in Figure 1. Firstly, the generated dialog state, which is crucial for task completion of task-oriented dialog systems, is usually inaccurate across the end-to-end task-oriented dialog system training. Secondly, the previously generated dialog state and response are encoded to create the current
1The code is available at https://github.com/JD-AI-Research-NLP/BORT.
Findings of the Association for Computational Linguistics: NAACL 2022, pages 2156 - 2170
dialog state and response during inference, while the oracle previous dialog state and response are encoded during training. There exists a discrepancy between training and inference, affecting the quality of generated system responses.
We propose BORT, a back and denoising reconstruction approach for end-to-end task-oriented dialog systems to alleviate these issues. To improve dialog state learning ability, back reconstruction is used to reconstruct the generated dialog state back to the original input context to ensure that the information in the input side is completely transformed to the output side. In addition, to enhance the denoising capability of the task-oriented dialog system to reduce the impact of error propagation, denoising reconstruction is used to reconstruct the corrupted dialog state and response. It guarantees that the system learns enough internal information of the dialog context to recover the original version. Experimental results on MultiWOZ 2.0 and CamRest676 show that our proposed BORT substantially outperforms baseline systems. This paper primarily makes the following contributions:
- We propose two effective reconstruction strategies, i.e., back and denoising reconstruction strategies, to improve the performance of end-to-end task-oriented dialog systems.
- Extensive experiments and analysis on MultiWOZ 2.0 and CamRest676 show the effectiveness of BORT.
- BORT achieves promising performance in zero-shot domain scenarios and alleviates poor performance in low-resource scenarios.
2 Task-Oriented Dialog Framework
As illustrated in Figure 2(a), we construct an encoder-decoder framework for an end-to-end task-oriented dialog system via dialog state tracking and response generation tasks. One shared encoder encodes dialog context, and two different decoders decode dialog state and system response, respectively. The objective function  $\mathcal{L}_{all}$  of the entire training process is optimized as:
$$
\mathcal {L} _ {\text {a l l}} = \mathcal {L} _ {B} + \mathcal {L} _ {R}, \tag {1}
$$
where  $\mathcal{L}_B$  is the objective function for dialog state tracking, and  $\mathcal{L}_R$  is the objective function for response generation.
2.1 Dialog State Tracking
Motivated by Lin et al. (2020), we model the Levenshtein dialog state, which means the difference between the current dialog state and the previous dialog state, for dialog state tracking task to generate minimal dialog state and reduce the inference latency. The Levenshtein dialog state  $\Delta B_{t}$  of dialog turn  $t$ , is generated based on the previous dialog state  $B_{t-1}$ , the previous system response  $R_{t-1}$ , and the current user utterance  $U_{t}$  via the encoder-decoder framework:
$$
H _ {e b} = \operatorname {e n c o d e r} \left(B _ {t - 1}, R _ {t - 1}, U _ {t}\right), \tag {2}
$$
$$
\Delta B _ {t} = \operatorname {d e c o d e r} _ {b} \left(H _ {e b}\right), \tag {3}
$$
where  $H_{eb}$  denotes the hidden representation of the encoder for dialog state tracking. Therefore, the dialog state tracking objective function can be optimized by minimizing:
$$
\mathcal {L} _ {B} = \sum_ {i = 1} ^ {N} \sum_ {t = 1} ^ {n _ {i}} - l o g P \left(\Delta B _ {t} \mid B _ {t - 1}, R _ {t - 1}, U _ {t}\right), \tag {4}
$$
where  $N$  denotes the number of dialog sessions,  $n_i$  denotes the number of dialog turns in the dialog session  $i$ .
For inference, a predefined function  $\Omega(\cdot)$  is used to generate the dialog state  $B_{t}$  as
$$
B _ {t} = \Omega \left(\Delta B _ {t}, B _ {t - 1}\right). \tag {5}
$$
The predefined function  $\Omega(\cdot)$  deletes the slot-value pair in  $B_{t-1}$  when the NULL symbol appears in the  $\Delta B_t$ , and it updates the  $B_{t-1}$  when new slot-value pair or new value for one slot appears in the  $\Delta B_t$ . Refer to Lin et al. (2020) for more details. The generated dialog state  $B_t$  is used to query the corresponding database. The database state embedding  $DB_t$  represents the number of matched entities and whether the booking is available or not. The embedding  $DB_t$  is used as the start token embedding of the response decoder for response generation.
2.2 Response Generation
The response  $R_{t}$  of dialog turn  $t$  is generated based on the previous system response  $R_{t-1}$ , the current user utterance  $U_{t}$ , the current dialog state  $B_{t}$ , and the database state embedding  $DB_{t}$ , which is formulated as:
$$
H _ {e r} = \operatorname {e n c o d e r} \left(R _ {t - 1}, U _ {t}, B _ {t}\right), \tag {6}
$$
$$
R _ {t} = \operatorname {d e c o d e r} _ {r} \left(H _ {\text {e r}}, D B _ {t}\right), \tag {7}
$$
(a) Base architecture
(b) Back reconstruction
(c) Denoising reconstruction
Figure 2: Illustration of the task-oriented dialog training process. We take turn  $t$  of a dialog session as an example.
where  $H_{er}$  denotes the hidden representation of the encoder for response generation. Therefore, the response generation objective function can be optimized by minimizing:
$$
\mathcal {L} _ {R} = \sum_ {i = 1} ^ {N} \sum_ {t = 1} ^ {n _ {i}} - \log P \left(R _ {t} \mid R _ {t - 1}, U _ {t}, B _ {t}, D B _ {t}\right). \tag {8}
$$
In this section, we proposes two reconstruction strategies, i.e., back reconstruction and denoising reconstruction, respectively. Generally, during task-oriented dialog training, objective functions  $\mathcal{L}_{BR}$  and  $\mathcal{L}_{DR}$  are added to enhance model learning ability. The general objective function of a task-oriented dialog system can be reformulated as follows:
$$
\mathcal {L} _ {a l l} = \mathcal {L} _ {B} + \mathcal {L} _ {R} + \lambda_ {1} \mathcal {L} _ {B R} + \lambda_ {2} \mathcal {L} _ {D R}, \tag {9}
$$
where  $\mathcal{L}_{BR}$  and  $\mathcal{L}_{DR}$  denote the objective functions for back reconstruction and denoising reconstruction.  $\lambda_{1}$  and  $\lambda_{2}$  are hyper-parameters that adjust the weights of the objective functions.
3.1 Back Reconstruction
Dialog state is essential for the task completion of a task-oriented dialog system (Wang et al., 2022). As illustrated in Figure 2(b), we propose a
back reconstruction strategy to mitigate the generation of inaccurate dialog states, including encoder-reconstructor and encoder-decoder-reconstructor modules. For the encoder-reconstructor module, the dialog context  $C(t) = (B_{t-1}, R_{t-1}, U_t)$  could be reconstructed to enhance encoder information representation by the encoder hidden representation  $H_{eb}$ . For the encoder-decoder-reconstructor module, the decoder hidden representation  $H_{db}$  could be used to reconstruct the dialog context  $C(t)$  to encourage the dialog state decoder to achieve complete information of dialog context.
The dialog state would be reconstructed back to the source input and the corresponding reconstruction score would be calculated to measure the adequacy of the dialog state. The objective function  $\mathcal{L}_{BR}$  for the back reconstruction is optimized by minimizing:
$$
\begin{array}{l} \mathcal {L} _ {B R} = \sum_ {i = 1} ^ {N} \sum_ {t = 1} ^ {n _ {i}} - \log P (C (t) | H _ {e b}) \tag {10} \\ + \sum_ {i = 1} ^ {N} \sum_ {t = 1} ^ {n _ {i}} - l o g P (C (t) | H _ {d b}). \\ \end{array}
$$
3.2 Denoising Reconstruction
To enhance the denoising capability of the task-oriented dialog system, we propose denoising reconstruction to guarantee that the system learns enough dialog context representation to recover the
original version, as illustrated in Figure 2(c). Motivated by denoising auto-encoder strategy that maps a corrupted input back to the original version (Vincent et al., 2010), we introduce noise in the form of random token deleting and masking in the source input to improve the dialog model learning ability. Specifically, we delete or mask every token in the previous dialog state and system response with a probability  $\alpha$ . More concretely, we propose two denoising reconstruction modules, i.e., dialog state denoising and response denoising modules.
For the dialog state denoising module, we reconstruct the new Levenshtein dialog state, which means the corrupted part of the dialog state rather than the complete dialog state in the original denoising auto-encoder. The Levenshtein dialog state  $\Delta B_{t - 1}^{\prime}$  of dialog turn  $t$ , is generated based on the noisy dialog context  $N_B(t) = (N(B_{t - 1}),R_{t - 1},U_t)$ .  $N(B_{t - 1})$  is the previous corrupted dialog state. For example, the Levenshtein dialog state 'taxidestination=stevenage train station' is reconstructed from the corrupted dialog state where 'taxi destination' is masked and 'train' is deleted, as shown in Figure 2(c). For response denoising module, the previous system response  $R_{t - 1}$  of dialog turn  $t$  is reconstructed based on the noisy dialog context  $N_R(t) = (N(R_{t - 1}),U_t,B_t,DB_{t - 1})$ .  $N(R_{t - 1})$  is the previous noisy system response. Therefore, the objective function  $\mathcal{L}_{DR}$  for the denoising reconstruction is optimized by minimizing:
$$
\begin{array}{l} \mathcal {L} _ {D R} = \sum_ {i = 1} ^ {N} \sum_ {t = 1} ^ {n _ {i}} - \log P \left(\Delta B _ {t - 1} ^ {\prime} \mid N _ {B} (t)\right) \tag {11} \\ + \sum_ {i = 1} ^ {N} \sum_ {t = 1} ^ {n _ {i}} - l o g P \left(R _ {t - 1} \mid N _ {R} (t)\right). \\ \end{array}
$$
3.3 Training and Inference Details
There exists inconsistency between the lexicalized user utterance and delexicalized system response, which is used to reduce the influence of different slot values on evaluation (Zhang et al., 2020b). This adds an extra burden for the system to generate a delexicalized system response. To alleviate this issue, we introduce delexicalized user utterances for response generation while lexicalized user utterances are still used for dialog state tracking. For example, '02:15' is converted into delexicalized form '[taxi_arryby]' for response generation, as shown in Figure 2(a). Different forms of user utterances take better training of both tasks, ultimately improving task completion.
For inference of dialog state tracking, generated previous dialog state, oracle previous system response, and current user utterance are used as dialog context to generate the current Levenshtein dialog state. For inference of response generation, motivated by Yang et al. (2021), we use generated previous system response instead of oracle previous system response to generate the current system response to maintain coherence throughout the whole dialog session.
To establish our proposed end-to-end task-oriented dialog system, we consider two task-oriented dialog datasets, MultiWOZ 2.0 (Budzianowski et al., 2018) and CamRest676 (Wen et al., 2017).
MultiWOZ 2.0 is a large-scale human-to-human multi-domain task-oriented dialog dataset. The dataset consists of seven domains: attraction, hospital, police, hotel, restaurant, taxi, and train. It contains 8438, 1000, and 1000 dialog sessions for training, validation, and testing datasets. Each dialog session covers 1 to 3 domains, and multiple different domains might be mentioned in a single dialog turn. Particularly, there are no hospital and police domains in the validation and testing dataset. To make our experiments comparable with previous work (Zhang et al., 2020b; Lin et al., 2020; Yang et al., 2021), we use the pre-processing script released by Zhang et al. (2020b) and follow the automatic evaluation metrics to evaluate the response quality for the task-oriented dialog system. Inform rate measures if a dialog system has provided a correct entity; Success rate measures if a dialog system has provided a correct entity and answered all the requested information; BLEU score (Papineni et al., 2002) measures the fluency of the generated response; the combined score, which is computed by  $(\text{Inform} + \text{Success}) \times 0.5 + \text{BLEU}$ , measures the overall quality of the dialog system. To evaluate the performance of dialog state tracking, we use the joint goal accuracy to measure the accuracy of generated dialog states.
CamRest676 is a small-scale restaurant-domain dataset. It contains 408, 136, 132 dialog sessions for training, validation, and testing datasets. To make our experiments comparable with previous work (Lei et al., 2018; Wu et al., 2021), we use the same delexicalization strategy and use BLEU score and Success F1 to evaluate the response qual
Model	Pre-trained	Inform	Success	BLEU	Combined
DAMD (Zhang et al., 2020b)	n/a	76.3	60.4	16.6	85.0
SimpleTOD (Hosseini-Asl et al., 2020)	DistilGPT2	84.4	70.1	15.0	92.3
MinTL-T5-small (Lin et al., 2020)	T5-small	80.0	72.7	19.1	95.5
SOLOIST (Peng et al., 2020)	GPT-2	85.5	72.9	16.5	95.7
MinTL-BART (Lin et al., 2020)	BART-large	84.9	74.9	17.9	97.8
LAVA (Lubis et al., 2020)	n/a	91.8	81.8	12.0	98.8
UBAR* (Yang et al., 2021)	DistilGPT2	91.5	77.4	17.0	101.5
SUMBT+LaRL (Lee et al., 2020)	BERT-base	92.2	85.4	17.9	106.7
Baseline (mask=0)	T5-small	89.0	78.8	17.9	101.8
Baseline (mask=0.15)	T5-small	88.0	77.6	17.7	100.5
BORT	T5-small	93.8++	85.8++	18.5	108.3++
Table 1: Comparison of end-to-end models evaluated on MultiWOZ 2.0. "+" after a score indicates that the proposed BORT is significantly better than Baseline (mask=0) at significance level  $p < 0.01$ . * denotes the re-evaluated result by the author-released model since the result reported in this original paper (Yang et al., 2021) was evaluated using the ground truth dialog state instead of generated dialog state to query the database entities.
ity for the task-oriented dialog system. The success rate measures if the system answered all the requested information to assess recall while Success F1 balances recall and precision.
In the training process for the task-oriented dialog system, we select two backbone models. BORT is a transformer-based system initialized by a pretrained model. BORT_G is a GRU-based system without a pre-trained model. The detailed training settings and results of the BORT_G backbone are provided in Appendix A.1.
For the BORT backbone, we use pre-trained T5-small (Raffel et al., 2020) to initialize the dialog system, based on the HuggingFace Transformers library (Wolf et al., 2020) and follow the settings of Lin et al. (2020). There are six layers for the encoder and the decoder. The dimension of hidden layers is set to 512, and the head of attention is 8. The batch size is set to 96. The AdamW optimizer (Loshchilov and Hutter, 2019) is used to optimize the model parameters. The learning rate is 0.0025, and the learning rate decay is 0.8. The hyper-parameters  $\lambda_{1}$  and  $\lambda_{2}$  are set to 0.05 and 0.03, respectively. For the denoising reconstruction strategy, the noise probability  $\alpha$  is 0.15. The hyperparameter analysis is provided in Appendix A.2. Training early stops when no improvement on the combined score of the validation set for five epochs. All results in the low-resource scenario are the average scores of three runs. One P40 GPU is used to train all dialog systems.
Compared with other previous work, our proposed BORT is evaluated in two context-to-response settings: end-to-end modeling to generate dialog state and system response, and policy optimization to generate system response based on ground truth dialog state. Policy optimization results are provided in Appendix A.3.
Sequicity (Lei et al., 2018) and DAMD (Zhang et al., 2020b) are GRU-based end-to-end task-oriented dialog systems with a copy mechanism. Decoder based pre-trained model GPT-2 (Radford et al., 2019) is used in SimpleTOD (Hosseini-Asl et al., 2020), SOLOIST (Peng et al., 2020), and UBAR (Yang et al., 2021). Encoder-decoder based pre-trained model T5 (Raffel et al., 2020) and BART (Lewis et al., 2020) is used in MinTL (Lin et al., 2020). Reinforcement learning is used in LAVA (Lubis et al., 2020) and SUMBT+LaRL (Lee et al., 2020). Especially, SUMBT+LaRL merges a dialog state tracking model SUMBT (Lee et al., 2019) and a dialog policy model LaRL (Zhao et al., 2019) and fine-tune them via reinforcement learning, achieving the state-of-the-art performance.
In addition, we implement two baseline systems. One baseline is the base architecture of a task-oriented dialog system, as illustrated in Figure 2(a). The other is a noise-based baseline system, just masking  $15\%$  tokens in the dialog context for dialog training.
4.4 Main Results
Table 1 presents the detailed inform rates, success rates, BLEU scores,and combined scores of end
to-end dialog models on the MultiWOZ 2.0. Our re-implemented baseline system performs better than MinTL (Lin et al., 2020), using the same pretrained T5-small model. This indicates that the baseline is a strong system. Our proposed BORT significantly outperforms our re-implemented baseline system by 6.5 combined scores, while the simple noise-based method (Baseline (mask=0.15)) doesn't achieve better performance. Moreover, BORT outperforms the previous state-of-the-art SUMBT+LaRL by 1.6 combined scores, achieving the best performance in terms of inform rate, success rate, and combined score. This demonstrates the effectiveness of our proposed BORT.
Model	Success F1	BLEU
Sequicity (Lei et al., 2018)	85.4	25.3
ARDM (Wu et al., 2021)	86.2	25.4
SOLOIST (Peng et al., 2020)	87.1	25.5
BORT	89.7	25.9
To better assess the generalization capability of BORT, we fine-tune BORT on the CamRest676. The detailed Success F1 and BLEU scores on the CamRest676 are presented in Table 2. Our proposed BORT outperforms the previous state-of-the-art SOLOIST by 2.6 Success F1, achieving the best performance in terms of Success F1. This demonstrates the generalization capability of our proposed BORT.
4.5 Further Evaluation Analysis
Nekvinda and Dusek (2021) identify inconsistencies between previous task-oriented dialog methods in data preprocessing and evaluation metrics and introduce a standalone standardized evaluation script. BLEU score is computed with references, which have been obtained from the delexicalized MultiWOZ 2.2 span annotations.
Table 2: Comparison of end-to-end task-oriented dialog systems on CamRest676.
Model	Inform	Success	BLEU	Combined
DAMD (Zhang et al., 2020b)	57.9	47.6	16.4	69.2
LABES (Zhang et al., 2020a)	68.5	58.1	18.9	82.2
AuGPT (Kulhánek et al., 2021)	76.6	60.5	16.8	85.4
MinTL-T5-small (Lin et al., 2020)	73.7	65.4	19.4	89.0
SOLOIST (Peng et al., 2020)	82.3	72.4	13.6	91.0
DoTS (Jeon and Lee, 2021)	80.4	68.7	16.8	91.4
UBAR (Yang et al., 2021)	83.4	70.3	17.6	94.5
BORT	85.5	77.4	17.9	99.4
To get a more complete picture of the effectiveness of reconstruction strategies, we also use this evaluation script to evaluate our proposed BORT which is trained on MultiWOZ 2.0. As shown in Table 3, BORT also substantially outperforms the previous state-of-the-art UBAR by a large margin (4.9 combined scores), achieving the best performance in terms of inform rate, success rate, and combined score. This further demonstrates the effectiveness of our proposed BORT.
We empirically investigate the performance of the different components of BORT as shown in Table 4. Our introduced user utterance delexicalization strategy gains 1.9 combined scores, indicating the effectiveness of the user utterance delexicalization strategy. Back reconstruction performs slightly better than denoising reconstruction by 1 combined score regarding the two proposed reconstruction strategies. Moreover, the combination of both reconstruction strategies can complement each other to further improve the performance of the dialog system. The detailed analysis on different modules of every reconstruction strategy is provided in Appendix A.4.
Table 3: Comparison of end-to-end task-oriented dialog systems evaluated on the standardized setting (Nekvinda and Dušek, 2021).
Model	Inform	Success	BLEU	Combined
BORT	93.8	85.8	18.5	108.3
w/o DR	92.9	84.0	18.8	107.3
w/o BR	92.0	84.4	18.1	106.3
w/o BR & DR	90.4	81.4	17.8	103.7
w/o BR & DR& UD	89.0	78.8	17.9	101.8
Table 4: The performance of the different components of our proposed BORT on MultiWOZ 2.0. BR denotes back reconstruction, DR denotes denoising reconstruction, UD denotes user utterance delexicalization.
4.7 Dialog State Tracking
Table 5 reports the dialog state tracking performance of the end-to-end task-oriented dialog systems on MultiWOZ 2.0. BORT substantially outperforms MinTL (Lin et al., 2020) using the same pre-trained T5-small model by 2.8 points, achieving 54.0 joint goal accuracy. Moreover, BORT achieves the highest joint goal accuracy among the end-to-end task-oriented dialog systems. This indicates that our proposed reconstruction strategies could improve dialog state learning ability.
4.8 Case Study and Human Evaluation
Moreover, we analyze translation examples and conduct a human evaluation to further explore the