Venue: NAACL 2022

# AISFG: Abundant Information Slot Filling Generator

Yang Yan, Junda Ye, Zhongbao Zhang, Liwen Wang

Beijing University of Posts and Telecommunications, Beijing, China

{yanyang42,jundaye,zhongboazb,w_liwen}@bupt.edu.cn

Contributed equally.The source code is available at [https://github.com/realyanyang/AISFG](https://github.com/realyanyang/AISFG).

###### Abstract

As an essential component of task-oriented dialogue systems, slot filling requires enormous labeled training data in a certain domain. However, in most cases, there is little or no target domain training data is available in the training stage. Thus, cross-domain slot filling has to cope with the data scarcity problem by zero/few-shot learning. Previous researches on zero/few-shot cross-domain slot filling focus on slot descriptions and examples while ignoring the slot type ambiguity and example ambiguity issues. To address these problems, we propose **A**bundant **I**nformation **S**lot **F**iling **G**enerator (**A**iSFG), a generative model with a novel query template that incorporates domain descriptions, slot descriptions, and examples with context. Experimental results show that our model outperforms state-of-the-art approaches in zero/few-shot slot filling task.1

Footnote 1: The source code is available at [https://github.com/realyanyang/AISFG](https://github.com/realyanyang/AISFG).

+
Footnote †: _Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_, pages 4180 - 4187

## 1 Introduction

Slot filling is a critical part of downstream tasks in natural language understanding (NLU) such as dialogue systems. Recently, some supervised slot filling models have achieved state-of-the-art results within deep learning Mesnil et al. (2013); Yao et al. (2013); Louvan and Magnini (2018); Kim et al. (2019).Nonetheless, these methods have a strong dependency on the domain-specific labels, which is not capable of transferring to new domains that always contain little or no data.

To alleviate the problem of resource gap between source and target domains, cross-domain zero-shot has become an important research direction in slot filling. However, most researches on slot filling utilize token-level classification frameworks, which means they either convert it to a BIO tag labeling task Shah et al. (2019); Bapna et al. (2017); Du et al. (2021) or predict the start and end position as QA task in the sentence Du et al. (2021); Yu et al. (2021), to extract spans. Recently, prompt learning methods reformulate the downstream tasks to a similar form with the pre-training tasks, which can fully utilize the knowledge encoded in PLMs and improve the performance of the downstream tasks in the scenarios of data scarcity. Motivated by this, we consider making the slot filling task consistent with PLMs pre-training tasks.

In this paper, we propose a generative template-based zero-shot slot filling framework named Abundant Information Slot Filling Generator (AISFG), which utilizes pre-trained generative model as the backbone and generates responses in natural language style. Thus, the slot filling task is consistent with the pre-training task of the PLM. In particular, we notice that shared cross-domain slot types sometimes refer to totally different entities across different domains. For example, in the SNIPS dataset Coucke et al. (2018), domains _RateBook_ and _SearchScreeningEvent_ share a common slot type _object_type_, but it refers to book type and movie schedule, respectively. We call this slot type ambiguity issue. Moreover, we argue that incorporating slot examples only is far from fully utilizing the example information. For instance, for sentence _give 5 out of 6 stars to creatures of light and darkness_, we give the slot description to find _best rating_ with examples _like 6 and 5_, the model produces the wrong answer 5 but not 6. We conjecture that, the model just learn that predicting a number is satisfying from these two examples but does not understand what is the best rating means. We call this example ambiguity issue. Thus, we attach our attention to domain-specific descriptions and examples with context to alleviate the above two issues. Specifically, we design the query template by incorporating domain descriptions, slot descriptions and examples with context.

The contribution of this paper can be summarized in three aspects:

* We propose a generative template-based zero-shot slot filling framework. To the best of our knowledge, we are the first to apply the generative framework to perform the zero-shot cross-domain slot filling task.
* We focus on slot type ambiguity and example ambiguity, which is ignored by previous researches. We incorporate domain-specific descriptions and examples with context to the query template to handle these two issues.
* The experimental results show that AISFG achieves better performance than the existing methods on the setting of zero/few-shot.

## 2 Methodology

### Slot Filling as Generation

Given a sentence \(x\) from domain \(d\in{\cal D}\), slot filling aims to predict a set of (slot type, span) pairs \((s,y)\), where \(s\in{\cal S}\) is a specific entity type from a fixed set of slot types, and \(y=\{y_{1},y_{2},\ldots,y_{i}\}\) is a set of spans in sentence \(x\). Most work on slot filling utilizes token-level classification frameworks, which means they either convert it to BIO tag labeling or predict the start and end position in the sentence, to extract spans.

In contrast to this convention, we frame slot filling as a conditional sequence generation task and solve it in a sequence-to-sequence manner. As shown in Figure 1, given a sentence \(x\), domain \(d\), slot type \(s\) and target \(y\), we construct natural language query \(q=t_{q}(x,d,s)\) and response \(r=t_{r}(y)\) based on the predefined template. The generative model takes \(q\) as input and directly predicts \(r\) in a generation manner.

Leveraging the rich knowledge in the pre-trained generative model is important for cross-domain slot filling, especially in zero/few-shot setting. Thus, we construct query and response as natural language sentences to naturally utilize them.

Query ConstructionTo solve the slot type ambiguity and example ambiguity, we synthesize the query by incorporating domain descriptions, slot descriptions and examples with context. Specifically, query construction template is formulated as: in domain 1, find the 2, like 3 in 4, in sentence: \(x\).

where the blank 1, 2, 3, 4 are filled with domain description, slot description, example entity and context for example, respectively. To avoid data leakage, we construct examples and contexts manually to ensure the example entities are not appeared in the dataset. The predefined specific mappings between domain/slot and description/example/context are reported in Appendix A.1 and A.3. Moreover, a query example is illustrated in Figure 1.

Response ConstructionIdeally, the response should be as simple as possible to alleviate the generation difficulty. Besides, an explicit template is easily converted to the original format for compatibility. Thus, we construct response by concatenating target entities with commas:

\[\underline{y_{1},y_{2},\ldots,y_{i}}.\]

where \(y_{i}\) is the \(i\)-th entity span in sentence \(x\). A response example is illustrated in Figure 1.

### Train and Inference

The sequence-to-sequence model is instantiated as a pre-trained generative language model, such as BART Lewis et al. (2020). In the training stage, for the input sentence \(x\) with multiple slots, we build a training pair, which consists of a query and a response based on the templates above for each slot. We further fine-tune the parameters based on these training pairs to maximize the log likelihood for predicting gold responses just like in an ordinary generation task. In the inference stage, we

Figure 1: Illustration of our proposed AISFG.

 

also build a query for each candidate slot, and the response \(\hat{r}\) is generated in an auto-regressive manner, which means selecting the token as the next token with the highest probability over the vocabulary set at each time step. Note that, although we do not explicitly restrict the response tokens should originate from the input sentence \(x\), AISFG can always do it (but there are exceptions, we show some cases in Section 3.4). Then, the response \(\hat{r}\) is split by commas to recover to the original format \(\hat{y}\) for evaluation.

## 3 Experiments

### Setup

DatasetWe evaluate our method on SNIPS Coucke et al. (2018), a public spoken language understanding dataset which contains 7 domains and 39 slots. To simulate the cross-domain scenarios, we choose one domain as the target domain for test and the left six domains as the source domains for training following the setup of Liu et al. (2020).

However, domains in SNIPS are not completely independent with each other. To achieve a real cross-domain scenario, we use another commonly used dataset namely ATIS Price (1990) as the target domain to test our model which is trained on SNIPS.

BaselinesWe compare our method against a number of representative baselines as follows:

* **Concept Tagger (CT)** A slot-filling framework proposed by Gobbi et al. (2018), which utilizes original slot descriptions to generalize to unseen slot types.
* **Robust Zero-shot Tagger (RZT)** Besides leveraging slot descriptions like CT, RZT Shah et al. (2019) further introduces examples to improve the robustness of zero-shot slot filling.
* **Coarse-to-fine Approach (Coach)** Coach Liu et al. (2020) is a two-step coarse-to-fine model for slot-filling, which performs coarse-grained BIO labeling task in the first step and performs fine-grained slot type classification task in the second step. It also encodes slot descriptions to help recognize unseen slot types.
* **QA-driven slot filling (QASF)** QASF Du et al. (2021) uses a linguistically motivated question generation strategy for converting slot descriptions and example values into natural questions and solves the slot filling by extracting spans from utterances with a span-based QA model.

### Main Results

Cross-Domain Slot FillingThe cross-domain slot filling results are reported in Table 1. In SNIPS dataset, AISFG outperforms the state-of-the-art models (QASF for zero-shot and Coach for few-shot) by 7.47% on the average F1 under zero-shot setting, 13.57% under 20-shot setting and 8.88% under 50-shot setting, which demonstrates the superiority of our method. We train AISFG on SNIPS dataset and test it on the ATIS dataset for simulating a real cross-domain scenario, results are reported in the bottom of Table 1. AISFG consistently outperforms the existing state-of-the-art approaches, especially in zero-shot setting, where AISFG achieves 32.31% F1 score improvement. The significant performance improvement proves that utilizing rich domain information to prompt knowledge in PLMs may be a shortcut for solving data scarcity issue.

Analysis on Seen versus Unseen SlotsWe divide the samples in each target domain into "seen" and "unseen" categories in the SNIPS dataset for further understanding the transferring ability of our model. Following Liu et al. (2020), an example is categorized as "unseen" as long as the slot does not

\begin{table}
\begin{tabular}{l|l|c c c c|c c c c|c c c c} \hline Corpus & Training Setting & \multicolumn{6}{c|}{Zero-shot} & \multicolumn{6}{c|}{ Few-shot on 20 samples} & \multicolumn{6}{c}{ Few-shot on 50 samples} \\ \cline{2-13}  & Domain\_Model\(\rightarrow\) & CT & RZT & Coach & QASF & AISFG & CT & RZT & Coach & AISFG & CT & RZT & Coach & AISFG \\ \hline \multirow{9}{*}{SNIPS} & AdToPlaylist & 38.82 & 42.77 & 50.90 & **57.57** & 56.20 & 58.36 & 63.18 & 62.76 & **81.64** & 68.69 & 74.89 & 74.68 & **83.51** \\  & BookRestaurant & 27.54 & 30.68 & 34.01 & 48.75 & **65.94** & 45.65 & 50.54 & 65.97 & **78.06** & 54.22 & 54.49 & 74.82 & **84.60** \\  & GetWebather & 46.45 & 50.28 & 50.47 & 61.27 & **67.66** & 54.22 & 58.86 & 67.89 & **82.63** & 63.35 & 58.87 & 79.64 & **83.73** \\  & PlayMusic & 32.86 & 31.32 & 32.01 & 38.54 & **50.12** & 46.35 & 47.20 & 54.04 & **77.59** & 54.32 & 50.20 & 66.38 & **78.79** \\  & RateBook & 14.54 & 16.43 & 22.06 & 36.51 & **41.05** & 64.37 & 63.33 & 74.68 & **79.06** & 76.45 & 76.87 & 84.62 & **92.85** \\  & SearchCreativeWork & 39.79 & 44.45 & 46.65 & 60.82 & **67.46** & 57.83 & 63.39 & 57.19 & **71.95** & 66.33 & 67.81 & 64.56 & **76.00** \\  & SearchScreeningEvent & 13.83 & 12.25 & 26.63 & 27.72 & **35.05** & 48.59 & 49.18 & 67.38 & **73.91** & 70.67 & 74.58 & 83.85 & **91.29** \\ \cline{2-13}  & Average F1 & 30.55 & 32.85 & 37.39 & 47.31 & **54.78** & 53.62 & 56.53 & 64.27 & **77.84** & 64.85 & 66.67 & 75.51 & **84.39** \\ \hline ATIS & AirlineTravel & 2.14 & 2.86 & 1.64 & - & **35.17** & 26.05 & 41.37 & 54.91 & **64.04** & 35.87 & 51.80 & 66.99 & **75.31** \\ \hline \end{tabular}
\end{table}
Table 1: F1-scores (%) on SNIPS and ATIS for different target domains under zero-shot and few-shot learning settings. **Bold** indicates the best results. ‘-’ represents the result is missed in published papers.

 

exist in the remaining six source domains. Otherwise, it is tagged as "seen".

Table 3 shows the average F1 results on seen and unseen slots in target domains under zero-shot and 50-shot settings. From this table, AISFG achieves the best results in both seen and unseen slots, especially on unseen slots in zero-shot scenario, which indicates our method is more effective for transferring knowledge from source to the target domain. Besides, approaches leverage PLMs (_i.e._, QASF and AISFG) have significant advantages over others, which demonstrates utilizing knowledge encoded in PLMs can enhance the transferring ability.

### Ablation Studies

To solve the slot type ambiguity issue and example ambiguity issue, we incorporate domain description and context for example to the query template, respectively. To understand the influence of these components in the query template, we perform ablation studies on the SNIPS dataset and report results in Table 2. In this table, SD, DD, E, EC refers to Slot Description, Domain Description, Example and Example with Context, respectively. For example, "SD+EC" represents the query template is built by domain description plus example with context.

The Effect of Domain DescriptionComparing SD versus SD+DD and SD+EC versus AISFG, we observe that incorporating domain descriptions can improve the performance under few-shot setting but not for zero-shot setting. We conjecture the reason is that, the domain description we used (shown in Table 4) is simply converted from the domain name and contains limited information, which is not enough to provide domain knowledge. However, when some target domain examples are given (_i.e._, few-shot), the meaning of domain descriptions can be enhanced by these training examples. Then, the domain description becomes a domain indicator and can be used to distinguish the slot types shared by different domains, resulting in improved performance.

The Effect of Context ExampleComparing SD versus SD+EC and SD+DD versus AISFG, we observe that incorporating examples with contexts can further boost the performance in both zero-shot and few-shot settings, which demonstrates these examples with contexts are helpful for cross-domain slot filling. Moreover, SD+EC outperforms SD+E in both zero-shot and few-shot settings. Especially, for the zero-shot setting in domain _RateBook_, which contains plenty of ambiguity examples, SD+EC achieves 8.06% improvement over SD+E, which indicates contexts are useful to alleviate example ambiguity.

### Error Analysis and Case Study

To better understand our proposed model, we analyze the error predictions in all cross-domain experiments on SNIPS dataset (Table 1) and categorize them into three types: boundary mistakes, vocabu

\begin{table}
\begin{tabular}{l|c c c c|c c c c c} \hline Training Setting & \multicolumn{7}{c|}{Zero-shot} & \multicolumn{7}{c}{Few-shot on 50 samples} \\ \hline Domain\textbackslash{} Model\(\rightarrow\) & SD & SD+DD & SD+EC & ADSFG & SD & SD+DD & SD+E & SD+EC & AISFG \\ \hline AddToPlaylist & 54.57 & 54.94 & 55.03 & **56.22** & 56.20 & 80.84 & 83.49 & 82.84 & 83.23 & **83.51** \\ BookRestaurant & 63.48 & 63.85 & 62.82 & 65.93 & **65.94** & 81.11 & 83.15 & 83.16 & 83.86 & **84.60** \\ GetWeather & 61.09 & 61.35 & 64.73 & 65.56 & **67.66** & 83.35 & 83.50 & 83.17 & 83.66 & **83.73** \\ PlayMusic & 44.41 & 44.99 & 46.51 & 47.48 & **50.12** & 78.62 & 78.50 & 78.06 & 78.68 & **78.79** \\ RateBook & 30.67 & 30.61 & 33.38 & **41.44** & 41.05 & 92.04 & 92.08 & 91.23 & 92.22 & **92.85** \\ SearchCreativeWork & 65.60 & 64.02 & 66.14 & 66.45 & **67.46** & 75.78 & **76.36** & 74.19 & 74.89 & 76.00 \\ SearchScreeningEvent & 30.33 & 30.18 & 32.81 & 34.61 & **35.05** & 89.53 & 90.25 & 90.49 & 90.82 & **91.29** \\ \hline Average F1 & 50.02 & 49.99 & 51.63 & 53.95 & **54.78** & 83.03 & 83.90 & 83.30 & 83.90 & **84.39** \\ \hline \end{tabular}
\end{table}
Table 2: F1-scores (%) on SNIPS for different target domains under zero-shot and few-shot learning settings. **Bold** indicates the best results.

\begin{table}
\begin{tabular}{l|c c|c c} \hline \multirow{2}{*}{Setting\(\rightarrow\)} & \multicolumn{2}{c|}{0 sample} & \multicolumn{2}{c}{50 samples} \\ \cline{2-5}  & unseen & seen & unseen & seen \\ \hline CT & 3.38 & 37.23 & 52.65 & 65.66 \\ RZT & 2.19 & 40.99 & 50.28 & 61.63 \\ Coach & 9.31 & 46.22 & 68.59 & 74.55 \\ QASF & 41.73 & 56.23 & - & - \\ AISFG & **63.96** & **76.11** & **85.09** & **88.77** \\ \hline \end{tabular}
\end{table}
Table 3: Averaged F1-scores (%) over all target domains on SNIPS dataset for “seen” and “unseen” slots. ‘-’ represents the result is missed in published papers 

lary mistakes and others. Boundary mistakes indicate that AISFG predicts the main body of the gold label while missing/adding other words, which may lead to an inaccurate boundary (top example in Figure 3). Vocabulary mistakes refer to the predictions that i) some words in the gold label are replaced by synonyms (middle example in Figure 3); ii) some words in the gold label are substituted by different tenses (bottom example in Figure 3).

As shown in Figure 2, the most common mistakes are the boundary mistakes. Especially, in 50-shot experiment, the amount of boundary-mistake predictions reaches almost half of all mistakes. We find that the model often struggles to detect the exact same span as the ground truths, as shown in the top example in Figure 3. When the shot number increases from 0 to 20 and from 20 to 50, the amount of boundary mistakes reduces by 36.4% and 22.7%, respectively, indicating that increasing shot number is beneficial for addressing the boundary mistakes.

Vocabulary mistakes such as synonyms or tenses replacement are the least common. Since the proposed AISFG is a generative model, we carry no restriction on what the model produces, therefore it may come with the vocabulary mistakes. However, most of these mistakes can be solved by postprocessing the outcome of the model (_i.e._, comparing the prediction with the original sentence and replacing the wrong words), which is one of the improvements of our future AISFG. Moreover, as shown in Figure 2, increasing shot number is useless for tackling the vocabulary mistakes.

## 4 Related Work

The main challenge of cross-domain slot filling is to handle domain-specific slot types which have few or no supervision signals during the training stage. To handle the unseen slots, previous methods introduce slot descriptions Lee and Jha (2019); Liu et al. (2020); Bapna et al. (2017) and slot examples Guerini et al. (2018); Shah et al. (2019) to capture the semantic relationship between unseen slots and input sentences. Based on these researches, we incorporate more information, like domain description and example context, to boost the performance on unseen slots.

Typically, traditional methods formulate the slot filling task as a token-level classification task Liu et al. (2020); Bapna et al. (2017); Shah et al. (2019); Wang et al. (2021). To leverage the knowledge encoded in pre-trained language models, RCSF Yu et al. (2021) utilizes BERT Devlin et al. (2019) to predict the start/end position of the target slot entity. However, the downstream task (_i.e._, token/position prediction) is inconsistent with the pre-training task (_i.e._, Masked LM and NSP for BERT), which may limit the expressiveness of the PLM Mehri and Eskenazi (2021). To bridge the gap, we treat the slot filling as a generative task and utilize a PLM whose pre-training task is also a generative task.

## 5 Conclusion & Discussion

In this paper, we introduce a novel zero-shot cross-domain slot filling model named AISFG, which can adapt to unseen domains seamlessly with the help of domain and slot description and together with full context examples. Experiments show that our model significantly outperforms existing zero-shot cross-domain slot filling approaches. Moreover, we conduct error analysis and case study to better understand our proposed model and leave solving boundary mistakes and vocabulary mistakes as our future work.

Figure 3: Examples for boundary mistake and vocabulary mistake.

Figure 2: Number of samples against mistake types.

 



## Acknowledgments

This work was partially supported by National Natural Science Foundation under Grants U1936103 and 61921003.

## References

* Bapna et al. (2017) Ankur Bapna, Gokhan Tur, Dilek Hakkani-Tur, and Larry P. Heck. 2017. Towards zero-shot frame semantic parsing for domain scaling. In _Interspeech 2017, 18th Annual Conference of the International Speech Communication Association, Stockholm, Sweden, August 20-24, 2017_, pages 2476-2480. ISCA.
* Coucke et al. (2018) Alice Coucke, Alaa Saade, Adrien Ball, Theodore Bluche, Alexandre Caulier, David Leroy, Clement Doumouro, Thibault Gisselbrecht, Francesco Caltagirone, Thibaut Lavril, Mael Primet, and Joseph Dureau. 2018. Snips voice platform: an embedded spoken language understanding system for private-by-design voice interfaces. _CoRR_, abs/1805.10190.
* Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_, pages 4171-4186, Minneapolis, Minnesota. Association for Computational Linguistics.
* Du et al. (2021) Xinya Du, Luheng He, Qi Li, Dian Yu, Panupong Pasupat, and Yuan Zhang. 2021. QA-driven zero-shot slot filling with weak supervision pretraining. In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers)_, pages 654-664, Online. Association for Computational Linguistics.
* Gobbi et al. (2018) Jacopo Gobbi, Evgeny A. Stepanov, and Giuseppe Riccardi. 2018. Concept tagging for natural language understanding: Two decadelong algorithm development. In _Proceedings of the Fifth Italian Conference on Computational Linguistics (CLiC-it 2018), Torino, Italy, December 10-12, 2018_, volume 2253 of _CEUR Workshop Proceedings_. CEUR-WS.org.
* Guerini et al. (2018) Marco Guerini, Simone Magnolini, Vevake Balaraman, and Bernardo Magnini. 2018. Toward zero-shot entity recognition in task-oriented conversational agents. In _Proceedings of the 19th Annual SIGdial Meeting on Discourse and Dialogue_, pages 317-326, Melbourne, Australia. Association for Computational Linguistics.
* Kim et al. (2019) Kunho Kim, Rahul Jha, Kyle Williams, Alex Marin, and Imed Zitouni. 2019. Slot tagging for task oriented spoken language understanding in human-to-human conversation scenarios. In _Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL)_, pages 757-767, Hong Kong, China. Association for Computational Linguistics.
* Kingma and Ba (2015) Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In _3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings_.
* Lee and Jha (2019) Sungjin Lee and Rahul Jha. 2019. Zero-shot adaptive transfer for conversational language understanding. In _The Thirty-Third AAAI Conference on Artificial Intelligence, 2019_, pages 6642-6649. AAAI Press.
* Lewis et al. (2020) Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_, pages 7871-7880, Online. Association for Computational Linguistics.
* Liu et al. (2020) Zihan Liu, Genta Indra Winata, Peng Xu, and Pascale Fung. 2020. coach: A coarse-to-fine approach for cross-domain slot filling. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_, pages 19-25, Online. Association for Computational Linguistics.
* Louvan and Magnini (2018) Samuel Louvan and Bernardo Magnini. 2018. Exploring named entity recognition as an auxiliary task for slot filling in conversational language understanding. In _Proceedings of the 2018 EMNLP Workshop SCAI: The 2nd International Workshop on Search-Oriented Conversational AI_, pages 74-80, Brussels, Belgium. Association for Computational Linguistics.
* Mehri and Eskenazi (2021) Shikib Mehri and Maxine Eskenazi. 2021. GenSF: Simultaneous adaptation of generative pre-trained models and slot filling. In _Proceedings of the 22nd Annual Meeting of the Special Interest Group on Discourse and Dialogue_, pages 489-498, Singapore and Online. Association for Computational Linguistics.
* Mesnil et al. (2013) Gregoire Mesnil, Xiaodong He, Li Deng, and Yoshua Bengio. 2013. Investigation of recurrent-neural-network architectures and learning methods for spoken language understanding. In _INTERSPEECH 2013, 14th Annual Conference of the International Speech Communication Association, Lyon, France, August 25-29, 2013_, pages 3771-3775. ISCA.
* Price (1990) Patti J. Price. 1990. Evaluation of spoken language systems: the ATIS domain. In _Speech and Natural Language: Proceedings of a Workshop Held at Hidden Valley, Pennsylvania, USA, June 24-27, 1990_. Morgan Kaufmann.
* Shah et al. (2019) Darsh Shah, Raghav Gupta, Amir Fayazi, and Dilek Hakkani-Tur. 2019. Robust zero-shot cross-domainslot filling with example values. In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_, pages 5484-5490, Florence, Italy. Association for Computational Linguistics

Lited by: SS1, SS2.
* L. Wang, X. Li, J. Liu, K. He, Y. Yan, and W. Xu (2021)Bridge to target domain by prototypical contrastive learning and label confusion: re-explore zero-shot learning for slot filling. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, Online and Punta Cana, Dominican Republic, pp. 9474-9480. External Links: Link Cited by: SS3.5.
* K. Yao, G. Zweig, M. Hwang, Y. Shi, and D. Yu (2013)Recurrent neural networks for language understanding. In INTERSPEECH 2013, 14th Annual Conference of the International Speech Communication Association, Lyon, France, August 25-29, 2013, pp. 2524-2528. External Links: Link Cited by: SS1, SS2.
* M. Yu, J. Liu, Y. Chen, J. Xu, and Y. Zhang (2021)Cross-domain slot filling as machine reading comprehension. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21, Virtual Event / Montreal, Canada, 19-27 August 2021, pp. 3992-3998. External Links: Link Cited by: SS1, SS2.
* M. Yu, J. Liu, Y. Chen, J. Xu, and Y. Zhang (2021)Cross-domain slot filling as machine reading comprehension. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI 2021, Virtual Event / Montreal, Canada, 19-27 August 2021, pp. 3992-3998. External Links: Link Cited by: SS1, SS2.

## Appendix A Appendix

### Domain Descriptions

For a fair comparison with QASF, which is based on BERT Du et al. (2021), we instantiate the sequence-to-sequence model as BART-base, which contains a similar parameter size as BERT. Following Liu et al. (2020), we reserve 500 samples in target domain as the validation set and regard the rest as the test set. We use Adam optimizer Kingma and Ba (2015) with a learning rate 2e-5 to fine-tune all parameters. The batch size is set to 16 and the decoding beam search size for BART is set to 2. The early stop strategy with patience of 5 is used to save the best checkpoint based on validation set.

Similar to the baselines, we use F1 score as the evaluation metric. For a given query, the true positive is the prediction with exactly matched entity extraction boundaries.

\begin{table}
\begin{tabular}{l|l} \hline \hline Domain & Domain Description \\ \hline AddToPlaylist & add to playlist \\ BookRestaurant & book restaurant \\ GetWeather & get weather \\ PlayMusic & play music \\ RateBook & rate book \\ SearchCreativeWork & search creative work \\ SearchScreeningEvent & search screening event \\ \hline \hline \end{tabular}
\end{table}
Table 4: Domain descriptions for different domains.

 



### Illustration of Slot Description, Context and Example

\begin{table}
\begin{tabular}{|p{56.9pt}|l|l|l|l|} \hline Domain & Slot & Slot Description & Context & Example \\ \cline{2-4}  & music_slot & music from & music from & music from \\ AddtFloHost & physlot_owner & daylight over & half the album by substantia spractice digital & musician \\ AddtFloHost & entity_name & entity name & in mma going to add live into two junct or purs list of parties & love letter \\ Play/Paz & play/Paz & play/Paz & add this is going to any karyotype brighter & stay happy \\ artist & artist artist & artist & artist & artist \\ \hline \multirow{7}{*}{BookRestaurant} & city & city & male reservation: at the best up in a handful & shanghai \\  & reality & facility & navigation serves drawn with bay chart & baby chair \\  & timeRange & time range & i need a reservation for the chinese food on echo in 10 minutes & in 10 minutes \\  & restaurant_name & restaurant name & iteming up to idle before only my friends & kite \\  & country & county & a gum emerge a reentrant in juveniles for 8 persons & yramalen \\  & continue & equine & it did be traverse to rule out for a small network in weekly & youth \\  & restaurant_type & restaurant type & jed like to leave a table for family & battle \\  & servedly_dish & served dish & find restaurant versus spot not good match a reservation & lbe get \\  & party_size_number & number & make às vice-person reservation an alpine when bar & six person \\  & psi & position & i donnt to reserve a restaurant near hay hotel & nare may hotel \\  & sort & type & get a specially regarded sandwich as apple to cathown & highly repelled \\  & spatial_relation & yeuli resolution & sexual rotation & bedroom a reference fan (10 minutes walk from bee) & 10 minutes wklk \\  & state & rate & we want to put to a browseren in months that stores socialism & cornha \\  & purvy_size_description & period & book able to fixe substantia peer and ledice & sachuation peers and ledice \\ \hline \multirow{7}{GenWebather} & city & city & is the temperature going down to 2 in shanghai & shanghai \\  & state & state & check whether he owtains & controls \\  & timeRange & time range & what is the forecast of Indian in next hall on hour & in neat half on hour \\  & current_location & current location & will it rain in previous local street around 11/10/2023 & present local street \\  & country & country & vuntually what the weather has injectiveness (flight now & Jerusalem \\  & spatial_relation & partial relation & portal relation) & is it going to raise within 10 minutes box distance & within 10 minutes bus distance \\  & progroysmkg_pci & geographic position & work last belt, how could all if he tomorrow & west lake park \\  & condition_temperature & temperature & until it got lower than 2 hours & hotter \\  & condino_description & weather & wall used here by a source stream & slow stem \\ \hline \multirow{7}{PlayMusic} & genre & genre & form a ballist in successive closed music & hillley \\  & music_jdem &music item & fonts one house place old space to pay plagistic & piece \\  & service & service & index than a ballist in science cloud music & increase cloud music \\  & year & year & play their product songs ago 2021 & 2021 \\  & flylight & flightout address you copy My thyrype plagistic & sky happy & sky happy \\  & album & album & playing the album getting ready to year soon & pairing ready \\  & sort & type & play ball yet to view coach chain & shall we talk \\  & artist & artist & artist & artist \\ \hline \multirow{7}{RattBook} & object_part_set_syte_yte & series & i are the speed of point & weight \\  & object_setset & this current & the body returns 5 star & the \\  & mating_value & rating value & the blood skewness = 5 star & 5 \\  & object_name & object name & lessons from machine click is a fine pick for anyone interested in fashion style & losses from mathematic disc \\  & object_type & object type & i am once trial to read robot fractures & horse \\  & timing_unit & string unit & this host-coverage 5 star & star \\  & best_rating & beats rating & the highest rating for this block to 10 & \\ \hline \multirow{7}{SearackCreatureWeak} & object_name & eight name & pair baggett is a bicycle behind himself in such items owned by the eye group & paris baggett \\  & object_type & object type & parish baggett is a battery chain based in south knees owned by the eye group & balicy \\ \hline \multirow{7}{SearackScreeningEvent} & intrRange & time range & the queue starts at half per night & buffet eight pm \\  & movie_spec & movie type & input to new eclipse press book & computer \\  & object_Location_type & reactive pipe & the cartoue theatre is the closest movie house showing green book & movie house \\  & object_Type & object type & show the movie poster of green book & movie poster \\  & location_name & location name & the cross there have the closest movie bounce showing green book & the casto theatre \\  & spatial_relation & spatial relation & the cartoue theatre is the closest movie bovine growing green book & closort \\  & movie_name & movie name & the cartoue theatre is the closest movie house showing green book & green book \\ \hline \end{tabular}
\end{table}
Table 5: Illustration of slot description, context and example.