Exploring the Effectiveness of Multi-Lingual Commonsense Knowledge-Aware Open-Domain Dialogue Response Generation
Authors: Sixing Wu
Jiong Yu
Tianshi Che
Yang Zhou
Wei Zhou
Venue: EMNLP 2023 Findings (2023)
Keywords: response generation
dialogue system
commonsense knowledge
multi-lingual

Abstract: Prior works have shown the promising results of commonsense knowledge-aware models in improving informativeness while reducing the hallucination issue. Nonetheless, prior works often can only use monolingual knowledge whose language is consistent with the dialogue context. Except for a few high-resource languages, such as English and Chinese, most languages suffer from insufficient knowledge issues, especially minority languages. To this end, this work proposes a new task, Multi-Lingual Commonsense Knowledge-Aware Response Generation (MCKRG), which tries to use commonsense knowledge in other languages to enhance the current dialogue generation. Then, we construct a MCKRG dataset MCK-Dialog of seven languages with multiple alignment methods. Finally, we verify the effectiveness of using multi-lingual commonsense knowledge with a proposed MCK-T5 model. Extensive experimental results demonstrate the great potential of using multi-lingual commonsense knowledge in high-resource and low-resource languages. To the best of our knowledge, this work is the first to explore Multi-Lingual Commonsense Knowledge-Aware Response Generation.

Exploring the Effectiveness of Multi-Lingual Commonsense Knowledge-Aware Open-Domain Dialogue Response Generation
Sixing W Jiong Yu Tiansi Che Yang Zhou', and Wei Zhou 'National Pilot School of Software, Yunnan University, Kunming, China
2 Engineering Research Center of Cyberspace, Yunnan University, Kunming, China 3 Auburn University, Auburn, Alabama, USA {wusixing, zwei Jeenueyuu.ee edu. cn
Abstract
(Maynez et al., 2020) means factually incorrect knowledge that violates the real world may be gen- crated; for example, 'iPhone 9 is my favorite.'.
Prior works have shown the promising results of commonsense knowledge-aware models in improving informativeness while reducing the hallucination issue. Nonetheless, prior works often can only use monolingual knowledge whose language is consistent with the dialogue context. Except for few high-resouree lan- guages, such English and Chinese, most lan- guages suffer from insufficient knowledge is- sues, especially minority languages. To this end, this work proposes new task, Multi- Lingual Commonsense Knowledge-Awan Re- sponse Ceneration (MCKRG), which tries to use commonsense knowledge in other lan- guages to enhance the current dialogue gen- eration. Then, we construct a MCKRG dataset MCK- "CCK-iialo'' of seven languages with multiple alignment methods. Finally, we verify the effec- tiveness of using multi-lingual commonsense knowledge with a proposed MCK-T5 T5 model. Extensive experimental results demonstrate the great potential of using mingi-lingual common- sense knowledge
Grounding dialogue response on the common- sense knowledge (Wu et al., 2020; Zhou et al., 2022; Cai et al., 2023) has shown great effective- ness in alleviating the aforementioned two prob- lems. Large-scale commonsense knowledge such as ConceptNet (Speer al., 2016) can provide various correct and informative real-world facts (Yu al., 2020). Nonetheless, such works often can only leverage the knowledge of the native lan- guage. For example, an English system can only access the English knowledge base, and an Indone- sian system can only access the Indonesian knowl- edge base. This restriction is not problematic for high-resource languages such as Chinese and En- glish (Kann et al., 2022) because we easily collect sufficient commonsense knowledge from the Internet. However, for many low-resource lan- guages, collecting enough knowledge in their own language is exceptionally challenging. Without suf- ficient knowledge, knowledge-aware models face the problem of making bricks without straw, which would dramatically reduce their real-world practi- calities (Wu al., 2021, 2022b).
and low- resource languages. To the best of our knowl- edge, this work is the first to explore Multi- Lingual Commonsense Knowledge-Awee Aware Re- sponse Generation.
high-resoune
1 Introduction
Given such challenges, this work proposes Multi-Lingual Commonsense Knowledge-Aware Response Generation (MCKRG), new task break the bottleneck that models can only use the knowledge of the native language. The primary challenge of MCKRG is the dataset. As of the beginning of this work, only KoWoW is a large- scale English-Korean cross-lingual knowledge- aware dataset (Kim et al., 2021). However, KoWoW is not grounded on commonsense knowledge and only involves two languages. Thus, this work con- structs multi- multi-lingual knowledge-alignee conversa- tional dataset MCK- -ii-li with several automatic processes. MCK- MCK-Dialog uses the XPersona of 7 lan- guages as the conversational corpus; hence, MCK-
Open-done Open-domain dialogue response generation is al- ways tricky problem (Kann al., 2022) because it allows users to converse in whatever they like, and then various knowledge will be involved (Zhou al., 2022). Even with the expensive large-scale language models, models still suffer from the non- informative and hallucination problem they can not access up-to-date and correct knowledge. The non-in con-informative problem (Li et al., 2016) means the generated responses are always boring and generic, such 'I don't it know.'; the hallucination problem
*The corresponding author.
'Our codes and dataset are released at https: thub. MC.-
Con/Y-NLL/Chatbobn pepoos//reee

---

Translated
Dialogue Context: want to eat roast duck. don't know where delicious?
Entities
RET Entities
TUR Entities
English
Roast, Duck, Roast
#8, 18F-
1e, 98
Chinese Entity Recognition
Entity Recognition
Duck, Delicious
1515,
Cross-Lingual Synorym
Duck
Roast Duck
Transferrad (N98 >
Translation
Translation Synonym
Bird
Knowledge Transferred/ rasfereed//ansslteed ranslated from Chinese
Quanjude
Link-after-
Delicious
Food
Cross-Lingual
Mixed-Code Code Knowledge
Native-Code Knowlodgo
Peking Duck
Synorym
Selecting the Native and Mixed-Code via Soft-Match
Response:
Quanjude's Peking Duck very delicious; often eat it (#E
Dialogue Context
Fact Candidates
MCK-T5 Model
Figure 1: An illustration MCKRG. In this example, English is the native language, and Chinese is used as an example show how both the native-code and mixed-code multi-lingual knowledge.
Dialog simultaneously supports English (En), Chi- nese (Cn), French (Fr), Japanese (Jp), Korean (Ko), Italian (It), and Indonesian (Id).
Extensive experiments verified the effect MCKRG and how to use multi-lingual com- monsense knowledge in dialogue generation.
As shown in Figure/ 1, adopt ConceptNet (Speer al., 2016) as the multi-lingual knowl- edge base. Besides the original ConceptNet, we have proposed two methods to transfet/translate multi-lingual knowledge into the current native lan- guage. Then, we propose one entity recognition method to retrieve entities in the native language and two methods to retrieve entities in other lan- guages. Such entity recognition methods allow model to find the knowledge of other languages easily. Last, we propose Hard Match and Soft Match to efficiently retrieve commonsense facts that can ground the current dialogue session.
Related Work
Knowledge-Aware Response Generation Tra- ditional response generation methods (Sutskever et al., 2014) often generates non-informative di- alogues (Li et al., 2016) and factually incorrect hallucination information. To this end, knowledge- aware works ground dialogue response on external knowledge (Yu et al., 2020). The knowledge can be gathered from text- encyclopedia (Lin et al., 2020a: Zhao et al., 2020), commonsense knowl- edge (Zhang et al., 2020; Wu et al., 2020; Zhou et al., 2022), search engines (Shuster et al., 2022), and many others (Moghe et al., 2020; Li et al., 2022). Although such works have achieved promis- ing results, they often only focus high-resource English Chinese (Kann et al., 2022). Once in the low-resource language scenarios that knowledge insufficient (Wu al., 2022b), such works will degenerate into traditional non-knowledge-awee works. Compared to them, this work explores using multi-lingaut commonsense knowledge be- sides the native language. In addition, although the prior work KoWoW (Kim et al., 2021) learns use Korean-English, cross-lingual knowledge, this work is totally different because: 1) use commonsense knowledge instead of text-based knowledge; 2) KoWoW only studies a much simpler two language scenario.
Experiments use the proposed mT5 based (Xue et al., 2021) MCK-T5 model to explore the ef- fectiveness of MCKRG. Extensive results demon- strate using multi-lingual commonsense knowledge can significantly enrich the performance in low- resource and high-resource languages. Our further analyses also reveal how to use it correctly in the open-domain dialogue response generation. To sum the contribution of this work is three-fold:
1. To our best knowledge, we are the first to use multi- multi-lingual commonsense knowledge to enhance dialogue generation (i.e., MCKRG);
2. We have constructed the first MCKRG bench- mark dataset CCCK-Dialog of languages and propose a MCK-75 as the baseline;

---

Multi-Source Knowledge Prior works try to seek more knowledge from multiple sources en- rich the dialogue generation (Liang et al.. 2021; Wu et al., 2021, 2022b). This work has a similar idea but the difference is also significant because we seek knowledge of other languages but the same knowledge type, while prior works seek knowledge of other knowledge types but the same language.
On
Fy
Jp
Train
16.9K/124K (The following units are also K)
Valid
1/7.3
0.25/1,9|0 100/1/99/.2//2.... 0.3/2.3/0. 1.114 0..48/3.8
Test
1/7.8 0.22/177 0.25/1.9 0.28/2.2 0.3/2.3 0.14/1. 0.48/3.8
Table I: The number of dialogue sessions/utterancess.
Native
Transferred
Translated
#E
#R
#F
#E
#R
#F
#E
#F
1.17M
47
3.28M
127K
31
391K
1.10M
1.59M
Cn
134K
25
367K
99.1K
47
458K
1.23M
47 2.18M
Multi-Lingual It allows a model to handle data of multiple languages in uniform model. Using multi-lingual pre-trained language models such as mBART (Liu et al., 2020) and mT5 (Xue et al., 2021) are convenient way. Although multi- lingual systems have been paid much attention in NLP, only few works in dialogue response genera- tion. MulZDG (Liu al., 2022) and XDailyDialog (Liu et al., 2023) studies the non pon-knowledge-awee multilingual response generation, X-Persona (Lin et al., 2020b) studies the multi-lingual personalized response generation, and ToD (Majewska et al., 2023) studies the multi-lingual task-oriented re- sponse generation. Compared to such works, this work is the first work study the multi-lingual commonsense Knowledge-aware open-domain dia- logue response generation.
Fr
367K
17
2.94M
93.1K
47
354K
1.08M
47 1.93M
84.4K
30
249K
185K
47
960K
1.40M 47 2.39M
3.51K
12
4.45K
34.1K
47
109K
1.45M
2.50M
511K
16
580K
75.8K
47
290K
1.55M 47 2.62M
Id
1.53M 47 2.62M
All 3.95M 47 9.51M
590K
47 2.56M
7.56M 47 15.8M
Table 2: Knowledge abundance statistics of different sub Lub-graphs. #E/R/F represents the number of distinct entities, relations and facts of each sub-graph.
revised by humans in the post-processing, which may change the dialogue or discard a dialogue; thus, as shown Table 1, the valid/test sets are not in parallel across languages. MCK-Dialog keeps all dialogues of XPersona.
3.2 Commonsense Knowledge
We adopt the multilingual commonsense knowl- edge base ConcepiNel (Speer et al., 2016). As reported in Table dances of Native knowledge across languages are different. High-resouree languages have IM+ facts while low-resource languages can only have 4K+ facts, or even no facts. To enrich the abundance of knowledge, we leverage the knowledge of other languages. We create two expanded sub-graphs Transferred and Translated for each specific lan- guage I; besides the original Native sub- -tb-graph: Native: For fact f (Chead, Eret, Ctait) in the ConceptNet, if the language labels of ehead and Ctail are the same 4, then belongs to the Nativer, Transferred: We transfer the knowledge of other languages to Transferred;, by using the cross- lingual synonyms facts. For example, given two English- Chinese synonym facts, (Apple Inc., Syn- onyms, # A)) and (Tim Cook, Synonyms, # b. F 3L), then the English fact (Apple Inc., Related To, Tim Cook) can be transferred to a Chinese fact (3F 2l, Related To, # - Translated: Mainly for the low-resource and zero-resource languages. We employ the NMT 2 to translate fact from the source language to
Cross-Lingual For its primary issue, translation, there are three common paradigms (Huang al., 2023; Zheng et al., 2023): 1) translation-first con- ducts translation before the generation while 2) translation-later conducts generation before the translation. The last 3) end2end does not ex- plicitly conduct translation. This work has ex- plored all paradigms: 1) Sec 3.2/Sec 3.3 uses the cnasslation-fisstlatte methods to transform knowl- edge of other languages to the current native lan- guage; 2) Our MCK-T5 uses mixed-code knowl- edge in an end2end way.
can see that the abun-
3 MCK-Dialog Dataset
3.1 Conversations
XPersona (Lin et al., 2020b) is a multi-lingual ex- tension of the monolingual English Persona-Chan (Zhang et al., 2018), which employed humans to generate dialogues grounded on an assigned per- sona. To obtain conversations in the other six lan- guages, XPersona first used Neural Machine Trans- lation (NMT) API with several -ntte--oo mistake correction processes to construct the paral- lel training sets in other languages efficiently. Be- sides the NMT, the valid set and test set were well-
We use the Microsoft Azure Translation API.

---

the target language. Similar the Transferred, we only translate the head entity and the tail entity. All newly translated facts are placed into the extension Translatedi, for the specific language 4,
Differently, another eecognttion-then-trnnlieee style RfTiaroung first uses the native ERi... identify the entities, then uses the cross-lingual synonym facts (oone-Indonesian) or NMT API (Indonesian) to obtain the entities in Ligt-
Note that although both Transferred; and Trans- latedi, are built with multi-lingual knowledge, they still represent knowledge in the native language 4. Hence. to avoid ambiguity, cross//ulti-lingua is only used tell the language relation between the dialogue context and the original knowledge source. Then, we use native-code to indicate the dialogue and knowledge are in the same language, and mixed-code to indicate the dialogue and knowl- edge may not be in the same language.
3.3.2 Fact Selection
Given a dialogue session H - (X1, ..,, of n turns and the identified entity set (Ei,'.., F.,), we propose a Hard Match and Soft Match to select the facts K that ground the current session H.
Hard Match For a knowledge fact f
(Cheads Cret, Stoit), if Chead E F while Crail E Et, or etail E E1:11 while ehead E Eg, then we assume H is partially grounded on f.
3.3 Knowledge Alignment
We then align the conversations with commonsense knowledge, which is two-stage pipeline: the first Entity Recognition stage identifies entities, and the next Fact Selection stage selects fact candidates.
Soft Match Hard Match method only considers the literal overlap but ignores the semantics, which may involve irrelevant knowledge. To this end, Soft Match first uses SentenceBERT (Reimers and Gurevych, 2020) to compute the fact embedding and the dialogue session embedding; then, Soft Match uses the Consine function to estimate the relevance in the embedding space. Then, facts are selected based on similarity scores.
3.3.1 Entity Recognition
We propose a native-code ER to identify the entities of the native language and two mixed-code functions arr and Rrg to identify the entities of other languages.
3,4 Statistics
Native-Code Given an utterance in language lare, E first tokenizes and lemmatizes (optional) to a n-gram list. Then, all stop-words are removed and we only retain the nouns, verbs, adjectives, and adverbs for all 1-gram candidate entities. Lastly, we filter the candidate entities using the entity set of the sub-graphs of the corresponding language Ire to obtain the native-code entities. In the imple- mentation, we employ several tools to handle each language. In detail, to tokenize a text, we use Natu- ral Language Toolkit (NLTK) package (Bird et al., 2009) for English, French, Italian, and Indonesian; mecab-python3 package for Japanese; pymecab- ko package for Korean; splitting to characters for Chinese. Then, we use the same tool as the part- of-speech (POS) tagger for each non-Chinese lan- guage, and we directly remove all 1-gram Chinese entities. Furthermore, the Simplemma package is used as the lemmatizer for English, French, Italian, and Indonesian.
Knowledge Abundance As mentioned in Sec- tion 3.2, there an obvious abundance gap be- tween high-resource and low-resource languages. To bridge this gap, we transfer/translate facts from other languages to the current language. Table 2 reports the final knowledge abundance of three -rapgrapss. It can be seen that Transferred can significantly expand the knowledge abundance for non-zero-resource languages. By using NMT APT, Translated can significantly enrich the knowledge abundance for not only the non-zero-resource lan- guages but also zero-resource Indonesian. Never- theless, we can not say Translated is better because it would introduce more noise in the translation.
Knowledge Alignment Figure reports the av- erage matched commonsense knowledge facts per dialogue session after the aligmment. Besides, we also report the number of possible related facts, which only require that at least one entity appears in the dialogue session. We first focus on the native- code knowledge (i.e., ER bars). We can see that En. Cn, Fr, and Jp are high-resource languages because they have sufficient matched/related facts even if only use the Naive ConceptNet (i.e., Native-ER).
Mixed-Code
The first TiRis is translation- then-recognition style. It first translates the given utterance into the target language hit using the NMT API and then employs the corresponding E Rg to identify the entities in the target language.

---

The Distribution of Matched Facts
10"
Pollio ER
Nokv TM
hutwo Hot Transaate
1200
The
100
Ang
The Distribution of Related Facts
hatweCC
hatwe/2T
Temakkmeb--
27k
200
The Number
Ang
Figure 2: The distribution of matched and related commonsense facts per session. We separately report the counts of 1) the native-code facts using the ER to recognize entities; 2) the mixed-code facts using the TiR and RTT. each bar, separately report the distribution, where the lighted color is Native, the moderate is Transferred, and the darkest is Translated. Note that Native-ER ER shows the alignment status without using mulli--inguu knowledge.
Unlike them, other languages are low-resource or zero-resource in the Naive ConceptNet. As for the next native-code knowledge Transferred, we can find its contribution (i.e., Transferred-Be -ERr is limited for most languages after alignment. This is because its expansion relies on the cross-lingual synonym facts, which are also limited in ConceptNet. Unlike Transferred, the last native-code knowledge Trans- lated can introduce more knowledge facts, es- pecially for the Indonesian (i.e., Translated-ENi with a higher tolerance for noise. By using all three sub-graphs, cach language can have a simi- lar knowledge abundance of the matched/related fact candidates. Then, we move to the mixed-code knowledge TrR and Rr7 bars). English have the counts among the three bars because only retrieve knowledge of English besides the native language. can be seen that the translation-later style RIT can retrieve more knowledge facts than the translation-fis style TiR.
some extent, irrelevant knowledge still commonly exists, and this alignment is session-level. There- fore, selecting the most relevant knowledge at each turn of the dialogue generation is still needed.
MCK-T5 Model
To explore the effectiveness of MCKRG, we pro- pose MCK-T5 Model, which is built upon the multi-lingual m15-base (582M) (Xue et al., 2021). Formally, each instance is denoted as (H₃ K,, Ri), where H; (X,,,".. Xi,)) consists of n-turns dialogue histories, K; = {fii; consists m knowledge facts, and R, is the target response.
Since the backbone mT5 is plain-text tein-text--aseed based Encoder-Decodeer network. we have to convert the structural knowledge facts K, into plain text and then concatenate the result with the dialogue his- tory Hj. Thus, the input is formatted follows:
Input [Prompt;; Sx (Kt); Qn (H.)) (1)
Overall, it must be noted that all counts illus- trated in Figure are non-repetitive in each lan- guage and the total number of available facts should sum all sub-factors together. Consequently, the av- erage number of matched/related facts per session is about 800 and 89K. Thus, similar to the construc- tion other wwwlledg--alggeed datasets, although the retrieved facts are relevant to the context to
where Prompt; is an instruction code to the task, which is "Generate a response based the fol- lowing terms:'. The next Sxx (K;) linearizes all facts with identifiers; for example, 'Contextual Facts: [0] (Apple Inc., RelatedTo, Steven Jobs); ....: Pihone, IsA, Device)'. The last QH (HH) concatenates all dialogue histories with role labels,

---

just like 'Dialogue History: User: X1; Bot: X2; User: X3.... mT5 does not need explicit labels to indicate the current language. Hence, we directly concatenate the lg((k) and S2r(HH)) even if they are in different languages.
compute the overlapping- relevance and the precision of the generated response at the same time; 3) B4: We use the 4-gram BLEU to estimate the quality of the generated response. Besides, we use Mean-A to estimate the overall performance among multiple languages. It first computes the local geometric mean score of three metrics in each language and then computes the weighted (the size of the test size is regarded over all local scores. Mean-E excludes the In- donesian. We do not use perplexity because of the notable vocabulary differences among models.
Training Thanks to the multi-lingual nature of mT5, this work considers two training strategies: 1) Singleton: We separately train a MCK-TS model for each language; 2) Joint: All languages share one same MCK-T5 model in specific scenario, For both strategies, we uniformly adopt the Max- imum Likelihood Estimation in training, and the corresponding objective function is defined as:
weight) mean score
5.2 Main Results
As shown Table 3, the multi multi-lingual knowledge- aware MCK-75s can significantly outperform the monolingual knowledge-aware MCK-T5s, espe- cially in the low-resource languages (Ko, Id, see Native-ER in Figure 2), demonstrating the effec- tiveness of using the multi-lingual commonsense knowledge. Mcanwhile, due to insufficient na- tive knowledge, MCK-T5+moCsk beaten by the persona-aware m + indicating that monolin- gual commonsense knowledge can not provide in- formation comparable to persona knowledge. Dif- ferently, MCK- MCK-T5+Csk uses multi multi-lingual com- monsense knowledge and the overall performance is comparable to mTS5Pers. This result is incred- ible because the construction of such dialogues is totally grounded on persona knowledge rather than commonsense knowledge, and MCK-T5+CS, +Csk can be applied to more scenarios but mT5+Pers can only be used in the persona-aware scenario. Be- sides, the best XPersona baseline, M-CausalBERT, is worse than mT5+Pers and our MCK-T5+CA Csk. We think the reason is the backbone PLM of M- CausalBERT is worse than mTT5.
L = Σ -log(P(Rana 1144 Ki))
(2)
5 Experiments
5.1 Settings
Model Baselines consist of: 1) T5 and 2) mT5: the monolingual English T5 (t5-large, 738M) (Raf-
fel et al., 2020) and the multi-lingual mT55 (mu5-
base, 582M) (Xue al., 2021); ; 3) BART and 4) mBART: the monolingual English BART (bart- large, 406M) (Lewis et al., 2020) and Chinese happ-bart-large, 407M) (Shao et al., ) BART, as well as the multi-lingual mBART (mbart-large, 61 (M) (Liu et al., 2020); 5) Bert2Bert, 6) M- Bert2Bert, 7) CausalBert and 8) M-CausalBert: models proposed by XPersona (Lin al., 2020b), which use encoder-decoder models and 220- 356M) or autoregressive models (7 and 8, 356M) for response generation. The prefix M- denotes is a multi-lingual model, whereas a monolingual model. Implementation: We use the official codes for 5-8 models. Other models are implemented by Py PyTorc and Transformers. Similar to XPersona, we fine-tune each model with 5 epochs. We use the learning rate le-4 for T-/sss models while le-5 for other models. The mini-bated size is 8, and the gradient accumulation step 4. Experiments were run on Nvidia-Al A100 or RTX3090.
For more fair comparison in the persona sce- nario, Cck-Ts+Csk+Perrs allowed access the persona knowledge. In this case, our MCK-T5 un- doubtedly achieves the best performance in every dimension. This result demonstrates commonsense knowledge is essential in dialogue generation and would not conflict with other knowledge.
Metrics We adopt three representative metrics: 1) ES: We first use SentenceBERT to get the em- bedding of the ground-trutt response and the gen- erated response, and then use cosine function to compute the embedding similarity to estimate the semantic relevance; 2) Fl: we use the CharF1 to
5.3 Native-Code Comparisons
Here, both the dialogue and the knowledge are in the same language. We test configurations: 1) Base uses the Native sub-graph: 2) Ext additionally uses the Transferred aub-graph; and 3) Tri uses Native+ Teiine=+ranfferrd--e Translated sub-graphs We also test the proposed two fact selection methods

---

Know
En
Fr
Ko
ld
Mean
Usage
ES
F1
B4
F8
F1
R4
ES
F1
R4
ES
F1
B4
ES
F1
B4
F$
F1
R4
FS
F1
R4
E
A
BART
None
31.4 65.5
6.6
35.9
29.0
4.8
MBART
None
30.6 63.0
6.0
36.0
29.2
5.3
34,8
63.1
6.0
34.1
36.6
3.5
31.8
41.2
7.1
33.3
61.5
2.7
33.8
65.4
3.9
4.51 4.50
Pers
33.9 64.4
38.5
31.0
36.5
64.7
36.3
37.0
35.5
42.9
8.8
36.7
63.3
36.6
67.7
5.10
5.10
75
Nome
31.5 65.4 6.7
mZ5
None
30,9 65.8
6.7
35.7
29.3
5.3
31.9
63.6
5.9
34.6
38.3
3.6
29.7
42.4
6.1
33.2
64.5
3.8
30.3
64.6
4.1
4.62 4.56
mI5
Pors
33.3 66.8
8,3
38.5
31.3
6.9
35,9
65.6
8.2
36.5
38.2
4.9
35.3
44.0
9.8
37.2
65.7 6.5
34.7
68.1
5.9
5.29
5.27
Rer2Bert
Pors
29.8 63.3
33.3
26.9
3.1
24.0
61.6
32.5
35.0
23.7
38.4
31.6
62.0
23.7
65.1
3,80
3.63
M. Berr2Ben
Pers
32.9 62.9
4.2
36.8
23.1
35.3
63.3 2.7
34.7
29.9
0.4
33.0
38.7
2.9
35.7 60.9
36.0
65.4
1.2
3.54 3,45
CausalBert
Pors
32.2 62.7
7.7
35.0
29.0 4.5
34,1 62.8
5.6
34.1
36.4
3.0
31.8
41.1
6.5
36.2
60.6
4.2
34.0
65.4
4.6
4.69 4.69
M. CausalBert
Pors
32,9 63.3
7.7
36.8
30.0
5.0
35.3
63.7
5.8
34.7
36.5
3.1
33.0
42.1
6.7
35.7
60.7
4.1
36.0
66.2
4.9
4.78 4.80
T5
MoCak:
34.0 65.8
38.3
30.5
35.1
65.4
36,9
38.7
31.5
42.6
7.8
36.0
64.0
30,3
64.6
5.12 4.97
MCK T5
Csk
34.6 65.7 8.6
38.7
31.1 6.1
36.2 65.7
8.1
37.8
38.5
4.7
34.7
43.8
9.3
36.2
64,9 5.3
35.1 67.0
5.4
5.27 5.23
MCK-T5
Porsk+Pers
35.4 67.0
9.6
38.3
30.1
7.0
37.0
66.4
9.3
38.3
38.9
5.7
35.1
43.8
9.6
36.0
65.6
6.6
34,7
68.1
5.9
5.55 5.48
MCK-T5
Cak+Pors
38.2 67.5
10.2
41.5
32.8
7.7
39,0
66.7
9.6
39.5
39.6
5.8
38.6
45.6
11.2
39.3
66.0
8,0
39,2
68.7
7.6
5.87 5.87
Table 3: Model Comparisons. In Know column, Nome uses no knowledge, Pers uses the persona provided by XPersona, Csk uses 30 commonsense knowledge facts retrieved by the best configuration 'Soft Match, Tri, ER+ Joint Training' found in Table 5, MoC'sk only uses the monolingual native commonsense knowledge.
En
Cn
Fr
Jp
Ko
Mean
ES
Fl
B4
ES
FI
B4
ES
B4
ES
B4
ES
F1
F1
B4
F1
B4
A
Vanilla (mm15)
30.9
65.8
6.7
35.7
29.3
5.3
31.9
63.6
5.9
34.6
38.3
3.6
29.7
42.4
6.1
33.2
64.5
38
30.3
64.6
4.1
4.62
4.56
Bose+HMM
33.6 66,0
8.0 37.0 29,9
6.1
34.1
65.3
7.4
35.5
38.3
4.5
31.0
42.7
7.9
34.2
64.9
4.6
5.04 4,90
EM++HM
33.4
65.6
8.2 37.5 30,0 6.1
35,2
65.1
7.9
36.7
38.8
4.7
32.3
42.7
8.1
34.8
65.0
4.8
5.13 4.97
TritHM
33.6 65.9
8.3 37,5 30.4 6.1
34.6
65.2
7.6
36.7
38.7
4.6
33.2
43.5
8.8
35.4
64.9 5.1
33,2 66.5 5.2
5.16 5.10
Base+SF
34.0 65.8
8.3 38.3 30.5
35.1
65.4
7.5
36,9
38.7
31.5
42.6
36.0
61.0 4.7
5.12 4.97
Ext+SK
34.3
65.6
8.2
37.7 30.3 6.2
34.1
65.3
37.9
39.0
34.0
35.3
64.6
5.16 5.00
Tr++SF
34.3 66.0 8.1 38.0
30.6 6.3
34.7
65.3
7.7
37.3
38.9
4.8
33.4
43.6
8.7
35,5
65.2
5.1
33.0
66.4 4.9 5.17 5.10
Table 4: Native-code knowledge comparisons. Models use the Singleton training. Except for Vanilla, HM randomly selects 20 facts for each dialogue session while SF select the top 20 most relevant.
Fin
Cn
Fr
Jp
Ko
Mean
ES
Fl
B4
ES
F1
B4
ES
F1
B4
ES
F1
B4
ES
F1
B4
ES
F
B4
ES
F1
B4
E
A
Vanilla (mT3)
30.9 65.8
6.7
35.7
29.3
5.3
31.9
63.6
5.9
34.6
38.3
3.6
29.7
42.4
6.1
33.2
64.5
3.8
30.3
64.6
4.1
4.62
4.56
TtR
34.3 65.3
8.5
37.2
29.6
5.8
34.3
65.0
7.2
36.3
38.0
3.9
33.2
42.9
8.2
35.5
64.8
4.4
34.1
66.2
5.0
5.09
5.04
RTT
34.3 65.3
8.5
38.0
29.6
5.4
33.9
64.6
6.8
36.5
38.3
3.9
33.3
43.0
8.6
34.5
64.8
4.0
32.0
65.8
4.2
5.08 4.97
ER+TiR
34.3 66.0
8.1
37.9
30.1
5.7
34.4
65.1
7.5
36.8
38.6
4.6
33.6
43.1
8.6
34.7
65.0
4.8
33.3
65.6
5.0
5.12 5.06
ER+RIT
34.3 66.0
8.1
37.8
30.4
6.1
34,1
65.1
7.4
37.1
38.7
4.7
33.5
43
8.7
35.3
64.7
4.6
32.8
66.2
4.8
5.13 5.05
ER+RIT+ RIT+TIR
34.3 66.0
8.1
38.2
30.4
6.2
35.2
64.6
7.6
37.2
38.6
4.8
35.0
43.3
8.8
35.6
64.1
4.8
32.4
65.8
4.6
5.17 5.10
34.4 65.6 8.3
38.1
30.9
6.3
35.9
65.4
7.9
37.4
38.5
4.7
35.0
43.8
8.9
36.4
65.0
5.2
34,9
66.7
5.2
5.22 5.17
Table 5: Mixed-code knowledge comparisons. Except for Vanilla, models use the Finlishh and/or the native Tri sub-graphs, and use SF to select the 20 facts, Models use the Singleton training except for the last uses Joint.
Hard Match (HM) and Soft Match (SF). Table 4 reports the results, and findings are discussed in the following paragraphs.
sian or not. It shows that using knowledge of other languages an effective way to alleviate the knowl- edge shortage in the native language in both the high-resource and low-resource scenarios.
The necessity of knowledge. Compared to the non-knowledge Vanilla, all knowledge-awee mod- els have achieved notable gains. It clearly indicates that I) commonsense knowledge is necessary for open-domain dialogue response generation, and 2) our commonsense knowledge construction and alignment methods are least not ineffective.
The necessity of semantic-driven alignment. Soft-Match can further consider the semantics dur- ing the alignment. The results have indicated this necessity because SF is always better than HM. Meanwhile, the performance gain of SF becomes weaker when using all three sub- sub-graphs. This is because we only adopt the non-fine-tuned Sentence- BERT. We believe the more advanced fine-tuned ranker, such as cross/poly- 2019; Humeau et al., 2019; Wu et al., 2022a), can take a step further and leave this as future work.
The necessity of native-code multi-lingun knowledge. Besides the Native, Ext and Tri use the knowledge transferee//ranslated from other lan- guages. We can find Ext is better than Base and Tri is the best no matter considering the Indone-
(Urbanek et al.,

---

Chinese Case 1
my Hello, do you have any vegan lunch recipes?
w/o, Csk
Response:
M K 1972? Yes, like vegetarian lunch. What about you?
w/ Chinese Csk
(0) (8 good looking, Causes, #33 like); (1) (T lunch, PartOf, 9 life);
Rexponse: R ER #e
FRUE? Yes, like vegetarian lunch. What about you?
(0) (animal, RelatedTo, horse); (1) (brown, IsA, color); (2) (brown, MannerOF, cook);
W/R English Csk
(3) (color, RelatedTo, set); ... (19) (vegan, ISA, vegetarian)
Response: 2 f2 & 1 X 2 1hUE? Yes, I like Vegetarianism. What about you?
(0) (38.2 dinner, RelatedTo, #3X like): (I) (1222 favourite, DerivedFrom, M like) ... (18) (vegetarianism, RelatedTo, animal) (22)(vegtaarian, RelatedTo, vegun)
w/ Multi- Multi-lingual Csk
Response:
No, I'm not a Vegetarianism. but like cooking.
English Case w/ English Csk
Hello, you have any vegan lunch recipes?
(0) (eat, Related dinner); ... (8Xanimal, Desires, cat): (9)lovve animal, HasLanttubeeeett cat);
Response No, don't. love animals. Do you?
w/ Multi-lingual Csk
(0) (enjoy, CausesDesire, cat); (1) (dinner, CapablcOf, cat); ...(()) (cook, MotivatedByGioal cat); Response No. I don't t. Do you like to cook?
Table 6: Case Study. We can only list a few relevant facts because of the space. entities are the translated entities.
5.4 Mixed-Code Comparisons
5.5 Case study
Table 6 has shown two real cases sampled from our MCK-Dialog dataset. In the first Chinese case, four responses are generated with different knowledge configurations. The first w/o. esk does not use any knowledge; thus, the response is just simple repetition. The second w/. Chinese esk only uses Chinese knowledge in the original ConceptNet. We find there is no enough relevant knowledge facts that can be used. Thus, the response is unchanged. The next w/ TIR English esk only uses the En- glish knowledge in the ConceptNet via the TiR and knows what is vegan. However, the information is still not enough. The last w/ Multi-lingual esk uses the mixed-code multi- multi-lingual knowledge; thus, the response is most relevant and informative. In the next case, we select an English case that is par- allel to the previous Chinese case. Unlike the first case, vegan-related fact has appeared because 1) there are too many general English facts, and our Soft Match is difficult to put vegan-related facts at the top; 2) we only use facts of the native language and English in this experiment, this makes English the least profitable of all languages. Nevertheless, the generated dialogues are still acceptable.
We also consider using the knowledge in other lan- guages without transferring or translating.
English Only We assume only English knowl- edge is available. For other languages, to use the cross- longull English knowledge in a mixed-code way, TIR first translates dialogues to English and then recognizes the corresponding English enti- ties; on the contrary, RIT first recognizes the native language entities and then translates them to En- glish. As reported in Table 5, we can find that both TiR and RrT can significantly outperform the non- knowledge Vanilla, demonstrating 1) knowledge can be directly transferred from one knowledge to another knowledge in the open-domain dialogue response generation; 2) comparing such two mod- els, TtR is a little better. We think the reason is the translation-firg TıR can reduce the error distor- tion between the entity recognition process and the knowledge alignment process.
English Plus Then, assume commonsense sub-graphs of both the native language knowledge and English are available. We can find that the cor- responding ER+TIR, ER+RIT, and ER+RITTTIR are better than the English-only TtR and RiT, showing more knowledge is better. Compared to ER+RIT+TIR, the last ER+RIT+TIR+Joon further uses the Joint Training strategy to train all instances of different languages in a uniform model. It can be seen that performance has increased significantly, indicating the knowledge of a specific language can be implicitly transferred to another language via joint training and a multi mintill backbone lan- guage model.
6 Future Work and Conclusion
6.1 Future Work
This work focuses on proposing the task and con- structing the dataset/baselines. Based on the foun- dation, there can be some new research topics: 1) Knowledge Selection: As mentioned, there are 800/89K+ matched/related facts in each dialogue session. Obviously, a model is difficult to handle such a large number of facts, and a response would not involve so many facts. Thus, how to select

---

proper facts are important. 2) Knowledge Conflicts:
Human Evaluation This work does not conduct human evaluation. The reason is two-fold: 1) This work involves seven different languages, pos- ing great challenges in employing volunteers, de- signing the human evaluation process and quality control; 2) We have conducted an early human eval- uation for English and Chinese dialogues, we find this evaluation may not have enough persuasive- ness. Non-English conversations from English, and their texts are quite different from human-written texts, even though they have been revised by humans (Lin et al., 2020b).
Using knowledge from multiple sources and languages simultaneously would inevitably lead to knowledge conflicts, such as translation errors, repetitions across languages, and many others. Eliminating these conflicts is necessary for further improving efficiency.
6.2 Conclusion
translated
Prior commonsense knowledge-aware dialogue re- sponse generation works have achieved promising results. However, they only study high-resource languages such as English and Chinese. Differently, this paper focuses on both the high-resource and low-resource languages. Hence, we first propose a new Multi-Lingual Commonsense Knowledge- Aware Response Generation (MCKRG) task, which tries to commonsense knowledge in other lan- guages enhance dialogue generation in the cur- rent language. Then, we construct a dataset MCK- Dialog of seven languages and a baseline model MCK-75. Extensive experimental results demon- strate the great potential of using multi-lingual com- monsense knowledge in high-resource and low- resource languages. To our best knowledge, this work is the first to explore MCKRG.
Ethical Consideration
This paper studies widely studied topic, namely, open-domain dialogue response generation, and open-released resources. We do not intro- duce any new ethical statement/consideration in this work. Consequently, we think this work has no additional ethical considerations compared to previous work from the technical perspective.
References
Steven Bird, Ewan Klein, and Edward Loper. 2009. Nat- ural language processing with Python: analyzing text with the natural language toolkit. " "O O'Reilly Media, Inc.".
Acknowledgement
This work is supported in part by the National Natural Science Foundation China under Grant 62162067 and 62101480, Research and Applica- tion of Object detection based on Artificial Intelli- gence, in part by the Yunnan Province expert work- stations under Grant AF1505
Hua Cai, Xuli Shen, Qing Xu, Weilin Shen. Xiaomei Wang, Weifeng Ge, Xiaoqing Zheng, and Xiangyang Xue. 2023. Improving empathetic dialogue genera- tion by dynamically infusing commonsense knowl- edge. CoRR, abs/2306.046. 04667777.
Zhiqi Huang. Puxuan Yu, and James Allan, 2023. Im- proving cross- lingull information retrieval on low- resource languages via optimal transport distillation. In Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining, WSDM '23, page 1048-1056, New York, NY, USA. Associa- tion for Computing Machinery.
Limitation
This work studies the multi-lingual commonsense knowledge-aware dialogue response generation. Besides the future work mentioned in Section 6.1, there are still some limitations in the work.
Samuel Humeau, Kurt Shuster, Marie-Anne Lachaux, and Jason Weston. 2019. Poly-encoders: Trans- former architectures and pre-training strategies for fast and accurate multi-sentence scoring. arXiv: Computation and Language.
Benchmark Dataset MCK-Dialog only targets to be a benchmark dataset because the construction mainly relied on automatic methods, which may im- pact the quality besides the English dialogues. This can be attributed to the backbone conversational corpus XPersona built in similarly automatic still believe that building
Katharina Kann, Abteen Ebrahimi, Joewie Koh, Shiran Dudy, and Alessandro Roncone. 2022. Open-domain dialogue generation: What and should next. In Proceedings of the Work- shop on NLP for Conversational Al, pages 148-165, Dublin, Ireland. Association for Computational Lin- guistics.
do, cannot do,
way. Nevertheless, benchmark datasets in an automated manner still necessary for multi-lingual studies, as the cost of manual construction is too high, especially when the number of languages studied increases.
San Kim, Jin Yea Jang, Minyoung Jung, and Saim Shin. 2021. A model of cross-lingual www.eede-groundee

---

response generation for open open-domain dialogue sys- tems. In Findings of the Association for Computa- rional Linguistics: EMNLP 2021, pages 352-365, Punta Cana, Dominican Republic. Association for Computational Linguistics.
Yongkang Liu, Shi Feng, Daling Wang, and Yifei Zhang. 2022. Mulzdg: Multilingual switchiin frame- work for zero-shot dialogue generation. In Proceed- ings of the 29th International Conference on Compu- tational Linguistics, COLING 2022, Gyeongju. Re- public of Korea, October 12-17, 2022, pages 648- 659. International Committee on Computational Lin- guistics.
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. pre-training
BART: Denoising for natural language generation, translation, and com- prehension. In Proceedings of the 58th Annual Meet- ing of the Association for Computational Linguistics, pages 7871-7880, Online, Association for Computa- tional Linguistics.
Zeming Liu, Ping Nie, Jie Cai, Haifeng Wang. Zheng- Yu Niu, Peng Zhang, Mrinmaya Sachan, and Kaiping Peng. 2023. XDailyDialog: A multilingual paral- lel dialogue corpus. In Proceedings of the 61 An- nual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pages 12240- 12253, Toronto, Canada. Association for Computa- tional Linguisties.
Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao,
and Bill Dolan. 2016. A enversiy--prmooiig ob- jective function for neural conversation models. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computa- tional L Linguistics: Human Language Technologies, pages I 110-119, San Diego, California. Association for Computational Linguistics.
Olga Majewska, Evgeniia Razumovskaia, Edoardo M. Ponti, Ivan Vulić, and Anna Korhonen. 2023, Cross- Lingual Dialogue Dataset Creation via Outline-Based Generation. Transactions of the Association for Com- putationaa Linguistics, 1:139-156.
Yu Li, Baolin Peng, Yelong Shen, Yi Mao, Lars Li- den, Zhou Yu, and Jianfeng Gao. 2022. Knowledge- grounded dialogue generation with unified knowl- edge representation. In Proceedings of the 2022 Con- ference of the North American Chapter of the Asso- ciation for Computational Linguisties: Human Lan- guage Technologies, pages 206-218, Seattle, United States, Association for Computational Linguistics.
Joshua Maynez, Shashi Narayan, Bernd Bohnet, and Ryan McDonald. 2020. On faithfulness and factu- ality in abstractive summarization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1906-1919, On- line. Association for Computational Linguistics.
Nikita Moghe, Priyesh Vijayan, Balaraman Ravindran. and Mitesh M. Khapra. 2020. On incorporating struc- tural information to improve dialogue response gen- eration. In Proceedings of the 2nd Workshop on Natural Language Processing for Conversational AT, pages 11-2, Online. Association for Computational Linguistics.
Yunlong I Liang, Fandong Meng, Ying Zhang, Yufeng Chen, Jinan Xu, and Jie Zhou. 2021. Infusing multi- source knowledge with heterogeneous graph neural network for emotional conversation generation. In Thirty-Fifth AAAI Conference on Artificial Intelli- gence, AAAI 2021, Thirty-Thire Conference on In- novative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Ad- vances in Artificial Intelligence, EAAI 2021, Vir- tual Event, February 2-9, 2027, pages 13343-13352. AAAT Press.
Colin Raffel, Noam Shazeer, Adam Roberts, Kather- ine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu, 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(140):1-67.
Xiexiong Lin, Weiyu Jian, Jianshan He, Taifeng Wang, and Wei Chu. 2020a. Generating informative con- versational response using recurrent knowledge- interaction and knowledge-copy. In Proceedings of the 58th Annual Meeting of the Association for Com- putational Linguistics, ACL 2020, Online, July 5-10. 2020, pages 41-52.
Nils Reimers and Tryna Gurevych. 2020. Making monolingual sentence embeddings multilingual us- ing knowledge distillation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics.
Zhaojiang Lin, Zihan Liu, Genta Indra Winata, Samuel Cahyawijaya, Andrea Madotto, Yejin Bang. Etsuko Ishii, and Pascale Fung. 2020b. Xpersona: Eval- uating multilingual personalized chatbot. CoRR, abs/2003.07556..
Yunfan Shao, Zhichao Geng, Yitao Liu, Junqi Dai, Fei Yang, Li Zhe, Hujun Bao, and Xipeng Qiu. 2021. CPT: A pre trained unbalanced transformer for both chinese language understanding and generation. CoRR, abs/2109.05729,
Yinhan Liu, Jiatao Gu, Naman Goyal, Xian Li. Sergey Edunov, Marjan Ghazvininejad, Mike Lewis. and Luke Zettlemoyer. 2020. Multilingual Denoising Pre-training for Neural Machine Translation. Trans- actions of the Association for Computational Linguis- fics, 8:726-742.
Kurt Shuster. Mojtaba Komeili, Leonard Adolphs. Stephen Roller, Arthur Szlam, and Jason Weston. 2022. Language models that seek for knowledge; Modular search & generation for dialogue and prompt completion. CoRR, abs/2203. 13224.

---

Robyn Speer, Joshua Chin, and Catherine Havasi. 2016. Conceptnet 5.3: An open multilingual graph of gen- eral knowledge. CoRR, htss/612.033955
A survey of knowledge-enhaneee text generation. CoRR, abs/2010.04389.
Houyu 7hang, 7henghao Liu, Chenyan Xiong, and Zhiyuan Liu. 2020. Grounded conversation genera- tion as guided traverses in commonsense knowledge graphs. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACI. 2020, Online, July 5-10, 2020, pages 2031-2044
Ilya Sutskever, Oriol Vinyals, and Quoe V. Lc. 2014, Sequence to sequence learning with neural networks. In Advances Neural Information Processing Sys- tems 27: Annual Conference on Neural Information Processing Systems 2014, December 8-13 2014, Mon- Ireal. Quebec, Canada, pages 3104-3112.
Saizheng Zhang, Emily Dinan, Jack Urbanek. Arthur Szlam, Douwe Kiela, and Jason Weston. 2018. Per- sonalizing dialogue agents: have dog, do you have pets too? In Proceedings of the 56th Annual Meeting of the Association for Computational Lin- guistics (Volume I: Long Papers), pages 2204-2213, Melbourne, Australia, Association for Computational Linguistics.
Jack Urhanek, Angela Fan, Siddharth Karamcheti, Saachi Jain, Samuel Humeau, Emily Dinan, Tim Rocktäschel, Douwe Kiela, Arthur Szlam, and Ja- Weston. 2019. Learning speak and act in fantasy text adventure game, empirical methods in natural language processing.
Sixing Wu, Ying Li, Minghui Wang, Dawei Zhang, Yang Zhou, and Zhonghai Wu. 2021. More is bet- ter: Enhancing open- ppen-domain dialogue generation via multi-source source heterogeneous knowledge. In Proceed- ings of the 2021 Conference on Empirical Methods Natural Language Processing, EMNLP 2021, Vir- tual Event/ Punta Cana, Dominican Republic, 7-11 November, 202], pages 2286-2301 Association for Computational Linguistics.
Xueliang Zhao, Wei Wu, Chongyang Tao, Can Xu, Dongyan Zhao, and Rui Yan. 2020. Low-resource knowledge grounded dialogue generation. 8th International Conference on Learning Representa- tions, ICLR 2020, Addis Ababa, Ethiopia, April 26- 30, 2020.
Shaohui Zheng, Zhixu Li, Jiaan Wang, Jianfeng Qu, An Liu, Lei Zhao, and Zhigang Chen. 2023. Long- document cross- pooss-lingual summarization. In Proceed- ings of the Sixteenth ACM International Conference on Web Search and Data Mining, WSDM '23, page 1084-1092, New York, NY, USA. Association for Computing Machinery.
Sixing Wu, Ying Li, Ping Xue, Dawei Zhang, and Zhonghai Wu. 2022a. Section-aware commonsense knowledge-grounddd dialogue generation with pre- trained language model. In Proceedings of the 29th International Conference on Computational Linguis- fics, COLING 2022, Gyeongju, Republic of Korea, October 12-17, 2022, pages 521-531. International Committee on Computational Linguistics.
Pei Zhou, Karthik Gopalakrishnan, Behnam Hedayatnia, Seokhwan Kim, Jay Pujara, Xiang Ren, Yang Liu. and Dilek Hakkani-Tur. 2022. Think before you speak: Explicitly generating implicit commonsense knowledge for response generation. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pages 1237-1252, Dublin, Ireland. Association for Computational Linguistics.
Sixing Wu, Ying Li. Dawei Zhang, Yang Zhou, and Zhonghai Wu. 2020. Diverse and informative di- alogue generation with context-specific common- sense knowledge awareness. In Proceedings of the 58th Annual Meeting of the Association for Compu- tational Linguistics, ACI. 2020, Online, July 5-10, 2020. pages 5811-5820. Association for Computa- tional Linguistics.
Sixing Wu, Minghui Wang, Ying Li, Dawei Zhang, and Zhonghai Wu. 2022b. Improving the applicability of knowledge-enhanee enhanced dialogue generation systems by using heterogeneous knowledge from multiple sources. In WSDM '22: The Fifteenth ACM Interna- tional Conference on Web Search and Data Mining, Virtual Event / Tempe, AZ, USA, February 21- 25, 2022, pages 11 1149-1157, 1157, ACM.
Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. 2021. m55 A massively multilingual pre-trained text-to-text transformer. In Proceedings of the 2021 Conference of the North American Chap- ter of the Association for Computational Linguistics: Human Language Technologies, pages 483-498, On- line. Association for Computational Linguistics.
Wenhao Yu, Chenguang Zhu, Zaitang Li, Zhiting Hu, Qingyun Wang, Heng Ji, and Meng Jiang. 2020.


Reviews:
[{"review_id": "t7AChlARHQ", "reviewer": "Reviewer_cLnM", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- This paper presents a multilingual dialogue dataset, known as MCK-Dialog, which aims to enhance dialogue systems of low-resource languages using commonsense knowledge from high-resource knowledge.\n- The paper introduces MCK-T5 as the baseline model ,and conducts a comprehensive analysis of the effects of each design element on the overall performance.", "reasons_to_reject": "- There is no comparison with translate-test baseline, which translates dialogue to English, generates a response using an English system, and then translates back to the original language. This baseline is straightforward (only use off-the-shelf English system and NMT system) and has been proven to be one of the most competitive approach in various multilingual NLP tasks.\n\n- Table 5 illustrates that combining knowledge from both English and the native language only slightly outperforms the English-only variants. In terms of TtR and RtT, the average improvement in overall performance is a marginal 0.03% and 0.05%, respectively. Therefore, based on these results, the argument for using multiple languages instead of solely relying on English, a high-resource language, is less compelling.\n\n- No human evaluation. Many studies have demonstrated a discrepancy between automatic evaluation results and human preferences on dialogue generation tasks.\n\n- (Minor) As mentioned in section 3.3.2, the proposed system rely on knowledge from two languages, i.e., the native language and English. Consequently, it might be more accurate to describe the task as a \"cross-lingual\" knowledge-aware dialogue, rather than \"multilingual\".", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper is the first to utilize multi-lingual commonsense knowledge for enhancing dialogue generation. It constructs a dataset in 7 languages through translation, and proposes a novel baseline model MCK-T5 based on mT5. Evaluation of this model on the newly collected dataset reveals that the incorporation of knowledge from high-resource languages can significantly enhance the dialogue systems for low-resource languages.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "bH2lAvn9ih", "reviewer": "Reviewer_wvbY", "summary": "", "questions": "What source language is used for the translated knowledge facts? Is it a mix of all other available languages, or is it just English like in the alignment step? Do you perform any deduplication of translated facts from different source languages?\n\nCan you explain why the number of Native-RtT facts is so high for the non-English languages? It makes sense that it is higher than Native-ER, since the non-English languages have smaller graphs in the original ConceptNet, but why is it so much higher than Native-TtR? Is this a translation noise issue? Section 5.4 mentions that TtR performs better than RtT, even though there are more RtT facts. Is this a knowledge selection issue?\n\nI may have missed this information in the text, but I cannot find what knowledge selector is used in the experiments. Section 3.4 lines 330-331 mention that turn-level knowledge selection is needed because the MCK-Dialog dataset contains facts grounded at the dialogue level. \n\nI am also curious as to why the dataset is grounded at the dialogue level? From Section 3.3, it seems to me that the knowledge alignment approach could produce utterance-level knowledge alignments simply by not aggregating the session entities. This would have the benefit of 1) not requiring a knowledge selection step when experimenting on the response generation step and 2) allowing for the evaluation of multilingual knowledge selectors.", "limitations": "", "rating": 3, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The proposed task is well-motivated; the dataset construction process is sensible, and the authors compare several different options for augmenting the fact triplets and aligning them to the dialogues. The experimental results show that the proposed multilingual approach outperforms monolingual approaches, even in high resource languages.", "reasons_to_reject": "The proposed approaches are straightforward and make use of existing methods (eg. translate-then-retrieve, retrieve-then-translate). However, I think this is fine because the authors perform an extensive analysis comparing these settings in their experimental results.\n\nA minor issue is that the experimental results seem to be confounded by an unknown knowledge selection step (see questions below).\n\nUpdate: My concerns were addressed in the author rebuttal, and I have raised my score accordingly.", "questions_to_authors": "What source language is used for the translated knowledge facts? Is it a mix of all other available languages, or is it just English like in the alignment step? Do you perform any deduplication of translated facts from different source languages?\n\nCan you explain why the number of Native-RtT facts is so high for the non-English languages? It makes sense that it is higher than Native-ER, since the non-English languages have smaller graphs in the original ConceptNet, but why is it so much higher than Native-TtR? Is this a translation noise issue? Section 5.4 mentions that TtR performs better than RtT, even though there are more RtT facts. Is this a knowledge selection issue?\n\nI may have missed this information in the text, but I cannot find what knowledge selector is used in the experiments. Section 3.4 lines 330-331 mention that turn-level knowledge selection is needed because the MCK-Dialog dataset contains facts grounded at the dialogue level. \n\nI am also curious as to why the dataset is grounded at the dialogue level? From Section 3.3, it seems to me that the knowledge alignment approach could produce utterance-level knowledge alignments simply by not aggregating the session entities. This would have the benefit of 1) not requiring a knowledge selection step when experimenting on the response generation step and 2) allowing for the evaluation of multilingual knowledge selectors.", "paper_topic_and_main_contributions": "The paper proposes the new task of multilingual commonsense knowledge-based dialogue response generation. The authors argue that commonsense knowledge is effective in improving dialogue response generation by reducing non-informative and factually incorrect generations. \n\nUsing XPersona and ConceptNet as their base dialogue and commonsense knowledge datasets, the authors note that commonsense knowledge is relatively scarce in languages other than English. They build the MCK-Dialog dataset by augmenting the knowledge triples in each language by 1) transferring triples from other languages using ConceptNet's cross-lingual Synonym relation and 2) translating triples from other languages using NMT. The triples are then aligned to utterances and aggregated by dialogue session.\n\nThe authors then train dialogue response generation models, based on multilingual T5, using knowledge from both the dialogue language and the other six languages in MCK-Dialog. The experimental results show that models using multilingual commonsense knowledge perform on par with baseline models using persona knowledge (despite the base dialogue dataset focusing on personas), and that combining multilingual commonsense knowledge with personas gives the best performance. The authors perform an extensive analysis; other major findings include the fact that triplets transferred and translated from other languages improve performance, even for English and the other high-resource languages, and training one model on all languages performs better than training separate models for different dialogue languages.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "Typos:\nSection 1, line 046: CocneptNet -> ConceptNet\nFigure 1: Brid -> Bird\nSection 2, line 166: generation -> translation \nSection 5.1, line 393: ER -> ES\nSection 5.1, line 408: perpleixty -> perplexity\nSection 5.2, lime 433: CausualBERT -> CausalBERT\nSection 5.5, line 520: no -> not\n\nBoth \"Cn\" and \"Zh\" are used as abbreviations for Chinese; using just one consistently would be more clear for the reader.\n\nOverall there are many awkward phrasings; the paper would benefit from another round of editing."}}, {"review_id": "wi3jqGaoCg", "reviewer": "Reviewer_z5FY", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The paper is well-organized, contains enough information in a limited number of pages, and is easy to understand.\n2. The paper evaluates different variants of the method and most of them achieve better results than baselines. The experiments in the paper are extensive and convincing. The chosen evaluation metrics are presentative, and ablation experiments show the efficacy of the new method.\n3. The paper is one of the first to explore the potential of introducing multi-lingual commonsense graph to dialogue generation.\n4. According to the experiment results, the new method can effectively boost model’s performance in low-resource languages such as Indonesian.", "reasons_to_reject": "1. I did not find any major reasons to reject this paper.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper constructs multi-lingual MCKRG dataset MCK-Dialog and proposes a new model MCK-T5 to flexibly integrate multiple knowledge sources into dialogue generation. Specifically, the new method use commonsense knowledge in other languages to enhance the current dialogue generation. At last, the method is tested on the XPersona dataset and proven to achieve noticeable improvements.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "On line 504, “ER+RtT+TTR” and “ER+RtT+TTR+Joint” should be “ER+RtT+TtR” and “ER+RtT+TtR+Joint”."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes to use multilingual commonsense knowledge to augment dialogue generation. It creates a dataset spanning seven languages through translation and introduces a baseline model, MCK-T5, built upon the foundation of mT5. An evaluation of this model on the curated dataset demonstrates a notable enhancement in dialogue systems for low-resource languages when high-resource language knowledge is integrated.

While the reviewers did express several concerns about the experiments, the majority of these concerns have been adequately addressed in the rebuttal.|This paper proposes to use multilingual commonsense knowledge to augment dialogue generation. It creates a dataset spanning seven languages through translation and introduces a baseline model, MCK-T5, built upon the foundation of mT5. An evaluation of this model on the curated dataset demonstrates a notable enhancement in dialogue systems for low-resource languages when high-resource language knowledge is integrated.

While the reviewers did express several concerns about the experiments, the majority of these concerns have been adequately addressed in the rebuttal.
Decision: Accept-Findings