Exploring the Effectiveness of Multi-Lingual Commonsense Knowledge-Aware Open-Domain Dialogue Response Generation (2023)

Abstract: Prior works have shown the promising results of commonsense knowledge-aware models in improving informativeness while reducing the hallucination issue. Nonetheless, prior works often can only use monolingual knowledge whose language is consistent with the dialogue context. Except for a few high-resource languages, such as English and Chinese, most languages suffer from insufficient knowledge issues, especially minority languages. To this end, this work proposes a new task, Multi-Lingual Commonsense Knowledge-Aware Response Generation (MCKRG), which tries to use commonsense knowledge in other languages to enhance the current dialogue generation. Then, we construct a MCKRG dataset MCK-Dialog of seven languages with multiple alignment methods. Finally, we verify the effectiveness of using multi-lingual commonsense knowledge with a proposed MCK-T5 model. Extensive experimental results demonstrate the great potential of using multi-lingual commonsense knowledge in high-resource and low-resource languages. To the best of our knowledge, this work is the first to explore Multi-Lingual Commonsense Knowledge-Aware Response Generation.

## introduction open-domain dialogue response generation is always a tricky problem @xcite because it allows users to converse in whatever they like, and then various knowledge will be involved @xcite . even with the expensive large-scale language models, models still suffer from the noninformative and hallucination problem if they can not access up-to-date and correct knowledge. the non-informative problem @xcite means the generated responses are always boring and generic, such as 'i don't know.'; the hallucination problem @xcite ) means factually incorrect knowledge that violates the real world may be generated; for example, 'iphone 9 is my favorite. @xcite . grounding dialogue response on the commonsense knowledge @xcite @xcite has shown great effectiveness in alleviating the aforementioned two problems. large-scale commonsense knowledge such as conceptnet @xcite can provide various correct and informative real-world facts @xcite . nonetheless, such works often can only leverage the knowledge of the native language. for example, an english system can only access the english knowledge base, and an indonesian system can only access the indonesian knowledge base. this restriction is not problematic for high-resource languages such as chinese and english @xcite because we can easily collect sufficient commonsense knowledge from the internet. however, for many low-resource languages, collecting enough knowledge in their own language is exceptionally challenging. without sufficient knowledge, knowledge-aware models face the problem of making bricks without straw, which would dramatically reduce their real-world practicalities @xcite . given such challenges, this work proposes multi-lingual commonsense knowledge-aware response generation (mckrg), a new task to break the bottleneck that models can only use the knowledge of the native language. the primary challenge of mckrg is the dataset. as of the beginning of this work, only kowow is a largescale english-korean cross-lingual knowledgeaware dataset @xcite . however, kowow is not grounded on commonsense knowledge and only involves two languages. thus, this work constructs a multi-lingual knowledge-aligned conversational dataset mck-dialog with several automatic processes. mck-dialog uses the xpersona of 7 languages as the conversational corpus; hence, mckdialog simultaneously supports english (en), chinese (cn), french (fr), japanese (jp), korean (ko), italian (it), and indonesian (id). as shown in figure 1 , we adopt conceptnet 2 related work knowledge-aware response generation traditional response generation methods @xcite often generates non-informative dialogues @xcite and factually incorrect hallucination information. to this end, knowledgeaware works ground dialogue response on external knowledge @xcite . the knowledge can be gathered from text-based encyclopedia @xcite , commonsense knowledge @xcite @xcite , search engines @xcite , and many others @xcite . although such works have achieved promising results, they often only focus on high-resource english or chinese @xcite . once in the low-resource language scenarios that knowledge is insufficient @xcite , such works will degenerate into traditional non-knowledge-aware works. compared to them, this work explores using multi-lingual commonsense knowledge besides the native language. in addition, although the prior work kowow @xcite learns to use korean-english cross-lingual knowledge, this work is totally different because: 1) we use graph-based commonsense knowledge instead of text-based knowledge; 2) kowow only studies a much simpler two language scenario. ## mck-t5 model to explore the effectiveness of mckrg, we propose a mck-t5 model, which is built upon the multi-lingual mt5-base (582m) @xcite . formally, each instance is denoted as (h i , k i , r i ), where h @xmath0,1 , • • • , x i,n ) consists of n-turns dialogue histories, k @xmath1,j } m consists of m knowledge facts, and r i is the target response. since the backbone mt5 is a plain-text-based encoder-decoder network, we have to convert the structural knowledge facts k i into plain text and then concatenate the result with the dialogue history h i . thus, the input is formatted as follows: training thanks to the multi-lingual nature of mt5, this work considers two training strategies: 1) singleton: we separately train a mck-t5 model for each language; 2) joint: all languages share one same mck-t5 model in a specific scenario. for both strategies, we uniformly adopt the maximum likelihood estimation in training, and the corresponding objective function is defined as: metrics we adopt three representative metrics: 1) es: we first use sentencebert 3 to get the embedding of the ground-truth response and the generated response, and then use cosine function to compute the embedding similarity to estimate the semantic relevance; 2) f1: we use the charf1 to 3 paraphrase-multilingual-minilm-l12-v2 compute the overlapping-based relevance and the precision of the generated response at the same time; 3) b4: we use the 4-gram bleu to estimate the quality of the generated response. besides, we use mean-a to estimate the overall performance among multiple languages. it first computes the local geometric mean score of three metrics in each language and then computes the weighted (the size of the test size is regarded as a weight) mean score over all local scores. mean-e excludes the indonesian. we do not use perplexity because of the notable vocabulary differences among models.