pwc-archive/papers-with-abstracts huggingface dataset:
The UPC RDF-to-Text System at WebNLG Challenge 2020
Authors: David Bergés, Roser Cantenys, Roger Creus, Oriol Domingo, José A. R. Fonollosa
Date: 2020-12-01 00:00:00
Proceeding: ACL (WebNLG, INLG) 2020 12
Tasks: Machine Translation, Translation

This work describes the end-to-end system architecture presented at WebNLG Challenge 2020. The system follows the traditional Machine Translation (MT) pipeline, based on the Transformer model, applied in most text-totext problems. Our solution is enriched by means of a Back Translation step over the original corpus. Thus, the system directly relies on lexicalise format since the synthetic data limits the use of delexicalisation.
Train Hard, Finetune Easy: Multilingual Denoising for RDF-to-Text Generation
Authors: Zdeněk Kasner, Ondřej Dušek
Date: 2020-12-01 00:00:00
Proceeding: ACL (WebNLG, INLG) 2020 12
Tasks: Denoising, Text Generation

We describe our system for the RDF-to-text generation task of the WebNLG Challenge 2020. We base our approach on the mBART model, which is pre-trained for multilingual denoising. This allows us to use a simple, identical, end-to-end setup for both English and Russian. Requiring minimal taskor languagespecific effort, our model placed in the first third of the leaderboard for English and first or second for Russian on automatic metrics, and it made it into the best or second-best system cluster on human evaluation.
WebNLG 2020 Challenge: Semantic Template Mining for Generating References from RDF
Authors: Trung Tran, Dang Tuan Nguyen
Date: 2020-12-01 00:00:00
Proceeding: ACL (WebNLG, INLG) 2020 12

We present in this paper our mining system for shared task WebNLG Challenge 2020. The general idea of the system is that we generate the semantic template of the output reference from the input RDF XML structure. In the training process, we perform the following subtasks: (i) extract the core information from input RDF; (ii) generate semantic templates from corresponding references. With new RDF XML data, we detect the core information, in turn add the new template into the warehouse and determine the output semantic template. We will evaluate the output natural language references in two processes: automatic and human evaluations. The results of the first tested process show that our system generates the high quality English descriptions from testing RDF XML structures and has a good contribution to the NLG state-of-the-art.
WebNLG Challenge 2020: Language Agnostic Delexicalisation for Multilingual RDF-to-text generation
Authors: Giulio Zhou, Gerasimos Lampouras
Date: 2020-12-01 00:00:00
Proceeding: ACL (WebNLG, INLG) 2020 12
Tasks: Text Generation

This paper presents our submission to the WebNLG Challenge 2020 for the English and Russian RDF-to-text generation tasks. Our first of three submissions is based on Language Agnostic Delexicalisation, a novel delexicalisation method that match values in the input to their occurrences in the corresponding text through comparison of pretrained multilingual embeddings, and employs a character-level post-editing model to inflect words in their correct form during relexicalisation. Our second submission forfeits delexicalisation and uses SentencePiece subwords as basic units. Our third submission combines the previous two by alternating between the output of the delexicalisation-based system when the input contains unseen entities and/or properties and the output of the SentencePiece-based system when the input is seen during training.
Taking Things Personally: Third Person to First Person Rephrasing
Authors: Marcel Granero Moya, Panagiotis Agis Oikonomou Filandras
Date: 2021-11-01 00:00:00
Proceeding: EMNLP (NLP4ConvAI) 2021 11
Tasks: Sentence

The recent advancement of digital assistant technologies has opened new possibilities in the experiences they can provide. One of them is the ability to converse with a persona, e.g. celebrities, famous imaginary characters, etc. This experience requires that the replies are answered from the point of view of the persona, i.e. the first person. Since the facts about characters are typically found expressed in the third person, there is a need to rephrase them to the first person in order for the assistant not to break character and the experience to remain immersive. However, the automatic solution to such a problem is largely unexplored by the community. In this work, we present a new task for NLP: third person to first person rephrasing. We define the task and analyze its major challenges. We create and publish a novel dataset with 3493 human-annotated pairs of celebrity facts in the third person with their rephrased sentence in the first person. Moreover, we propose a transformer-based pipeline that correctly rephrases 92.0% of sentences compared to 77.0% rephrased by a rule-based baseline system.
Few-Shot Intent Classification by Gauging Entailment Relationship Between Utterance and Semantic Label
Authors: Jin Qu, Kazuma Hashimoto, Wenhao Liu, Caiming Xiong, Yingbo Zhou
Date: 2021-11-01 00:00:00
Proceeding: EMNLP (NLP4ConvAI) 2021 11
Tasks: Classification, intent-classification, Intent Classification, Natural Language Inference

Zhang et al. (2020) proposed to formulate few-shot intent classification as natural language inference (NLI) between query utterances and examples in the training set. The method is known as discriminative nearest neighbor classification or DNNC. Inspired by this work, we propose to simplify the NLI-style classification pipeline to be the entailment prediction on the utterance-semantic-label-pair (USLP). The semantic information in the labels can thus been infused into the classification process. Compared with DNNC, our proposed method is more efficient in both training and serving since it is based upon the entailment between query utterance and labels instead of all the training examples. The DNNC method requires more than one example per intent while the USLP approach does not have such constraint. In the 1-shot experiments on the CLINC150 (Larson et al., 2019) dataset, the USLP method outperforms traditional classification approach by >20 points (in-domain ac- curacy). We also find that longer and semantically meaningful labels tend to benefit model performance, however, the benefit shrinks as more training data is available.
Personalized Extractive Summarization Using an Ising Machine Towards Real-time Generation of Efficient and Coherent Dialogue Scenarios
Authors: Hiroaki Takatsu, Takahiro Kashikawa, Koichi Kimura, Ryota Ando, Yoichi Matsuyama
Date: 2021-11-01 00:00:00
Proceeding: EMNLP (NLP4ConvAI) 2021 11
Tasks: Extractive Summarization

We propose a personalized dialogue scenario generation system which transmits efficient and coherent information with a real-time extractive summarization method optimized by an Ising machine. The summarization problem is formulated as a quadratic unconstraint binary optimization (QUBO) problem, which extracts sentences that maximize the sum of the degree of user’s interest in the sentences of documents with the discourse structure of each document and the total utterance time as constraints. To evaluate the proposed method, we constructed a news article corpus with annotations of the discourse structure, users’ profiles, and interests in sentences and topics. The experimental results confirmed that a Digital Annealer, which is a simulated annealing-based Ising machine, can solve our QUBO model in a practical time without violating the constraints using this dataset.
Multilingual Paraphrase Generation For Bootstrapping New Features in Task-Oriented Dialog Systems
Authors: Subhadarshi Panda, Caglar Tirkaz, Tobias Falke, Patrick Lehnen
Date: 2021-11-01 00:00:00
Proceeding: EMNLP (NLP4ConvAI) 2021 11
Tasks: intent-classification, Intent Classification, Multilingual Paraphrase Generation, Paraphrase Generation

The lack of labeled training data for new features is a common problem in rapidly changing real-world dialog systems. As a solution, we propose a multilingual paraphrase generation model that can be used to generate novel utterances for a target feature and target language. The generated utterances can be used to augment existing training data to improve intent classification and slot labeling models. We evaluate the quality of generated utterances using intrinsic evaluation metrics and by conducting downstream evaluation experiments with English as the source language and nine different target languages. Our method shows promise across languages, even in a zero-shot setting where no seed data is available.
Not So Fast, Classifier – Accuracy and Entropy Reduction in Incremental Intent Classification
Authors: Lianna Hrycyk, Alessandra Zarcone, Luzian Hahn
Date: 2021-11-01 00:00:00
Proceeding: EMNLP (NLP4ConvAI) 2021 11
Tasks: Classification, intent-classification, Intent Classification

Incremental intent classification requires the assignment of intent labels to partial utterances. However, partial utterances do not necessarily contain enough information to be mapped to the intent class of their complete utterance (correctly and with a certain degree of confidence). Using the final interpretation as the ground truth to measure a classifier’s accuracy during intent classification of partial utterances is thus problematic. We release inCLINC, a dataset of partial and full utterances with human annotations of plausible intent labels for different portions of each utterance, as an upper (human) baseline for incremental intent classification. We analyse the incremental annotations and propose entropy reduction as a measure of human annotators’ convergence on an interpretation (i.e. intent label). We argue that, when the annotators do not converge to one or a few possible interpretations and yet the classifier already identifies the final intent class early on, it is a sign of overfitting that can be ascribed to artefacts in the dataset.
Collaborative Data Relabeling for Robust and Diverse Voice Apps Recommendation in Intelligent Personal Assistants
Authors: Qian Hu, Thahir Mohamed, Zheng Gao, Xibin Gao, Radhika Arava, Xiyao Ma, Mohamed AbdelHady
Date: 2021-11-01 00:00:00
Proceeding: EMNLP (NLP4ConvAI) 2021 11
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Diversity, Natural Language Understanding, Recommendation Systems, speech-recognition, Speech Recognition

Intelligent personal assistants (IPAs) such as Amazon Alexa, Google Assistant and Apple Siri extend their built-in capabilities by supporting voice apps developed by third-party developers. Sometimes the smart assistant is not able to successfully respond to user voice commands (aka utterances). There are many reasons including automatic speech recognition (ASR) error, natural language understanding (NLU) error, routing utterances to an irrelevant voice app or simply that the user is asking for a capability that is not supported yet. The failure to handle a voice command leads to customer frustration. In this paper, we introduce a fallback skill recommendation system to suggest a voice app to a customer for an unhandled voice command. One of the prominent challenges of developing a skill recommender system for IPAs is partial observation. To solve the partial observation problem, we propose collaborative data relabeling (CDR) method. In addition, CDR also improves the diversity of the recommended skills. We evaluate the proposed method both offline and online. The offline evaluation results show that the proposed system outperforms the baselines. The online A/B testing results show significant gain of customer experience metrics.
Semi-supervised Intent Discovery with Contrastive Learning
Authors: Xiang Shen, Yinge Sun, Yao Zhang, Mani Najmabadi
Date: 2021-11-01 00:00:00
Proceeding: EMNLP (NLP4ConvAI) 2021 11
Tasks: Clustering, Contrastive Learning, Intent Discovery, Natural Language Understanding

User intent discovery is a key step in developing a Natural Language Understanding (NLU) module at the core of any modern Conversational AI system. Typically, human experts review a representative sample of user input data to discover new intents, which is subjective, costly, and error-prone. In this work, we aim to assist the NLU developers by presenting a novel method for discovering new intents at scale given a corpus of utterances. Our method utilizes supervised contrastive learning to leverage information from a domain-relevant, already labeled dataset and identifies new intents in the corpus at hand using unsupervised K-means clustering. Our method outperforms the state-of-the-art by a large margin up to 2% and 13% on two benchmark datasets, measured by clustering accuracy. Furthermore, we apply our method on a large dataset from the travel domain to demonstrate its effectiveness on a real-world use case.
CS-BERT: a pretrained model for customer service dialogues
Authors: Peiyao Wang, Joyce Fang, Julia Reinspach
Date: 2021-11-01 00:00:00
Proceeding: EMNLP (NLP4ConvAI) 2021 11

Large-scale pretrained transformer models have demonstrated state-of-the-art (SOTA) performance in a variety of NLP tasks. Nowadays, numerous pretrained models are available in different model flavors and different languages, and can be easily adapted to one’s downstream task. However, only a limited number of models are available for dialogue tasks, and in particular, goal-oriented dialogue tasks. In addition, the available pretrained models are trained on general domain language, creating a mismatch between the pretraining language and the downstream domain launguage. In this contribution, we present CS-BERT, a BERT model pretrained on millions of dialogues in the customer service domain. We evaluate CS-BERT on several downstream customer service dialogue tasks, and demonstrate that our in-domain pretraining is advantageous compared to other pretrained models in both zero-shot experiments as well as in finetuning experiments, especially in a low-resource data setting.
PLATO-KAG: Unsupervised Knowledge-Grounded Conversation via Joint Modeling
Authors: Xinxian Huang, Huang He, Siqi Bao, Fan Wang, Hua Wu, Haifeng Wang
Date: 2021-11-01 00:00:00
Proceeding: EMNLP (NLP4ConvAI) 2021 11
Tasks: Response Generation

Large-scale conversation models are turning to leveraging external knowledge to improve the factual accuracy in response generation. Considering the infeasibility to annotate the external knowledge for large-scale dialogue corpora, it is desirable to learn the knowledge selection and response generation in an unsupervised manner. In this paper, we propose PLATO-KAG (Knowledge-Augmented Generation), an unsupervised learning approach for end-to-end knowledge-grounded conversation modeling. For each dialogue context, the top-k relevant knowledge elements are selected and then employed in knowledge-grounded response generation. The two components of knowledge selection and response generation are optimized jointly and effectively under a balanced objective. Experimental results on two publicly available datasets validate the superiority of PLATO-KAG.
Personalized Search-based Query Rewrite System for Conversational AI
Authors: Eunah Cho, Ziyan Jiang, Jie Hao, Zheng Chen, Saurabh Gupta, Xing Fan, Chenlei Guo
Date: 2021-11-01 00:00:00
Proceeding: EMNLP (NLP4ConvAI) 2021 11
Tasks: Retrieval

Query rewrite (QR) is an emerging component in conversational AI systems, reducing user defect. User defect is caused by various reasons, such as errors in the spoken dialogue system, users’ slips of the tongue or their abridged language. Many of the user defects stem from personalized factors, such as user’s speech pattern, dialect, or preferences. In this work, we propose a personalized search-based QR framework, which focuses on automatic reduction of user defect. We build a personalized index for each user, which encompasses diverse affinity layers to reflect personal preferences for each user in the conversational AI. Our personalized QR system contains retrieval and ranking layers. Supported by user feedback based learning, training our models does not require hand-annotated data. Experiments on personalized test set showed that our personalized QR system is able to correct systematic and user errors by utilizing phonetic and semantic inputs.
Dialogue Response Generation via Contrastive Latent Representation Learning
Authors: Shuyang Dai, Guoyin Wang, Sunghyun Park, Sungjin Lee
Date: 2021-11-01 00:00:00
Proceeding: EMNLP (NLP4ConvAI) 2021 11
Tasks: Contrastive Learning, Decoder, Representation Learning, Response Generation, Sentence

Large-scale auto-regressive models have achieved great success in dialogue response generation, with the help of Transformer layers. However, these models do not learn a representative latent space of the sentence distribution, making it hard to control the generation. Recent works have tried on learning sentence representations using Transformer-based framework, but do not model the context-response relationship embedded in the dialogue datasets. In this work, we aim to construct a robust sentence representation learning model, that is specifically designed for dialogue response generation, with Transformer-based encoder-decoder structure. An utterance-level contrastive learning is proposed, encoding predictive information in each context representation for its corresponding response. Extensive experiments are conducted to verify the robustness of the proposed representation learning mechanism. By using both reference-based and reference-free evaluation metrics, we provide detailed analysis on the generated sentences, demonstrating the effectiveness of our proposed model.
Think Before You Speak: Learning to Generate Implicit Knowledge for Response Generation by Self-Talk
Authors: Pei Zhou, Behnam Hedayatnia, Karthik Gopalakrishnan, Seokhwan Kim, Jay Pujara, Xiang Ren, Yang Liu, Dilek Hakkani-Tur
Date: 2021-11-01 00:00:00
Proceeding: EMNLP (NLP4ConvAI) 2021 11
Tasks: Common Sense Reasoning, Response Generation

Humans make appropriate responses not only based on previous dialogue utterances but also on implicit background knowledge such as common sense. Although neural response generation models seem to produce human-like responses, they are mostly end-to-end and not generating intermediate grounds between a dialogue history and responses. This work aims to study if and how we can train an RG model that talks with itself to generate implicit knowledge before making responses. We further investigate can such models identify when to generate implicit background knowledge and when it is not necessary. Experimental results show that compared with models that directly generate responses given a dialogue history, self-talk models produce better-quality responses according to human evaluation on grammaticality, coherence, and engagingness. And models that are trained to identify when to self-talk further improves the response quality. Analysis on generated implicit knowledge shows that models mostly use the knowledge appropriately in the responses.
Influence of user personality on dialogue task performance: A case study using a rule-based dialogue system
Authors: Ao Guo, Atsumoto Ohashi, Ryu Hirai, Yuya Chiba, Yuiko Tsunomori, Ryuichiro Higashinaka
Date: 2021-11-01 00:00:00
Proceeding: EMNLP (NLP4ConvAI) 2021 11

Endowing a task-oriented dialogue system with adaptiveness to user personality can greatly help improve the performance of a dialogue task. However, such a dialogue system can be practically challenging to implement, because it is unclear how user personality influences dialogue task performance. To explore the relationship between user personality and dialogue task performance, we enrolled participants via crowdsourcing to first answer specified personality questionnaires and then chat with a dialogue system to accomplish assigned tasks. A rule-based dialogue system on the prevalent Multi-Domain Wizard-of-Oz (MultiWOZ) task was used. A total of 211 participants’ personalities and their 633 dialogues were collected and analyzed. The results revealed that sociable and extroverted people tended to fail the task, whereas neurotic people were more likely to succeed. We extracted features related to user dialogue behaviors and performed further analysis to determine which kind of behavior influences task performance. As a result, we identified that average utterance length and slots per utterance are the key features of dialogue behavior that are highly correlated with both task performance and user personality.
Towards Code-Mixed Hinglish Dialogue Generation
Authors: Vibhav Agarwal, Pooja Rao, Dinesh Babu Jayagopi
Date: 2021-09-01 00:00:00
Proceeding: RANLP 2021 9
Tasks: Dialogue Generation

Code-mixed language plays a crucial role in communication in multilingual societies. Though the recent growth of web users has greatly boosted the use of such mixed languages, the current generation of dialog systems is primarily monolingual. This increase in usage of code-mixed language has prompted dialog systems in a similar language. We present our work in Code-Mixed Dialog Generation, an unexplored task in code-mixed languages, generating utterances in code-mixed language rather than a single language that is more often just English. We present a new synthetic corpus in code-mix for dialogs, CM-DailyDialog, by converting an existing English-only dialog corpus to a mixed Hindi-English corpus. We then propose a baseline approach where we show the effectiveness of using mBART like multilingual sequence-to-sequence transformers for code-mixed dialog generation. Our best performing dialog models can conduct coherent conversations in Hindi-English mixed language as evaluated by human and automatic metrics setting new benchmarks for the Code-Mixed Dialog Generation task.
KMI-Panlingua-IITKGP @SIGTYP2020: Exploring rules and hybrid systems for automatic prediction of typological features
Authors: Ritesh Kumar, Deepak Alok, Akanksha Bansal, Bornini Lahiri, Atul Kr. Ojha
Date: 2020-11-01 00:00:00
Proceeding: EMNLP (SIGTYP) 2020 11
Tasks: Missing Values

This paper enumerates SigTyP 2020 Shared Task on the prediction of typological features as performed by the KMI-Panlingua-IITKGP team. The task entailed the prediction of missing values in a particular language, provided, the name of the language family, its genus, location (in terms of latitude and longitude coordinates and name of the country where it is spoken) and a set of feature-value pair are available. As part of fulfillment of the aforementioned task, the team submitted 3 kinds of system - 2 rule-based and one hybrid system. Of these 3, one rule-based system generated the best performance on the test set. All the systems were ‘constrained’ in the sense that no additional dataset or information, other than those provided by the organisers, was used for developing the systems.
Imputing typological values via phylogenetic inference
Authors: Gerhard Jäger
Date: 2020-11-01 00:00:00
Proceeding: EMNLP (SIGTYP) 2020 11
Tasks: Imputation, Missing Values, Position

This paper describes a workflow to impute missing values in a typological database, a sub- set of the World Atlas of Language Structures (WALS). Using a world-wide phylogeny de- rived from lexical data, the model assumes a phylogenetic continuous time Markov chain governing the evolution of typological val- ues. Data imputation is performed via a Max- imum Likelihood estimation on the basis of this model. As back-off model for languages whose phylogenetic position is unknown, a k- nearest neighbor classification based on geo- graphic distance is performed.