pwc-archive/papers-with-abstracts huggingface dataset:
Towards a Better Understanding of Label Smoothing in Neural Machine Translation
Authors: Yingbo Gao, Weiyue Wang, Christian Herold, Zijian Yang, Hermann Ney
Date: 2020-12-01 00:00:00
Proceeding: Asian Chapter of the Association for Computational Linguistics 2020
Tasks: Machine Translation, Translation

In order to combat overfitting and in pursuit of better generalization, label smoothing is widely applied in modern neural machine translation systems. The core idea is to penalize over-confident outputs and regularize the model so that its outputs do not diverge too much from some prior distribution. While training perplexity generally gets worse, label smoothing is found to consistently improve test performance. In this work, we aim to better understand label smoothing in the context of neural machine translation. Theoretically, we derive and explain exactly what label smoothing is optimizing for. Practically, we conduct extensive experiments by varying which tokens to smooth, tuning the probability mass to be deducted from the true targets and considering different prior distributions. We show that label smoothing is theoretically well-motivated, and by carefully choosing hyperparameters, the practical performance of strong neural machine translation systems can be further improved.
Comparing Probabilistic, Distributional and Transformer-Based Models on Logical Metonymy Interpretation
Authors: Giulia Rambelli, Emmanuele Chersoni, Alessandro Lenci, Philippe Blache, Chu-Ren Huang
Date: 2020-12-01 00:00:00
Proceeding: Asian Chapter of the Association for Computational Linguistics 2020
Tasks: Retrieval

In linguistics and cognitive science, Logical metonymies are defined as type clashes between an event-selecting verb and an entity-denoting noun (e.g. The editor finished the article), which are typically interpreted by inferring a hidden event (e.g. reading) on the basis of contextual cues. This paper tackles the problem of logical metonymy interpretation, that is, the retrieval of the covert event via computational methods. We compare different types of models, including the probabilistic and the distributional ones previously introduced in the literature on the topic. For the first time, we also tested on this task some of the recent Transformer-based models, such as BERT, RoBERTa, XLNet, and GPT-2. Our results show a complex scenario, in which the best Transformer-based models and some traditional distributional models perform very similarly. However, the low performance on some of the testing datasets suggests that logical metonymy is still a challenging phenomenon for computational modeling.
Unsupervised KB-to-Text Generation with Auxiliary Triple Extraction using Dual Learning
Authors: Zihao Fu, Bei Shi, Lidong Bing, Wai Lam
Date: 2020-12-01 00:00:00
Proceeding: Asian Chapter of the Association for Computational Linguistics 2020
Tasks: Text Generation

KB-to-text task aims at generating texts based on the given KB triples. Traditional methods usually map KB triples to sentences via a supervised seq-to-seq model. However, existing annotated datasets are very limited and human labeling is very expensive. In this paper, we propose a method which trains the generation model in a completely unsupervised way with unaligned raw text data and KB triples. Our method exploits a novel dual training framework which leverages the inverse relationship between the KB-to-text generation task and an auxiliary triple extraction task. In our architecture, we reconstruct KB triples or texts via a closed-loop framework via linking a generator and an extractor. Therefore the loss function that accounts for the reconstruction error of KB triples and texts can be used to train the generator and extractor. To resolve the cold start problem in training, we propose a method using a pseudo data generator which generates pseudo texts and KB triples for learning an initial model. To resolve the multiple-triple problem, we design an allocated reinforcement learning component to optimize the reconstruction loss. The experimental results demonstrate that our model can outperform other unsupervised generation methods and close to the bound of supervised methods.
All-in-One: A Deep Attentive Multi-task Learning Framework for Humour, Sarcasm, Offensive, Motivation, and Sentiment on Memes
Authors: Dushyant Singh Chauhan, Dhanush S R, Asif Ekbal, Pushpak Bhattacharyya
Date: 2020-12-01 00:00:00
Proceeding: Asian Chapter of the Association for Computational Linguistics 2020
Tasks: All, Multi-Task Learning, Sarcasm Detection, Sentiment Analysis

In this paper, we aim at learning the relationships and similarities of a variety of tasks, such as humour detection, sarcasm detection, offensive content detection, motivational content detection and sentiment analysis on a somewhat complicated form of information, i.e., memes. We propose a multi-task, multi-modal deep learning framework to solve multiple tasks simultaneously. For multi-tasking, we propose two attention-like mechanisms viz., Inter-task Relationship Module (iTRM) and Inter-class Relationship Module (iCRM). The main motivation of iTRM is to learn the relationship between the tasks to realize how they help each other. In contrast, iCRM develops relations between the different classes of tasks. Finally, representations from both the attentions are concatenated and shared across the five tasks (i.e., humour, sarcasm, offensive, motivational, and sentiment) for multi-tasking. We use the recently released dataset in the Memotion Analysis task @ SemEval 2020, which consists of memes annotated for the classes as mentioned above. Empirical results on Memotion dataset show the efficacy of our proposed approach over the existing state-of-the-art systems (Baseline and SemEval 2020 winner). The evaluation also indicates that the proposed multi-task framework yields better performance over the single-task learning.
Identifying Implicit Quotes for Unsupervised Extractive Summarization of Conversations
Authors: Ryuji Kano, Yasuhide Miura, Tomoki Taniguchi, Tomoko Ohkuma
Date: 2020-12-01 00:00:00
Proceeding: Asian Chapter of the Association for Computational Linguistics 2020
Tasks: Extractive Summarization, Unsupervised Extractive Summarization

We propose Implicit Quote Extractor, an end-to-end unsupervised extractive neural summarization model for conversational texts. When we reply to posts, quotes are used to highlight important part of texts. We aim to extract quoted sentences as summaries. Most replies do not explicitly include quotes, so it is difficult to use quotes as supervision. However, even if it is not explicitly shown, replies always refer to certain parts of texts; we call them implicit quotes. Implicit Quote Extractor aims to extract implicit quotes as summaries. The training task of the model is to predict whether a reply candidate is a true reply to a post. For prediction, the model has to choose a few sentences from the post. To predict accurately, the model learns to extract sentences that replies frequently refer to. We evaluate our model on two email datasets and one social media dataset, and confirm that our model is useful for extractive summarization. We further discuss two topics; one is whether quote extraction is an important factor for summarization, and the other is whether our model can capture salient sentences that conventional methods cannot.
Unsupervised Aspect-Level Sentiment Controllable Style Transfer
Authors: Mukuntha Narayanan Sundararaman, Zishan Ahmad, Asif Ekbal, Pushpak Bhattacharyya
Date: 2020-12-01 00:00:00
Proceeding: Asian Chapter of the Association for Computational Linguistics 2020
Tasks: Aspect-Based Sentiment Analysis, Aspect-Based Sentiment Analysis (ABSA), Decoder, Language Modelling, Sentence, Sentiment Analysis, Style Transfer

Unsupervised style transfer in text has previously been explored through the sentiment transfer task. The task entails inverting the overall sentiment polarity in a given input sentence, while preserving its content. From the Aspect-Based Sentiment Analysis (ABSA) task, we know that multiple sentiment polarities can often be present together in a sentence with multiple aspects. In this paper, the task of aspect-level sentiment controllable style transfer is introduced, where each of the aspect-level sentiments can individually be controlled at the output. To achieve this goal, a BERT-based encoder-decoder architecture with saliency weighted polarity injection is proposed, with unsupervised training strategies, such as ABSA masked-language-modelling. Through both automatic and manual evaluation, we show that the system is successful in controlling aspect-level sentiments.
Chinese Content Scoring: Open-Access Datasets and Features on Different Segmentation Levels
Authors: Yuning Ding, Andrea Horbach, Torsten Zesch
Date: 2020-12-01 00:00:00
Proceeding: Asian Chapter of the Association for Computational Linguistics 2020

In this paper, we analyse the challenges of Chinese content scoring in comparison to English. As a review of prior work for Chinese content scoring shows a lack of open-access data in the field, we present two short-answer data sets for Chinese. The Chinese Educational Short Answers data set (CESA) contains 1800 student answers for five science-related questions. As a second data set, we collected ASAP-ZH with 942 answers by re-using three existing prompts from the ASAP data set. We adapt a state-of-the-art content scoring system for Chinese and evaluate it in several settings on these data sets. Results show that features on lower segmentation levels such as character n-grams tend to have better performance than features on token level.
Analysis of Hierarchical Multi-Content Text Classification Model on B-SHARP Dataset for Early Detection of Alzheimer's Disease
Authors: Renxuan Albert Li, Ihab Hajjar, Felicia Goldstein, Jinho D. Choi
Date: 2020-12-01 00:00:00
Proceeding: Asian Chapter of the Association for Computational Linguistics 2020
Tasks: text-classification, Text Classification

This paper presents a new dataset, B-SHARP, that can be used to develop NLP models for the detection of Mild Cognitive Impairment (MCI) known as an early sign of Alzheimer{'}s disease. Our dataset contains 1-2 min speech segments from 326 human subjects for 3 topics, (1) daily activity, (2) room environment, and (3) picture description, and their transcripts so that a total of 650 speech segments are collected. Given the B-SHARP dataset, several hierarchical text classification models are developed that jointly learn combinatory features across all 3 topics. The best performance of 74.1{\%} is achieved by an ensemble model that adapts 3 types of transformer encoders. To the best of our knowledge, this is the first work that builds deep learning-based text classification models on multiple contents for the detection of MCI.
An Exploratory Study on Multilingual Quality Estimation
Authors: Shuo Sun, Marina Fomicheva, Fr{\'e}d{\'e}ric Blain, Vishrav Chaudhary, Ahmed El-Kishky, Adithya Renduchintala, Francisco Guzm{\'a}n, Lucia Specia
Date: 2020-12-01 00:00:00
Proceeding: Asian Chapter of the Association for Computational Linguistics 2020
Tasks: Machine Translation, Translation

Predicting the quality of machine translation has traditionally been addressed with language-specific models, under the assumption that the quality label distribution or linguistic features exhibit traits that are not shared across languages. An obvious disadvantage of this approach is the need for labelled data for each given language pair. We challenge this assumption by exploring different approaches to multilingual Quality Estimation (QE), including using scores from translation models. We show that these outperform single-language models, particularly in less balanced quality label distributions and low-resource settings. In the extreme case of zero-shot QE, we show that it is possible to accurately predict quality for any given new language from models trained on other languages. Our findings indicate that state-of-the-art neural QE models based on powerful pre-trained representations generalise well across languages, making them more applicable in real-world settings.
English-to-Chinese Transliteration with Phonetic Auxiliary Task
Authors: Yuan He, Shay B. Cohen
Date: 2020-12-01 00:00:00
Proceeding: Asian Chapter of the Association for Computational Linguistics 2020
Tasks: Machine Translation, Multi-Task Learning, NMT, Translation, Transliteration

Approaching named entities transliteration as a Neural Machine Translation (NMT) problem is common practice. While many have applied various NMT techniques to enhance machine transliteration models, few focus on the linguistic features particular to the relevant languages. In this paper, we investigate the effect of incorporating phonetic features for English-to-Chinese transliteration under the multi-task learning (MTL) setting{---}where we define a phonetic auxiliary task aimed to improve the generalization performance of the main transliteration task. In addition to our system, we also release a new English-to-Chinese dataset and propose a novel evaluation metric which considers multiple possible transliterations given a source name. Our results show that the multi-task model achieves similar performance as the previous state of the art with a model of a much smaller size.
Predicting and Using Target Length in Neural Machine Translation
Authors: Zijian Yang, Yingbo Gao, Weiyue Wang, Hermann Ney
Date: 2020-12-01 00:00:00
Proceeding: Asian Chapter of the Association for Computational Linguistics 2020
Tasks: Decoder, Machine Translation, Prediction, Translation

Attention-based encoder-decoder models have achieved great success in neural machine translation tasks. However, the lengths of the target sequences are not explicitly predicted in these models. This work proposes length prediction as an auxiliary task and set up a sub-network to obtain the length information from the encoder. Experimental results show that the length prediction sub-network brings improvements over the strong baseline system and that the predicted length can be used as an alternative to length normalization during decoding.
Grounded PCFG Induction with Images
Authors: Lifeng Jin, William Schuler
Date: 2020-12-01 00:00:00
Proceeding: Asian Chapter of the Association for Computational Linguistics 2020
Tasks: Prepositional Phrase Attachment

Recent work in unsupervised parsing has tried to incorporate visual information into learning, but results suggest that these models need linguistic bias to compete against models that only rely on text. This work proposes grammar induction models which use visual information from images for labeled parsing, and achieve state-of-the-art results on grounded grammar induction on several languages. Results indicate that visual information is especially helpful in languages where high frequency words are more broadly distributed. Comparison between models with and without visual information shows that the grounded models are able to use visual information for proposing noun phrases, gathering useful information from images for unknown words, and achieving better performance at prepositional phrase attachment prediction.
Building Location Embeddings from Physical Trajectories and Textual Representations
Authors: Laura Biester, Carmen Banea, Rada Mihalcea
Date: 2020-12-01 00:00:00
Proceeding: Asian Chapter of the Association for Computational Linguistics 2020

Word embedding methods have become the de-facto way to represent words, having been successfully applied to a wide array of natural language processing tasks. In this paper, we explore the hypothesis that embedding methods can also be effectively used to represent spatial locations. Using a new dataset consisting of the location trajectories of 729 students over a seven month period and text data related to those locations, we implement several strategies to create location embeddings, which we then use to create embeddings of the sequences of locations a student has visited. To identify the surface level properties captured in the representations, we propose a number of probing tasks such as the presence of a specific location in a sequence or the type of activities that take place at a location. We then leverage the representations we generated and employ them in more complex downstream tasks ranging from predicting a student{'}s area of study to a student{'}s depression level, showing the effectiveness of these location embeddings.
Self-Supervised Learning for Pairwise Data Refinement
Authors: Gustavo Hernandez Abrego, Bowen Liang, Wei Wang, Zarana Parekh, Yinfei Yang, YunHsuan Sung
Date: 2020-12-01 00:00:00
Proceeding: Asian Chapter of the Association for Computational Linguistics 2020
Tasks: Machine Translation, Self-Supervised Learning, Translation

Pairwise data automatically constructed from weakly supervised signals has been widely used for training deep learning models. Pairwise datasets such as parallel texts can have uneven quality levels overall, but usually contain data subsets that are more useful as learning examples. We present two methods to refine data that are aimed to obtain that kind of subsets in a self-supervised way. Our methods are based on iteratively training dual-encoder models to compute similarity scores. We evaluate our methods on de-noising parallel texts and training neural machine translation models. We find that: (i) The self-supervised refinement achieves most machine translation gains in the first iteration, but following iterations further improve its intrinsic evaluation. (ii) Machine translations can improve the de-noising performance when combined with selection steps. (iii) Our methods are able to reach the performance of a supervised method. Being entirely self-supervised, our methods are well-suited to handle pairwise data without the need of prior knowledge or human annotations.
Point-of-Interest Oriented Question Answering with Joint Inference of Semantic Matching and Distance Correlation
Authors: Yifei Yuan, Jingbo Zhou, Wai Lam
Date: 2020-12-01 00:00:00
Proceeding: Asian Chapter of the Association for Computational Linguistics 2020
Tasks: Question Answering, Retrieval, Semantic Parsing, TAG

Point-of-Interest (POI) oriented question answering (QA) aims to return a list of POIs given a question issued by a user. Recent advances in intelligent virtual assistants have opened the possibility of engaging the client software more actively in the provision of location-based services, thereby showing great promise for automatic POI retrieval. Some existing QA methods can be adopted on this task such as QA similarity calculation and semantic parsing using pre-defined rules. The returned results, however, are subject to inherent limitations due to the lack of the ability for handling some important POI related information, including tags, location entities, and proximity-related terms (e.g. {``}nearby{''}, {``}close{''}). In this paper, we present a novel deep learning framework integrated with joint inference to capture both tag semantic and geographic correlation between question and POIs. One characteristic of our model is to propose a special cross attention question embedding neural network structure to obtain question-to-POI and POI-to-question information. Besides, we utilize a skewed distribution to simulate the spatial relationship between questions and POIs. By measuring the results offered by the model against existing methods, we demonstrate its robustness and practicability, and supplement our conclusions with empirical evidence.
Leveraging Structured Metadata for Improving Question Answering on the Web
Authors: Xinya Du, Ahmed Hassan Awadallah, Adam Fourney, Robert Sim, Paul Bennett, Claire Cardie
Date: 2020-12-01 00:00:00
Proceeding: Asian Chapter of the Association for Computational Linguistics 2020
Tasks: Question Answering, Reranking

We show that leveraging metadata information from web pages can improve the performance of models for answer passage selection/reranking. We propose a neural passage selection model that leverages metadata information with a fine-grained encoding strategy, which learns the representation for metadata predicates in a hierarchical way. The models are evaluated on the MS MARCO (Nguyen et al., 2016) and Recipe-MARCO datasets. Results show that our models significantly outperform baseline models, which do not incorporate metadata. We also show that the fine-grained encoding{'}s advantage over other strategies for encoding the metadata.
STIL - Simultaneous Slot Filling, Translation, Intent Classification, and Language Identification: Initial Results using mBART on MultiATIS++
Authors: Jack FitzGerald
Date: 2020-12-01 00:00:00
Proceeding: Asian Chapter of the Association for Computational Linguistics 2020
Tasks: Classification, intent-classification, Intent Classification, Language Identification, Natural Language Understanding, slot-filling, Slot Filling, Translation

Slot-filling, Translation, Intent classification, and Language identification, or STIL, is a newly-proposed task for multilingual Natural Language Understanding (NLU). By performing simultaneous slot filling and translation into a single output language (English in this case), some portion of downstream system components can be monolingual, reducing development and maintenance cost. Results are given using the multilingual BART model (Liu et al., 2020) fine-tuned on 7 languages using the MultiATIS++ dataset. When no translation is performed, mBART{'}s performance is comparable to the current state of the art system (Cross-Lingual BERT by Xu et al. (2020)) for the languages tested, with better average intent classification accuracy (96.07{\%} versus 95.50{\%}) but worse average slot F1 (89.87{\%} versus 90.81{\%}). When simultaneous translation is performed, average intent classification accuracy degrades by only 1.7{\%} relative and average slot F1 degrades by only 1.2{\%} relative.
Generating Sports News from Live Commentary: A Chinese Dataset for Sports Game Summarization
Authors: Kuan-Hao Huang, Chen Li, Kai-Wei Chang
Date: 2020-12-01 00:00:00
Proceeding: Asian Chapter of the Association for Computational Linguistics 2020
Tasks: Articles

Sports game summarization focuses on generating news articles from live commentaries. Unlike traditional summarization tasks, the source documents and the target summaries for sports game summarization tasks are written in quite different writing styles. In addition, live commentaries usually contain many named entities, which makes summarizing sports games precisely very challenging. To deeply study this task, we present SportsSum, a Chinese sports game summarization dataset which contains 5,428 soccer games of live commentaries and the corresponding news articles. Additionally, we propose a two-step summarization model consisting of a selector and a rewriter for SportsSum. To evaluate the correctness of generated sports summaries, we design two novel score metrics: name matching score and event matching score. Experimental results show that our model performs better than other summarization baselines on ROUGE scores as well as the two designed scores.