pwc-archive/papers-with-abstracts huggingface dataset:
A Light Label Denoising Method with the Internal Data Guidance
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Denoising

Samples with incorrect labels are common in datasets, even annotated by humans. Some approaches have been proposed to alleviate the negative impact of mislabeling on the training process by removing erroneous data or reducing their weights. Unlike previous works, this paper introduces a light yet effective denoising method based on the relationship between the samples within the dataset, namely internal guidance. We examine the method on five datasets with mainstream models. The results demonstrate that this light denoising approach can obtain consistent improvement for all the datasets and models.
Diversifying Neural Text Generation with Part-of-Speech Guided Softmax and Sampling
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Diversity, POS, Text Generation

Neural text generation models are likely to suffer from the low-diversity problem. Various decoding strategies and training-based methods have been proposed to promote diversity only by exploiting contextual features, but rarely do they consider incorporating syntactic structure clues. In this work, we propose using linguistic annotation, i.e., part-of-speech (POS), to guide the text generation. In detail, we introduce POS Guided Softmax to explicitly model two posterior probabilities: (i) next-POS, and (ii) next-token from the vocabulary of the target POS. A POS Guided Sampling strategy is further proposed to address the low-diversity problem by enriching the diversity of POS. Extensive experiments and human evaluations demonstrate that, compared with existing state-of-the-art methods, our POS Guided Softmax and Sampling (POSG) can generate more diverse text while maintaining comparable quality.
Pairwise proximity metrics for topic modelling evaluation based on BERT embeddings.
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Articles

The use of topic modelling methods is a popular way to describe natural language text with a representative set of words. 
In order to evaluate such methods, objective metrics such as coherence and silhouette scores are commonly used. 
However, it has been shown that topic assessment based on such metrics does not align well with human judgment for classical document corpora such as articles, books and server logs and, at the same time, it is still unclear how appropriate they are for dialog data. 
In this paper, we investigate the most commonly used topic modelling evaluation scores in terms of their alignment with human judgment in the specific area of dialog speech. We show that there is still space for improvement in the objective evaluation of topic modelling, and propose a new group of metrics, called Pairwise Proximity metrics, that are shown to align better with human judgment, when compared to coherence and silhouette scores.
Chinese Word Attention based on Valid Division of Sentence
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: NER, Sentence, valid

Chinese word attention (CWA) with word-level information is very important for natural language processing. The purpose is how to attention words in a sentence. We first explore the valid divisions of a sentence by splitting word tools. We use BERT for character and word pre-training. Each character embedding with its word in one division is encoded in block local attention. We use attention with prior to assign attention weights to each splitting result, and finally combine the global attention mechanism to get the optimal recognition result in Chinese NER.
Tackling Situated Multi-Modal Task-Oriented Dialogs with a Single Transformer Model
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: coreference-resolution, Coreference Resolution, Decoder, dialog state tracking, Retrieval

The Situated Interactive Multi-Modal Conversations (SIMMC) 2.0 aims to create virtual shopping assistants that can accept complex multi-modal inputs, i.e. visual appearances of objects and user utterances. It consists of four subtasks, multi-modal disambiguation (MM-Disamb), multi-modal coreference resolution (MM-Coref), multi-modal dialog state tracking (MM-DST), and response retrieval and generation. While many task-oriented dialog systems usually tackle each subtask separately, we propose a jointly learned encoder-decoder that performs all four subtasks at once for efficiency. Moreover, we handle the multi-modality of the challenge by representing visual objects as special tokens whose joint embedding is learned via auxiliary tasks. This approach won the MM-Coref and response retrieval subtasks and nominated runner-up for the remaining subtasks using a single unified model. In particular, our model achieved 81.5\% MRR, 71.2\% R@1, 95.0\% R@5, 98.2\% R@10, and 1.9 mean rank in response retrieval task, setting a high bar for the state-of-the-art result in the SIMMC 2.0 track of the Dialog Systems Technology Challenge 10 (DSTC10).
Clause Attention based on Signal Words Division
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: NER, NMT, Position, Sentence

Clause Attention (CA) is very important for long sentences processing. We build and label datasets for signal word training. According to the position of signal word, the long sentences are divided into clauses which are assigned to additional block attention. The original sentence is mapped and fed into the shared encoder to learn the extraneous representation of words in its clause sentences. We use attention with prior to balance global attention with local attention. It improves the quality of long sentence processing in NER and NMT task.
Understanding Attention in Machine Reading Comprehension
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Machine Reading Comprehension, Reading Comprehension, Span-Extraction MRC

Achieving human-level performance on some of Machine Reading Comprehension (MRC) datasets is no longer challenging with the help of powerful Pre-trained Language Models (PLMs). However, the internal mechanism of these artifacts still remains unclear, placing an obstacle for further understanding these models. This paper focuses on conducting a series of analytical experiments to examine the relations between the multi-head self-attention and the final performance, trying to analyze the potential explainability in PLM-based MRC models.
We perform quantitative analyses on SQuAD (English) and CMRC 2018 (Chinese), two span-extraction MRC datasets, on top of BERT, ALBERT, and ELECTRA in various aspects. We discover that {\em passage-to-question} and {\em passage understanding} attentions are the most important ones, showing strong correlations to the final performance than other parts. Through visualizations and case studies, we also observe several general findings on the attention maps, which could be helpful to understand how these models solve the questions.
Entailment Graph Learning with Textual Entailment and Soft Transitivity
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Graph Learning, Natural Language Inference

Typed entailment graphs try to learn the entailment relations between predicates from text and model them as edges between predicate nodes. The construction of entailment graphs usually suffers from severe sparsity and unreliability of distributional similarity. We propose a two-stage method, Entailment Graph with Textual Entailment and Transitivity (EGT2). EGT2 learns the local entailment relations by recognizing the textual entailment between template sentences formed by typed CCG-parsed predicates. Based on the generated local graph, EGT2 then uses three novel soft transitivity constraints to consider the logical transitivity in entailment structures. Experiments on benchmark datasets show that EGT2 can well model the transitivity in entailment graph to alleviate the sparsity, and leads to signifcant improvement over current state-of-the-art methods.
Accurate, yet Inconsistent? Consistency Analysis on Language Models
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Language Modeling, Language Modelling

Consistency, which refers to generating the same predictions for semantically similar contexts, is highly desirable for a sound language model. Although recent pre-trained language models (PLMs) deliver an outstanding performance in various downstream tasks,  they should also exhibit a  consistent behaviour, given that the models truly understand language. In this paper, we propose a simple framework, called consistency analysis on language models (CALM), to evaluate a model's lower-bound consistency ability. Via experiments, we confirm that current PLMs are prone to generate inconsistent predictions even for semantically identical inputs with high confidence. We also observe that multi-task training is of benefit to improve consistency, increasing the value by 17% on average.
Unsupervised Preference-Aware Language Identification
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Language Identification

Recognizing the language of ambiguous texts has become a main challenge in language identification (LID). When using multilingual applications, users have their own language preferences, which can be regarded as external knowledge for LID. Nevertheless, current studies do not consider the inter-personal variations due to the lack of user annotated training data. To fill this gap,  we introduce preference-aware LID and propose a novel unsupervised learning strategy. Concretely, we construct pseudo training set for each user by extracting training samples from a standard LID corpus according to his/her historical language distribution. Besides,  we contribute the first user labeled LID test set called "U-LID". Experimental results reveal that our model can incarnate user traits and significantly outperforms existing LID systems on handling ambiguous texts. Our code and dataset are released at XXX.
Towards Collaborative Neural-Symbolic Graph Semantic Parsing via Uncertainty
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Semantic Parsing

Recent work in task-independent graph semantic parsing has shifted from grammar-based symbolic approaches to data-intensive neural approaches, and has shown strong performance on different types of meaning representations.  However, it is still unclear that what are the limitations of these neural parsers, and whether these limitations can be compensated by collaborating with symbolic parsers. In this paper, we attempt to answer these questions by taking English Resource Grammar (ERG) parsing as a case study. Specifically, we first develop a state-of-the-art neural ERG parser, and then conduct detailed analyses on fine-grained linguistic phenomena. The results suggest that the neural parser's performance degrades significantly on long-tail examples, while the symbolic parser performs more robustly. To address this, we further propose a collaborative neural-symbolic semantic parsing framework. Specifically, we improve the beam search strategy by designing a decision criterion that incorporates both the model uncertainty about the testing data distribution and the prior knowledge from a symbolic parser. Experimental results show that this collaborative parsing framework can outperform the single neural parser and concretely improve the model's performance on long-tail examples.
Freezing the Pivot for Triangular Machine Translation
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Language Modeling, Language Modelling, Machine Translation, Transfer Learning, Translation

Triangular machine translation is a special case of low-resource machine translation where the language pair of interest has limited parallel data, but both languages have abundant parallel data with a pivot language. Naturally, the key to triangular machine translation is the successful exploitation of such auxiliary data. In this work, we propose a transfer-learning-based approach that utilizes all types of auxiliary data. As we train auxiliary source-pivot and pivot-target translation models, we initialize some parameters of the pivot side with a pre-trained language model and freeze them to encourage both translation models to work in the same pivot language space, so that they can be smoothly transferred to the source-target translation model. Experiments show that our approach can outperform previous ones.
Are We Evaluating Paraphrase Generation Accurately?
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Paraphrase Generation, Survey

Paraphrase is a restatement of a text that conveys the same meaning using different expressions. The evaluation of paraphrase generation (PG) is a complex task and currently lacks a complete picture of the criteria and metrics. In this paper, we survey the automatic evaluation metrics and human evaluation criteria of PG evaluation. Base on the survey result,  we propose a reference-free automatic toolkit and list clear human evaluation criteria. Moreover, we notice the paraphrases selection in downstream tasks and propose a simple but effective evaluation Filter model. It can fusion multi automatic metrics to fit the human evaluation without any references.
Unified Speech-Text Pre-training for Speech Translation and Recognition
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Decoder, Multi-Task Learning, speech-recognition, Speech Recognition, Translation

In this work, we describe a method to jointly pre-train 
speech and text in an encoder-decoder modeling framework for speech translation and recognition. 
The proposed method utilizes multi-task learning to integrate four self-supervised and supervised subtasks for cross modality learning. 
A self-supervised speech subtask, which leverages unlabelled speech data, and a (self-)supervised text to text subtask, which makes use of 
abundant text training data,  take up the majority of the pre-training time. 
Two auxiliary supervised speech tasks are included to unify speech and text modeling space. 
Detailed analysis reveals learning interference among subtasks. In order to alleviate the subtask interference, two pre-training configurations are proposed for speech translation and speech recognition respectively.  
Our experiments show the proposed method can effectively fuse speech and text information into one model. It achieves between 1.7 and 2.3 BLEU improvement above the state of the art on the MuST-C speech translation dataset and comparable WERs to wav2vec 2.0 on the Librispeech speech recognition task.
Are Shortest Rationales the Best Explanations For Human Understanding?
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11

Existing self-explaining models typically favor extracting the shortest rationales possible (“shortest yet coherent subset of input to predict the same label”), with the assumption that short rationales are more intuitive to humans, even though short rationales lead to lower accuracy.   However, there is a lack of human studies on validating the effect of rationale length on human understanding. Is the shortest rationale indeed the most understandable for humans? To answer this question, we design a self-explaining model that can take control on rationale length.  Our model incorporates contextual information and supports flexibly extracting rationales at any target length. Through quantitative evaluation on model performance, we further verify that our method  LIMITEDINK outperforms existing self-explaining baselines on both end-task prediction and human-annotated rationale agreement.  We use it to generate rationales at 5 length levels, and conduct user studies to understand how much rationale would be sufficient for humans to confidently make predictions. We show that while most prior work extracts 10%-30% of the text to be the rationale, human accuracy tends to stabilize after seeing 40% of the full text.  Our result suggests the need for more careful design of the best human rationales.
Placing (Historical) Events on a Timeline: A Classification cum Co-ref Resolution Approach
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR September 2021 9
Tasks: Articles, coreference-resolution, Coreference Resolution, Event Coreference Resolution, Event Detection, Timeline Summarization

The event timeline provides one of the most effective ways to visualize the important historical events that occurred over a period of time, presenting the insights that may not be so apparent from reading the equivalent information in textual form. By leveraging generative adversarial learning for important event classification and by assimilating knowledge based tags for improving the performance of event coreference resolution we introduce a two staged system for event timeline generation from multiple (historical) text documents. In addition, we propose a vis-timeline based visualization technique to portray the event timeline. We demonstrate our results on two very well known historical documents --  the Collected Works of Mahatma Gandhi (CWMG) and the Collected Works of Abraham Lincoln (CWAL). Our results can be extremely helpful for historians, in advancing research in history and in understanding the socio-political landscape of a country as reflected in the writings of political leaders/scholars. Our work has some parallels with timeline summarization (TLS) tasks and therefore we use these as baselines. Rigorous experiments demonstrate that prior event detection which was hitherto absent in the TLS methods can improve summarization performance.  In order to show that our methods are very generic we reuse our method to visualize the evolution of coronavirus related events in India from a collection of various COVID-19 articles.
PromptBERT: Improving BERT Sentence Embeddings with Prompts
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Denoising, Semantic Similarity, Semantic Textual Similarity, Sentence, Sentence Embeddings, STS

  The poor performance of the original BERT for sentence semantic similarity has been widely discussed in previous works. We find that unsatisfactory performance is mainly due to the static token embeddings biases and the ineffective BERT layers, rather than the high cosine similarity of the sentence embeddings. To this end, we propose a prompt based sentence embeddings method which can reduce token embeddings biases and make the original BERT layers more effectively. By reformulating the sentence embeddings task as the fillin-the-blanks problem, our method significantly improves the performance of original BERT. We discuss two prompt representing methods and three prompt searching methods for prompt based sentence embeddings. Moreover, we propose a novel unsupervised training objective by the technology of template denoising, which substantially shortens the performance gap between the supervised and unsupervised setting. For experiments, we evaluate our method on both non fine-tuned and fine-tuned settings. Even a non fine-tuned method can outperform the fine-tuned methods like unsupervised ConSERT on STS tasks. Our fine-tuned method outperforms the state-of-the-art method SimCSE in both unsupervised and supervised settings. Compared to SimCSE, we achieve 2.29 and 2.58 points improvements on BERT and RoBERTa respectively under the unsupervised setting.
MoFE: Mixture of Factual Experts for Controlling Hallucinations in Abstractive Summarization
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Abstractive Text Summarization, Hallucination

Neural abstractive summarization models are susceptible to generating factually inconsistent content, a phenomenon known as hallucination. This limits the usability and adoption of these systems in real-world applications. To reduce the presence of hallucination, we propose the Mixture of Factual Experts (MoFE) model, which combines multiple summarization experts that each target a specific type of factual error. We construct MoFE by combining the experts using weights and logits ensembling strategies and find that the MoFE provides a modular approach to control different factual errors while maintaining performance on standard ROUGE metrics.
Non-Parametric Domain Adaptation for End-to-End Speech Translation
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Domain Adaptation, Translation, Triplet

The end-to-end speech translation (E2E-ST) has received increasing attention due to the potential of its less error propagation, lower latency, and fewer parameters. However, the effectiveness of neural-based approaches to this task is severely limited by the available training corpus, especially for domain adaptation where in-domain triplet training data is scarce or nonexistent. In this paper, we propose a novel non-parametric method that leverages domain-specific text translation corpus to achieve domain adaptation for the E2E-ST system. To this end, we first incorporate an additional encoder into the pre-trained E2E-ST model to realize text translation modeling and then unify the decoder's output representation for text and speech translation tasks by reducing the correspondent representation mismatch in available triplet training data. During domain adaptation, a $k$-nearest-neighbor ($k$NN) classifier is introduced to produce the final translation distribution using the external datastore built by the domain-specific text translation corpus, while the universal output representation is adopted to perform a similarity search. Experiments on the Europarl-ST benchmark demonstrate that when in-domain text translation data is used only, our proposed approach significantly improves the baseline by 12.82 BLEU on average in all translation directions.
Differentiable Learning of Rules with Constants in Knowledge Graph
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11

Knowledge reasoning, helping overcome the incompleteness issue of knowledge graph(KG), significantly contributes to the development of large KG, which consists of relations and constants. Rule mining studies the problem of capturing interpretable patterns over KG, which is one of the key tasks of knowledge reasoning. However, previous works mainly focus on the combination of different relations, and are limited for ignoring the importance of constants. In this paper, we propose that constants should be considered in rule mining process, and introduce an Elegant  Differentiable rUle learning with Constant mEthod (EduCe). Based on soft constant operator and dynamic weight, the model we proposed can mine more diverse and accurate logical rules while controlling the number of parameters, which is also a great challenge to this problem. Experiment results on several benchmark datasets demonstrate the effectiveness and accuracy of rule with constants.