pwc-archive/papers-with-abstracts huggingface dataset:
Few-Shot Knowledge Graph Completion with Data Fusion and Augmentation
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Few-Shot Learning, Knowledge Graph Completion, Knowledge Graphs, World Knowledge

This paper addresses the few-shot knowledge graph completion problem, which aims to infer facts for long-tail distributed relations for completing knowledge graphs. The few-shot knowledge graph completion task confronts with the difficulties of insufficient structural evidence caused by structure sparsity and heterogeneity, and few training samples caused by the long-tail distribution property of few-shot relations. To overcome the above difficulties, we propose to adaptively fuse entity structure information with rich textual content features, and leverage a generative approach to augment entity samples to enrich the training samples for few-shot relations. We seamlessly integrate the adaptive feature fusion and generative sample augmentation components with the few-shot learning task into an end-to-end framework, with the feature fusion and sample augmentation able to be adjusted for the few-shot learning objective through error backpropagation. We conduct few-shot knowledge graph completion experiments on two real-world knowledge graphs, showing the significant advantage of the proposed algorithm over state-of-the-art baselines, and the effectiveness of the proposed feature fusion and sample augmentation components.
HSC-Rocket: An interactive dialogue assistant to make agents composing service better through human feedback
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Service Composition

Facing the current dynamic service environment, fast and efficient service composition has attracted great attention in recent years. Users prefer to express their personal requirements based on natural language, and their real-time feedback could better reflect the effect of service composition to a great extent. Consequently, this paper designs an interactive dialogue assistant, HSC-Rocket, to better provide service composition by considering human feedback. Firstly, we propose a human-computer interaction dynamic service composition algorithm based on reinforcement learning. The design of the reward mechanism considers the quality of service (QoS) and real-time feedback, which can more accurately meet the demands of users. Then, the functional requirements are analyzed through word embedding, to realize the dynamic composition of abstract and concrete services. Furthermore, we utilize the sample enhancement method to alleviate the issue of fewer sample data in the initial stage of user interaction, which improves the robustness of our system. Accordingly, we have implemented the HSC-Rocket prototype, which allows users to obtain multi-domain dialogue requirements. Extensive experiments on the RapidAPI dataset have demonstrated the superiority and effectiveness of the HSC-Rocket.
MDG: Metaphorical Sentence Detection and Generation with Masked Metaphor Modeling
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Sentence, Sentence Classification

This study tackles literal to metaphorical sentence generation, presenting a framework that can potentially lead to the production of an infinite number of new metaphors. To achieve this goal, we propose a complete workflow that tackles metaphorical sentence classification and metaphor reconstruction. Unlike similar research works regarding metaphor generation, our approach does not require any custom
or closed-source model, hence with this work we introduce a complete literal to metaphorical open-source model. The obtained results show that a good ratio of originally literal sentences, coming from different data sources and topics, are turned to metaphorical. Human evaluation shows that our constructed metaphors are considered more fluent, creative and metaphorical than figurative statements created by a real person. Furthermore, by using our artificial data to increase the training size of a metaphorical sentence classification dataset, we register an improvement of 3% over the baseline.
Roles of Words: What Should (n’t) Be Augmented in Text Augmentation on Text Classification Tasks?
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Classification, Text Augmentation, text-classification, Text Classification

Text augmentation techniques are widely used in text classification problems to improve the performance of classifiers, especially in low-resource scenarios. Previous text-editing-based methods augment the text in a non-selective manner: the words in the text are treated without difference during augmentation, which may result in unsatisfactory augmented samples. In this work, we present four kinds of roles of words (ROWs) which have different functions in text classification tasks, and design effective methods to automatically extract these ROWs based on statistical and semantic perspectives. Systematic experiments are conducted on what ROWs should (n't) be augmented during augmentation for classification tasks. Based on these experiments, we discover some interesting and instructive potential patterns that certain ROWs are especially suitable or unsuitable for certain augmentation operations. Guided by these patterns, we propose a set of Selective Text Augmentation (STA) operations, which significantly outperform traditional methods and show outstanding generalization performance.
NEWSFARM: the Largest Chinese Corpus for Long News Summarization
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: News Summarization, Semantic Similarity, Semantic Textual Similarity, Text Summarization

Recently, driven by a large number of datasets, the field of natural language processing(NLP) has developed rapidly. However, the lack of large-scale and high-quality Chinese datasets is still a critical bottleneck for further research on automatic text summarization. To close this gap, we searched Chinese news websites of domestic and abroad media, designed the algorithm HSS(hidden text topic, semantic similarity, and syntactic similarity) to crawl and filter these records to construct NEWSFARM. NEWSFARM is the largest highest quality Chinese long news summarization corpus, containing more than 200K Chinese long news and summaries written by professional editors or authors, which are all released to the public. Based on the corpus, we calculated the static metrics and designed many experiments with the baseline models. By comparing with the common datasets, the experiment results show that the high quality of our dataset and training effect of the models, which not only demonstrates the usefulness and challenges of the proposed corpus for automatic text summarization but also provides a benchmark for further research.
How Do Seq2Seq Models Perform on End-to-End Data-to-Text Generation?
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Data-to-Text Generation, Text Generation

With the rapid development of deep learning, Seq2Seq paradigm has become prevalent for end-to-end data-to-text generation, and the BLEU scores have been increasing in recent years. However, it is widely recognized that there is still a gap between the quality of the texts generated by models and the texts written by human. In order to better understand the ability of Seq2Seq models, evaluate their performance and analyze the results, we choose to use Multidimensional Quality Metric(MQM) to evaluate several representative Seq2Seq models on end-to-end data-to-text generation. We annotate the outputs of five models on four datasets with eight error types and find that  1) copy mechanism is helpful for the improvement in Omission and Inaccuracy Extrinsic errors but it increases other types of errors such as Addition; 2) pre-training techniques are highly effective, and pre-training strategy and model size are very significant; 3) the structure of the dataset also influences the model's performance greatly; 4) some specific types of errors are generally challenging for seq2seq models.
Morphosyntactic Tagging with Pre-trained Language Models for Arabic and its Dialects
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11

We present state-of-the-art results on morphosyntactic tagging across different varieties of Arabic using fine-tuned pre-trained transformer language models. Our models consistently outperform existing systems in Modern Standard Arabic and all the Arabic dialects we study, achieving 2.6% absolute improvement over the previous state-of-the-art in Modern Standard Arabic, 2.8% in Gulf, 1.6% in Egyptian, and 8.3% in Levantine. We explore different training setups for fine-tuning pre-trained transformer language models, including training data size, the use of external linguistic resources, and the use of annotated data from other dialects in a low-resource scenario. Our results show that strategic fine-tuning using datasets from other high-resource dialects is beneficial for a low-resource dialect. Additionally, we show that high-quality morphological analyzers as external linguistic resources are beneficial especially in low-resource settings.
ERNIE-SPARSE: Learning Hierarchical Efficient Transformer Through Regularized Self-Attention
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Sparse Learning, text-classification, Text Classification

Sparse Transformer has recently attracted a lot of attention since the ability for reducing the quadratic dependency on the sequence length. We argue that two factors, information bottleneck sensitivity and inconsistency between different attention topologies, could affect the performance of the Sparse Transformer. This paper proposes a well-designed model named ERNIE-Sparse. It consists of two distinctive parts: (i) Hierarchical Sparse Transformer (HST) to sequentially unify local and global information. (ii) Self-Attention Regularization (SAR) method, a novel regularization designed to minimize the distance for transformers with different attention topologies. To evaluate the effectiveness of ERNIE-Sparse, we perform extensive evaluations. Firstly, we perform experiments on a multi-modal long sequence modeling task benchmark, Long Range Arena (LRA). Experimental results demonstrate that ERNIE-Sparse significantly outperforms a variety of strong baseline methods including the dense attention and other efficient sparse attention methods and achieves improvements by 2.77% (57.78% vs. 55.01%). Secondly, to further show the effectiveness of our method, we pretrain ERNIE-Sparse and verified it on 3 text classification and 2 QA downstream tasks, achieve improvements on classification benchmark by 0.83% (92.46% vs. 91.63%), on QA benchmark by 3.24% (74.67% vs. 71.43%). Experimental results continue to demonstrate its superior performance.
AMRize, then Parse! Enhancing AMR Parsing with PseudoAMR Data
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Abstract Meaning Representation, AMR Parsing, Dependency Parsing, Semantic Role Labeling

As Abstract Meaning Representation (AMR) implicitly involves compound semantic annotations, we hypothesize auxiliary tasks which are semantically or formally related can better enhance AMR parsing. With carefully designed control experiments, we find that 1) Semantic role labeling (SRL) and dependency parsing (DP), would bring much more significant performance gain than unrelated tasks in the text-to-AMR transition. 2) To make a better fit for AMR, data from auxiliary tasks should be properly "AMRized'' to PseudoAMR before training. 3) Intermediate-task training paradigm outperforms multitask learning when introducing auxiliary tasks to AMR parsing. From an empirical perspective, we propose a principled method to choose, reform, and train auxiliary tasks to boost AMR parsing. Extensive experiments show that our method achieves new state-of-the-art performance on in-distribution, out-of-distribution, and few-shots benchmarks of AMR parsing.
ASSIST: Towards Label Noise-Robust Dialogue State Tracking
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR September 2021 9
Tasks: Dialogue State Tracking

The MultiWOZ 2.0 dataset has greatly boosted the research on dialogue state tracking (DST). However, substantial noise has been discovered in its state annotations. Such noise brings about huge challenges for training DST models robustly. Although several refined versions, including MultiWOZ 2.1-2.4, have been published recently, there are still lots of noisy labels, especially in the training set. Besides, it is costly to rectify all the problematic annotations. In this paper, instead of improving the annotation quality further, we propose a general framework, named ASSIST (lAbel noiSe-robuSt dIalogue State Tracking), to train DST models robustly from noisy labels. ASSIST first generates pseudo labels for each sample in the training set by using an auxiliary model trained on a small clean dataset, then puts the generated pseudo labels and vanilla noisy labels together to train the primary model. We show the validity of ASSIST theoretically. Experimental results also demonstrate that ASSIST improves the joint goal accuracy of DST by up to $28.16\%$ on the initial version MultiWOZ 2.0 and $8.41\%$ on the latest version MultiWOZ 2.4, respectively.
Towards Robust Online Dialogue Response Generation
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Re-Ranking, Response Generation

Although pre-trained sequence-to-sequence models have achieved great success in dialogue response generation, chatbots still suffer from generating inconsistent responses in real-world practice, especially in multi-turn settings. We argue that this can be caused by a discrepancy between training and real-world testing. At training time, chatbots generate response with the golden context, while it has to generate based on the context consisting of both user utterances and the model predicted utterances during real-world testing. With the growth of the number of utterances, this discrepancy becomes more serious in the multi-turn settings. In this paper, we propose a hierarchical sampling-based method consisting of both utterance-level sampling and semi-utterance-level sampling, to alleviate the discrepancy, which implicitly increases the dialogue coherence. We further adopt reinforcement learning and re-ranking methods to explicitly optimize the dialogue coherence during training and inference, respectively. Empirical experiments show the effectiveness of the proposed methods for improving the robustness of chatbots in real practice.
Stock Movement Prediction Based on Bi-typed and Hybrid-relational Market Knowledge Graph via Dual Attention Networks
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Prediction, Stock Prediction

Stock Movement Prediction (SMP) aims at predicting listed company's stock future price trend, which is a challenging task due to the volatile nature of financial markets. Recent financial studies show that the momentum spillover effect plays a significant role in stock fluctuation. However, previous studies typically only learn the simple connection information among related companies, which inevitably fail to model complex relations of listed companies in real financial market. To address this issue, we first construct a more comprehensive Market Knowledge Graph (MKG) which contains bi-typed entities including listed companies and their associated executives, and hybrid-relations including the explicit relations and implicit relation. Afterward, we propose \textsc{DanSmp}, a novel Dual Attention Networks to learn the momentum spillover signals based upon the constructed MKG for stock prediction. The empirical experiments on our constructed datasets against nine SOTA baselines demonstrate that the proposed \textsc{DanSmp} is capable of improving stock prediction with the constructed MKG.
Sentence-aware Contrastive Learning for Open-Domain Passage Retrieval
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Contrastive Learning, Passage Retrieval, Retrieval, Sentence

Training dense passage representations via contrastive learning has been shown effective for Open-Domain Passage Retrieval (ODPR). Existing studies focus on further optimizing by improving negative sampling strategy or extra pretraining. However, these studies keep unknown in capturing passage with internal representation conflicts from improper modeling granularity. This work thus presents a refined model on the basis of a smaller granularity, contextual sentences, to alleviate the concerned conflicts. In detail, we introduce an in-passage negative sampling strategy to encourage a diverse generation of sentence representations within the same passage. Experiments on three benchmark datasets verify the efficacy of our method, especially on datasets where conflicts are severe. Extensive experiments further present good transferability of our method across datasets.
On the Difficulties of using NLP for Language Revitalization
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11

This paper is dedicated to discussing the general difficulty of using emerging Natural Language Processing (NLP) technologies to the revitalization of languages. Previous literature had described the social causes of language shift; legal prohibitions, social and economic marginalization as well as a lack of inclusion in public life have been identified as the main factors for the non-viability of minority languages. As such, as innovative as they may be, these emerging tools are not enough to rescue languages and the core issues must be addressed if meaningful results are expected.
Event-Event Relation Extraction using Probabilistic Box Embedding
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Event Relation Extraction, Multi-class Classification, Relation, Relation Extraction

To understand a story with multiple events, it is important to capture the proper relations across these events. However, existing event relation extraction (ERE) framework regards it as a multi-class classification task and do not guarantee any coherence between different relation types, such as anti-symmetry. If a phone line "died" after "storm", then it is obvious that the "storm" happened before the "died". Current framework of event relation extraction do not guarantee this coherence and thus enforces it via constraint loss function (Wang et al., 2020). In this work, we propose to modify the underlying ERE model to guarantee coherence by representing each event as a box representation (BERE) without applying explicit constraints. From our experiments, BERE also shows stronger conjunctive constraint satisfaction while performing on par or better in F1 compared to previous models with constraint injection.
Data Augmentation and Learned Layer Aggregation for Improved Multilingual Language Understanding in Dialogue
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Data Augmentation, Natural Language Understanding

Scaling dialogue systems to a multitude of domains, tasks and languages relies on costly and time-consuming data annotation for different domain-task-language configurations. The annotation efforts might be substantially reduced by the methods that generalise well in zero- and few-shot scenarios, and also effectively leverage external unannotated data sources (e.g., Web-scale corpora). We propose two methods to this aim, offering improved dialogue natural language understanding (NLU) across multiple languages: 1) Multi-SentAugment, and 2) LayerAgg. Multi-SentAugment is a self-training method which augments available (typically few-shot) training data with similar (automatically labelled) in-domain sentences from large monolingual Web-scale corpora. LayerAgg learns to select and combine useful semantic information scattered across different layers of a Transformer model (e.g., mBERT); it is especially suited for zero-shot scenarios as semantically richer representations should strengthen the model's cross-lingual capabilities. Applying the two methods with state-of-the-art NLU models obtains consistent improvements across two standard multilingual NLU datasets covering 16 diverse languages. The gains are observed in zero-shot, few-shot, and even in full-data scenarios. The results also suggest that the two methods achieve a synergistic effect: the best overall performance in few-shot setups is attained when the methods are used together.
CalBERT - Code-mixed Adaptive Language representations using BERT
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Sentiment Analysis

A code-mixed language is a type of language that involves the combination of two or more language varieties in its script or speech. Code-mixed language has become increasingly prevalent in recent times, especially on social media. However, the exponential increase in the usage of code-mixed language, especially in a country like India which is linguistically diverse has led to various inconsistencies. Analysis of text is now becoming harder to tackle because the language present is not consistent and does not work with predefined existing models which are monolingual. We propose a novel approach to improve performance in Transformers by introducing an additional step called "Siamese Pre-Training", which allows pre-trained monolingual Transformers to adapt language representations for code-mixed languages with a few examples of code-mixed data. Our studies show that CalBERT is able to improve performance over existing pre-trained Transformer architectures on downstream tasks such as sentiment analysis. Multiple CalBERT architectures beat the state of the art F1-score on the Sentiment Analysis for Indian Languages (SAIL) dataset, with the highest possible improvement being 8.9%
Towards Coherent Visual Storytelling with Ordered Image Attention
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11
Tasks: Position, Sentence, Visual Storytelling

We address the problem of visual storytelling, i.e., generating a story for a given sequence of images. While each story sentence should describe a corresponding image, a coherent story also needs to be consistent and relate to both future and past images. Current approaches encode images independently, disregarding relations between images.  Our approach learns to encode images with different interactions based on the story position (i.e., past image or future image). To this end,  we develop a novel message-passing-like algorithm for ordered image attention (OIA) that collects interactions across all the images in the sequence. Finally, to generate the story's sentences, a second attention mechanism picks the important image attention vectors with an Image-Sentence Attention (ISA). The obtained results improve the METEOR score on the VIST dataset by 1%. Furthermore, a thorough human study confirms improvements and demonstrates that order-based Interactions significantly improve coherency (64.20% vs. 28.70%).
Automatic Identification of Cuneiform Fragments Using String Alignment Algorithms
Authors: Anonymous
Date: 2021-11-16 00:00:00
Proceeding: ACL ARR November 2021 11

The literature from ancient Mesopotamia is still riddled with textual lacunas. Scores of fragments which could potentially fill those lacunas lie unidentified in museums's cabinets, but their identification has traditionally been slow and laborious due to the ambiguities of cuneiform script. The present article presents a novel method for dealing with these ambiguities by using a string alignment algorithm adapted for cuneiform, which makes identification much easier and speeds up the process dramatically. The availability of this algorithm and of corpora on which to use it will advance significantly the reconstruction of Mesopotamian literature.