pwc-archive/papers-with-abstracts huggingface dataset:
Exploring the Effectiveness of Student Behavior in Prerequisite Relation Discovery for Concepts
Authors: Anonymous
Date: 2021-09-17 00:00:00
Proceeding: ACL ARR September 2021 9
Tasks: Relation

What knowledge should a student grasp before beginning a new MOOC course?  This question can be answered by discovering prerequisite relations of knowledge concepts.  In recent years, researchers have devoted intensive efforts to detecting such relations by analyzing various types of information. However, there are still a few explorations of utilizing student behaviors in this task. In this paper, we investigate the effectiveness of student behaviors in prerequisite relation discovery for course concepts. Specifically, we first construct a novel MOOC dataset to support the study. We then verify the effectiveness of student behaviors via serving as additional features for existing prerequisite relation discovery models. Moreover, we explore to better utilize student behaviors via graph-based modeling. We hope our study could call for more attention and efforts to explore the student behavior for prerequisite relation discovery.
Headed-Span-Based Projective Dependency Parsing
Authors: Anonymous
Date: 2021-09-17 00:00:00
Proceeding: ACL ARR September 2021 9
Tasks: Dependency Parsing, global-optimization

 We propose a new paradigm for projective dependency parsing based on headed spans. 
In a projective dependency tree, the subtree rooted at each word covers a contiguous sequence (i.e., a span) in the surface order. We call a span marked with a root word headed span. A projective dependency tree can be represented as a collection of headed spans. We decompose the score of a dependency tree into the scores of the headed spans and design a novel $O(n^3)$ dynamic programming algorithm to enable global training and exact inference. The advantages of our headed-span-based dependency parsing include that it captures subtree information more adequately than first-order graph-based methods and performs global optimization in decoding (in contrast to transition-based methods).  We evaluate our method on PTB, CTB, and UD and it achieves competitive results in comparison with previous methods.
BART-light: One Decoder Layer Is Enough
Authors: Anonymous
Date: 2021-09-17 00:00:00
Proceeding: ACL ARR September 2021 9
Tasks: Decoder, Language Modeling, Language Modelling, Text Generation

BART (Lewis et al., 2020), an encoder-decoder transformer language model (LM), has reached state-of-the-art results on several tasks in natural language generation and understanding. Similar to other pretrained encoder-decoder LMs, it uses the same number of hidden layers in the encoder and the decoder. In this paper, we show that one can easily remove all but one or two decoder layers for text generation tasks and even remove the whole decoder for classification tasks, with little to no compromises on performance. Our study presents that a shallow decoder is sufficient for most tasks when a deep encoder is used.
Achieving Reliable Human Assessment of Open-Domain Dialogue Systems
Authors: Anonymous
Date: 2021-09-17 00:00:00
Proceeding: ACL ARR September 2021 9
Tasks: Dialogue Evaluation

Evaluation of open-domain dialogue systems is highly challenging and development of better techniques is highlighted time and again as desperately needed. Despite substantial efforts to carry out reliable live evaluation of systems in recent competitions, annotations have been abandoned and reported as too unreliable to yield sensible results. This is a serious problem since automatic metrics are not known to provide a good indication of what may or may not be a high-quality conversation. Answering the distress call of competitions that have emphasized the urgent need for better evaluation techniques in dialogue, we present the successful development of human evaluation that is highly reliable while still remaining feasible and low cost. Self-replication experiments reveal almost perfectly repeatable results with a correlation of $r=0.969$. Furthermore, due to the lack of appropriate methods of statistical significance testing, the likelihood of potential improvements to systems occurring due to chance is rarely taken into account in dialogue evaluation, and the evaluation we propose facilitates application of standard tests. Since we have developed a highly reliable evaluation method, new insights into system performance can be revealed. We therefore include a comparison of state-of-the-art models (i) with and without personas, to measure the contribution of personas to conversation quality, as well as (ii) prescribed versus freely chosen topics. Interestingly with respect to personas, results indicate that personas do not positively contribute to conversation quality as expected.
CDM: Combining Extraction and Generation for Definition Modeling
Authors: Anonymous
Date: 2021-09-17 00:00:00
Proceeding: ACL ARR September 2021 9
Tasks: Text Generation

Definitions are essential for term understanding. Recently, there is an increasing interest in extracting and generating definitions of terms automatically. However, existing approaches for this task are either extractive or abstractive - definitions are either extracted from a corpus or generated by a language generation model. In this paper, we propose to combine extraction and generation for definition modeling: first extract self- and correlative definitional information of target terms from the Web and then generate the final definitions by incorporating the extracted definitional information. Experiments demonstrate our framework can generate high-quality definitions for technical terms and outperform state-of-the-art models for definition modeling significantly.
Unsupervised Personality-Aware Language Identification
Authors: Anonymous
Date: 2021-09-17 00:00:00
Proceeding: ACL ARR September 2021 9
Tasks: Language Identification

Recognizing the language of ambiguous texts remains a main challenge in language identification (LID). When using multilingual applications, users have their own language preferences, which can be regarded as external knowledge for LID. Nevertheless, current studies marginally consider the inter-personal variations due to the lack of user annotated training data. To fill this gap,  we introduce personality-aware LID and propose a novel unsupervised learning strategy. Concretely, we extract training samples for each user from a standard LID corpus according to his/her language preference. Furthermore,  we contribute the first user labeled LID test set called "U-LID''. Experimental results reveal that the proposed model can incarnate user traits and significantly outperforms existing LID systems on handling ambiguous texts. Our code and dataset will be released upon the acceptance.
Multi-Narrative Semantic Intersection Task: Evaluation and Benchmark
Authors: Anonymous
Date: 2021-09-17 00:00:00
Proceeding: ACL ARR September 2021 9
Tasks: Sentence, Text Summarization

In this paper, we introduce an important yet relatively unexplored NLP task called Multi-Narrative Semantic Intersection (MNSI), which entails generating a Semantic Intersection of multiple alternate narratives. As no benchmark dataset is readily available for this task, we created one by crawling 2,925 alternative narrative pairs from the web and then, went through the tedious process of manually creating 411 different ground-truth semantic intersections by engaging human annotators. As a way to evaluate this novel task, we first conducted a systematic study by borrowing the popular ROUGE metric from text summarization literature and discovered that ROUGE is not suitable for our task. Subsequently, we conducted further human annotations/validations to create 200 document-level and 1,518 sentence-level ground-truth labels which helped us formulate a new precision-recall style evaluation metric, called SEM F1 (semantic F1), based on presence, partial presence and absence of information. Experimental results show that the proposed SEM F1 metric yields higher correlation with human judgement as well as higher inter-rater agreement compared to ROUGE metric and thus, we recommend the community to use this metric for evaluating future research on this topic.
Constructing Multilingual CCG Treebanks from Universal Dependencies
Authors: Anonymous
Date: 2021-09-17 00:00:00
Proceeding: ACL ARR September 2021 9
Tasks: Binarization

This paper introduces an algorithm to convert Universal Dependencies (UD) treebanks to Combinatory Categorial Grammar (CCG) treebanks. As CCG encodes almost all grammatical information into the lexicon, obtaining a high quality CCG derivation from a dependency tree is a challenging task. Our algorithm contains four main steps: binarization of dependency trees, functor/argument identification, category assignment through hand-crafted rules, and category inference for unassigned constituents. To evaluate our converted treebanks, we perform lexical, sentential, and syntactic rule coverage analysis, as well as CCG parsing experiments. We achieve over 80% conversion rate on 68 treebanks of 44 languages, and over 90% lexical coverage on 81 treebanks of 52 languages.
Supervised Relation Classification as Two-way Span-Prediction
Authors: Anonymous
Date: 2021-09-17 00:00:00
Proceeding: ACL ARR September 2021 9
Tasks: Classification, Prediction, Question Answering, Relation, Relation Classification, Vocal Bursts Valence Prediction

Most of the current supervised relation classification (RC) algorithms use a single embedding to represent the relation between a pair of entities. We argue that a better approach is to treat the RC task as a Span-Prediction (SP) problem, similar to Question Answering (QA). We present an SP-based system for RC and evaluate its performance compared to the embedding-based system. We demonstrate that by adding a few improvements, the supervised SP objective works significantly better than the standard classification-based objective. We achieve state-of-the-art results on the TACRED, SemEval task 8, and the CRE datasets.
Consistent Crosslingual Data Transfer for Open Information Extraction
Authors: Anonymous
Date: 2021-09-17 00:00:00
Proceeding: ACL ARR September 2021 9
Tasks: Open Information Extraction, Relation, Sentence

Progress with supervised Open Information Extraction (OpenIE) has been primarily limited to English due to the scarcity of training data in other languages. In this paper, we explore techniques to automatically convert English text for training OpenIE systems in other languages. We introduce the Alignment Augmented Constrained Translation (AACTrans) model to translate English sentences and their corresponding extractions consistently with each other --- with no changes to vocabulary or semantic meaning which may result from independent translations. Using the data generated with AACTrans, we train a novel two-stage generative OpenIE model, which we call Gen2OIE, that outputs for each sentence: 1) relations in the first stage and 2) all extractions containing the relation in the second stage. Gen2OIE increases relation coverage using a training data transformation technique that is generalizable to multiple languages, in contrast to existing models that use an English-specific training loss. Evaluations on 5 languages --- Spanish, Portuguese, Chinese, Hindi and Telugu --- show that the Gen2OIE with AACTrans data outperforms prior systems by a margin of 6-40% in F1.
What Makes Reading Comprehension Questions Difficult? Investigating Variation in Passage Sources and Question Types
Authors: Anonymous
Date: 2021-09-17 00:00:00
Proceeding: ACL ARR September 2021 9
Tasks: Logical Reasoning, Multiple-choice, Natural Language Understanding, Reading Comprehension

In order for a natural language understanding benchmark to be useful in research, it has to consist of examples that are diverse and difficult enough to discriminate among current and near-future state-of-the-art systems. However, we do not yet know what kinds of passages and their sources help us collect a variety of challenging examples. In this study, we crowdsource multiple-choice reading comprehension questions for passages taken from seven qualitatively distinct sources, analyzing what attributes of passages contribute to the difficulty and question types of the collected examples. We find that passage source, length, and readability measures do not significantly affect question difficulty. Among seven question types we manually annotate, questions that require numerical reasoning and logical reasoning are relatively difficult but their frequencies depend on the passage sources. These results suggest that when creating a new benchmark dataset, we do not have to use difficult passages but select passage sources carefully so that it has questions that involve linguistic phenomena we are interested in.
Towards Comprehensive Patent Approval Predictions:Beyond Traditional Document Classification
Authors: Anonymous
Date: 2021-09-17 00:00:00
Proceeding: ACL ARR September 2021 9
Tasks: Classification, Document Classification

Predicting the approval chance of a patent application is a challenging problem involving multiple facets. The most crucial facet is arguably the novelty --- \emph{35 U.S. Code § 102} rejects more recent applications that have very similar prior arts. Such novelty evaluations differ the patent approval prediction from conventional document classification --- Successful patent applications may share similar writing patterns; however, too-similar newer applications would receive the opposite label, thus confusing standard document classifiers (e.g., BERT). To address this issue, we propose a novel framework \our that unifies the document classifier with handcrafted features, particularly time-dependent novelty scores. Specifically, we formulate the novelty scores by comparing each application with millions of prior arts using a hybrid of efficient filters and a neural bi-encoder. Moreover, we impose a new regularization term into the classification objective to enforce the monotonic change of approval prediction w.r.t. novelty scores. From extensive experiments on the large-scale USPTO dataset, we find that our time-dependent novelty features offer a boost on top of the document classifier. Also, our monotonic regularization, while shrinking the search space, can drive the optimizer to better local optima, yielding empirical performance gains. Ex-post analysis of prediction scores further confirms that the document classifier and handcrafted features capture distinct sets of learning information.
Knowledge Neurons in Pretrained Transformers
Authors: Anonymous
Date: 2021-09-17 00:00:00
Proceeding: ACL ARR September 2021 9

Large-scale pretrained language models are surprisingly good at recalling factual knowledge presented in the training corpus. In this paper, we present preliminary studies on how factual knowledge is stored in pretrained Transformers by introducing the concept of knowledge neurons. Specifically, we examine the fill-in-the-blank cloze task for BERT. Given a relational fact, we propose a knowledge attribution method to identify the neurons that express the fact. We find that the activation of such knowledge neurons is positively correlated to the expression of their corresponding facts. In our case studies, we attempt to leverage knowledge neurons to edit (such as update, and erase) specific factual knowledge without fine-tuning. Our results shed light on understanding the storage of knowledge within pretrained Transformers.
Probing, Generalization and Application of Metaphorical Knowledge in Pre-trained Language Models
Authors: Anonymous
Date: 2021-09-17 00:00:00
Proceeding: ACL ARR September 2021 9

Human languages are full of metaphorical expressions. Metaphors help people understand the world by connecting new concepts and domains to more familiar ones. Large pre-trained language models (PLMs) are therefore assumed to encode metaphorical knowledge useful for NLP systems when processing language. In this paper, we investigate this hypothesis for PLMs by probing the metaphoricity knowledge in their encodings, by measuring the cross-lingual and cross-dataset generalization of this knowledge, and by analyzing the application of this knowledge when generating metaphorical expressions. We present studies in multiple metaphoricity detection datasets and four languages (i.e., English, Spanish, Russian, and Farsi). Our extensive experiments suggest that contextual representations in PLMs do encode metaphoricity information, and mostly in their middle layers, and the knowledge is transferrable between languages and datasets in most cases. Finally, we show that PLMs face more challenges in generating metaphors, especially as their novelty increases. Our findings give helpful insights for both cognitive and NLP scientists.
Modeling Multi-granularity Segmentation for Rare Words in Neural Machine Translation
Authors: Anonymous
Date: 2021-09-17 00:00:00
Proceeding: ACL ARR September 2021 9
Tasks: Diversity, Machine Translation, NMT, Segmentation, Translation

Segmenting rare words into subwords has become a commonly used and effective way to alleviate the open vocabulary problem in Neural Machine Translation (NMT). The existing dominant segmentation methods either give rare words a single segmentation or a fixed segmentation, which leads to a lack of morphological diversity in representing words. For rare words, we first obtain segmentation with different granularities through Byte Pair Encoding (BPE) and BPE-Dropout, and then propose \textsc{BPEatt} model to dynamically mix the BPE subwords and BPE-Dropout subwords, which enhances the encoder's ability to represent rich morphological information. Experiments on six translation benchmarks of different scales show that our proposed method significantly outperforms the baseline model and has obvious advantages over related methods.
Draft, Command, and Edit: Controllable Text Editing in E-Commerce
Authors: Anonymous
Date: 2021-09-17 00:00:00
Proceeding: ACL ARR September 2021 9
Tasks: Attribute, Data Augmentation

Product description generation is a challenging and under-explored task. Most such work takes a set of product attributes as inputs then generates a description from scratch in a single pass. However, this widespread paradigm might be limited when facing the dynamic wishes of users on constraining the description, such as deleting or adding the content of a user-specified attribute based on the previous version. To address this challenge, we explore a new draft-command-edit manner in description generation, leading to the proposed new task ---controllable text editing in E-commerce. More specifically, we allow systems to receive a command (deleting or adding) from the user and then generate a description by flexibly modifying the content based on the previous version. It is easier and more practical to meet the new needs by modifying previous versions than generating from scratch. To accompany this new task, we present a human-written draft-command-edit dataset called E-cEdits. Furthermore, we design a data augmentation method to remedy the low resource challenge in this task, which contains a model-based and a rule-based strategy to imitate the edit by humans. Our experimental results show that using the new data augmentation method outperforms baselines to a greater extent in both automatic and human evaluations.
Contrastive Word Embedding Learning for Neural Machine Translation
Authors: Anonymous
Date: 2021-09-17 00:00:00
Proceeding: ACL ARR September 2021 9
Tasks: Machine Translation, NMT, Representation Learning, Semantic Similarity, Semantic Textual Similarity, Translation, Word Embeddings

Seq2seq models have shined in the field of Neural Machine Translation (NMT). However, word embeddings learned by NMT models tend to degenerate and be distributed into a narrow cone, named {\em{representation degeneration problem}}, which limits the representation capacity of word embeddings. In this paper, we propose a Contrastive Word Embedding Learning (CWEL) method to address this problem. CWEL combines the ideas of contrastive representation learning with embedding regularization, and adaptively minimizes the cosine similarity of word embeddings on the target side according to their semantic similarity. Experiments on multiple translation benchmark datasets show that CWEL significantly improves translation qualities. Additional analysis shows that the improvements mainly come from the well-learned word embeddings.
Attention Mechanism with Energy-Friendly Operations
Authors: Anonymous
Date: 2021-09-17 00:00:00
Proceeding: ACL ARR September 2021 9
Tasks: Machine Translation, Translation

Attention mechanism has become the dominant module in natural language processing models. It is computationally intensive and depends on massive power-hungry multiplications. In this paper, we rethink variants of attention mechanism from the energy consumption aspects. After reaching the conclusion that the energy costs of several energy-friendly operations are far less than their multiplication counterparts, we build a novel attention model by completely replacing multiplications with either selective operations or additions. Empirical results on three machine translation tasks demonstrate that the proposed method, against the vanilla one, achieves comparable accuracy while only consumes a half of energy. Our code will be released upon the acceptance.
Extractive Topical Summarization With Aspects
Authors: Anonymous
Date: 2021-09-17 00:00:00
Proceeding: ACL ARR September 2021 9
Tasks: Extractive Summarization

Extractive summarization is a task of highlighting the most important parts of the text. We introduce a new approach to extractive summarization task using hidden topical structure and information about aspects of the text. Experimental results on CNN/DailyMail demonstrate that our approach generates more accurate summarizations than baseline methods, achieving state-of-the-art results in terms of ROUGE metric. Additionally, we show that aspect information is extremely important in extractive summarization scenario.
RelO: An Overlapping Relation Extraction Dataset and Model
Authors: Anonymous
Date: 2021-09-17 00:00:00
Proceeding: ACL ARR September 2021 9
Tasks: model, Relation, Relation Extraction, Transfer Learning

We introduce an overlapping relation extraction dataset(RelO) constructed from Wikipedia and Wikidata. RelO consists of 308,579 sentences, representing 40 relations and three different overlapping types. We evaluate the state-of-the-art relation extraction methods on RelO and results show that RelO is challenging for these relation extraction methods. We also use RelO as a transfer learning resources and the fine-tuned two models achieve state-of-the-art results on other related datasets. To handle the overlapping relation extraction task, We extend a pipeline framework by utilizing overlapping type information and semantic distance information of relation representations. Through careful experiments, we validate the importance of these information for overlapping relation extraction. All details and resources about the dataset and baselines are released on https://github.com/disquietBookShare/Relo.
Test paper
Authors: F Correia
Date: 2020-06-02 00:00:00
Proceeding: gilealliance XP Workshop AMP 2020 6

This is just a test paper