Reviewerly/exHarmony huggingface dataset:
The Importance of Being Parameters: An Intra-Distillation Method for Serious Gains

Recent model pruning methods have demonstrated the ability to remove redundant parameters without sacrificing model performance.Common methods remove redundant parameters according to the parameter sensitivity, a gradient-based measure reflecting the contribution of the parameters.In this paper, however, we argue that redundant parameters can be trained to make beneficial contributions.We first highlight the large sensitivity (contribution) gap among high-sensitivity and lowsensitivity parameters and show that the model generalization performance can be significantly improved after balancing the contribution of all parameters.Our goal is to balance the sensitivity of all parameters and encourage all of them to contribute equally.We propose a general task-agnostic method, namely intradistillation, appended to the regular training loss to balance parameter sensitivity.Moreover, we also design a novel adaptive learning method to control the strength of intradistillation loss for faster convergence.Our experiments show the strong effectiveness of our methods on machine translation, natural language understanding, and zero-shot crosslingual transfer across up to 48 languages 1 , e.g., a gain of 3.54 BLEU on average across 8 language pairs from the IWSLT'14 dataset.
A Multilingual Perspective Towards the Evaluation of Attribution Methods in Natural Language Inference

Most evaluations of attribution methods focus on the English language.In this work, we present a multilingual approach for evaluating attribution methods for the Natural Language Inference (NLI) task in terms of faithfulness and plausibility.First, we introduce a novel cross-lingual strategy to measure faithfulness based on word alignments, which eliminates the drawbacks of erasure-based evaluations.We then perform a comprehensive evaluation of attribution methods, considering different output mechanisms and aggregation methods.Finally, we augment the XNLI dataset with highlight-based explanations, providing a multilingual NLI dataset with highlights, to support future exNLP studies.Our results show that attribution methods performing best for plausibility and faithfulness are different.
Hands-On Interactive Neuro-Symbolic NLP with DRaiL

We recently introduced DRaiL, a declarative neuro-symbolic modeling framework designed to support a wide variety of NLP scenarios.In this demo, we enhance DRaiL with an easy to use Python interface equipped with methods to define, modify and augment models interactively, as well as with methods to debug and visualize the predictions made.We demonstrate this interface with two challenging NLP tasks: analyzing moral sentiment in political discourse, and analyzing opinions about the Covid-19 vaccine.
Towards Reinterpreting Neural Topic Models via Composite Activations

Most Neural Topic Models (NTM) use a variational auto-encoder framework producing K topics limited to the size of the encoder's output.These topics are interpreted through the selection of the top activated words via the weights or reconstructed vector of the decoder that are directly connected to each neuron.In this paper, we present a model-free two-stage process to reinterpret NTM and derive further insights on the state of the trained model.Firstly, building on the original information from a trained NTM, we generate a pool of potential candidate "composite topics" by exploiting possible co-occurrences within the original set of topics, which decouples the strict interpretation of topics from the original NTM.This is followed by a combinatorial formulation to select a final set of composite topics, which we evaluate for coherence and diversity on a large external corpus.Lastly, we employ a user study to derive further insights on the reinterpretation process.
Modeling Information Change in Science Communication with Semantically Matched Paraphrases

Whether the media faithfully communicate scientific information has long been a core issue to the science community.Automatically identifying paraphrased scientific findings could enable large-scale tracking and analysis of information changes in the science communication process, but this requires systems to understand the similarity between scientific information across multiple domains.To this end, we present the SCIENTIFIC PARAPHRASE AND INFOR-MATION CHANGE DATASET (SPICED), the first paraphrase dataset of scientific findings annotated for degree of information change.SPICED contains 6,000 scientific finding pairs extracted from news stories, social media discussions, and full texts of original papers.We demonstrate that SPICED poses a challenging task and that models trained on SPICED improve downstream performance on evidence retrieval for fact checking of real-world scientific claims.Finally, we show that models trained on SPICED can reveal large-scale trends in the degrees to which people and organizations faithfully communicate new scientific findings.Data, code, and pre-trained models are available at http://www.copenlu.
TF-IDF or Transformers for Arabic Dialect Identification? ITFLOWS participation in the NADI 2022 Shared Task

This study targets the shared task of Nuanced Arabic Dialect Identification (NADI) organized with the Workshop on Arabic Natural Language Processing (WANLP).It further focuses on Subtask 1: the identification of the Arabic dialects at the country level.More specifically, it studies the impact of a traditional approach such as TF-IDF and then moves on to study the impact of advanced deep learning based methods.These methods include fully finetuning MARBERT as well as adapter based fine-tuning of MARBERT with and without performing data augmentation.The evaluation shows that the traditional approach based on TF-IDF scores the best in terms of accuracy on TEST-A dataset, while, the fine-tuned MAR-BERT with adapter on augmented data scores the second on Macro F1-score on the TEST-B dataset.This led to the proposed system being ranked second on the shared task on average.
SLATE: A Sequence Labeling Approach for Task Extraction from Free-form Inked Content

We present SLATE, a sequence labeling approach for extracting tasks from free-form content such as digitally handwritten (or "inked") notes on a virtual whiteboard.Our approach allows us to create a single, low-latency model to simultaneously perform sentence segmentation and classification of these sentences into task/non-task sentences.SLATE greatly outperforms a baseline two-model (sentence segmentation followed by classification model) approach, achieving a task F1 score of 84.4%, a sentence segmentation (boundary similarity) score of 88.4% and three times lower latency compared to the baseline.Furthermore, we provide insights into tackling challenges of performing NLP on the inking domain.We release both our code and dataset for this novel task.
Topic-Regularized Authorship Representation Learning

Authorship attribution is a task that aims to identify the author of a given piece of writing.We aim to develop a generalized solution that can handle a large number of texts from authors and topics unavailable in training data.Previous studies have proposed strategies to address only either unseen authors or unseen topics.Authorship representation learning has been shown to work in open-set environments with a large number of unseen authors but has not been explicitly designed for cross-topic environments at the same time.To handle a large number of unseen authors and topics, we propose Authorship Representation Regularization (ARR), a distillation framework that creates authorship representation with reduced reliance on topic-specific information.To assess the performance of our framework, we also propose a cross-topic-open-set evaluation method.Our proposed method has improved performances in the cross-topic-open set setup over baselines in 4 out of 6 cases.
Modeling Consistency Preference via Lexical Chains for Document-level Neural Machine Translation

In this paper we aim to relieve the issue of lexical translation inconsistency for documentlevel neural machine translation (NMT) by modeling consistency preference for lexical chains which consist of repeated words in a source-side document and provide a representation of the lexical consistency structure of the document.Specifically, we first propose lexical-consistency attention to capture consistency context among words in the same lexical chains.Then for each lexical chain we define and learn a consistency-tailored latent variable, which will guide the translation of corresponding sentences to enhance lexical translation consistency.Experimental results on Chinese→English and French→English document-level translation tasks show that our approach not only significantly improves translation performance in BLEU, but also substantially alleviates the problem of the lexical translation inconsistency.
Improving Text-to-SQL Semantic Parsing with Fine-grained Query Understanding

Most recent research on Text-to-SQL semantic parsing relies on either parser itself or simple heuristic based approach to understand natural language query (NLQ).When synthesizing a SQL query, there is no explicit semantic information of NLQ available to the parser which leads to undesirable generalization performance.In addition, without lexical-level finegrained query understanding, linking between query and database can only rely on fuzzy string match which leads to suboptimal performance in real applications.In view of this, in this paper we present a general-purpose, modular neural semantic parsing framework that is based on token-level fine-grained query understanding.Our framework consists of three modules: named entity recognizer (NER), neural entity linker (NEL) and neural semantic parser (NSP).By jointly modeling query and database, NER model analyzes user intents and identifies entities in the query.NEL model links typed entities to schema and cell values in database.Parser model leverages available semantic information and linking results and synthesizes tree-structured SQL queries based on dynamically generated grammar.Experiments on SQUALL, a newly released semantic parsing dataset, show that we can achieve 56.8% execution accuracy on WikiTableQuestions (WTQ) test set, which outperforms the state-of-the-art model by 2.7%.
Is it out yet? Automatic Future Product Releases Extraction from Web Data

Identifying the release of new products and their predicted demand in advance is highly valuable for E-Commerce marketplaces and retailers.The information of an upcoming product release is used for inventory management, marketing campaigns and pre-order suggestions.Often, the announcement of an upcoming product release is widely available in multiple web pages such as blogs, chats or news articles.However, to the best of our knowledge, an automatic system to extract future product releases from web data has not been presented.In this work we describe an ML-powered multistage pipeline to automatically identify future product releases and rank their predicted demand from unstructured pages across the whole web.Our pipeline includes a novel Longformerbased model which uses a global attention mechanism guided by pre-calculated Named Entity Recognition predictions related to product releases.The model training data is based on a new corpus of 30K web pages manually annotated to identify future product releases.
Cognitive Simplification Operations Improve Text Simplification

Text Simplification (TS) is the task of converting a text into a form that is easier to read while maintaining the meaning of the original text.A sub-task of TS is Cognitive Simplification (CS), converting text to a form that is readily understood by people with cognitive disabilities without rendering it childish or simplistic.This sub-task has yet to be explored with neural methods in NLP, and resources for it are scarcely available.In this paper, we present a method for incorporating knowledge from the cognitive accessibility domain into a TS model, by introducing an inductive bias regarding what simplification operations to use.We show that by adding this inductive bias to a TS-trained model, it is able to adapt better to CS without ever seeing CS data, and outperform a baseline model on a traditional TS benchmark.In addition, we provide a novel test dataset for CS, and analyze the differences between CS corpora and existing TS corpora, in terms of how simplification operations are applied.
IDIAPers @ Causal News Corpus 2022: Extracting Cause-Effect-Signal Triplets via Pre-trained Autoregressive Language Model

In this paper, we describe our shared task submissions for Subtask 2 in CASE-2022, Event Causality Identification with Casual News Corpus.The challenge focused on the automatic detection of all cause-effect-signal spans present in the sentence from news-media.We detect cause-effect-signal spans in a sentence using T5 -a pre-trained autoregressive language model.We iteratively identify all cause-effectsignal span triplets, always conditioning the prediction of the next triplet on the previously predicted ones.To predict the triplet itself, we consider different causal relationships such as cause→effect→signal.Each triplet component is generated via a language model conditioned on the sentence, the previous parts of the current triplet, and previously predicted triplets.Despite training on an extremely small dataset of 160 samples, our approach achieved competitive performance, being placed second in the competition.Furthermore, we show that assuming either cause→effect or effect→cause order achieves similar results. 1
Arabic dialect identification using machine learning and transformer-based models: Submission to the NADI 2022 Shared Task

Dialect is the language variation of a specific community.In this paper, we show the models we created to participate in the third Nuanced Arabic Dialect Identification (NADI) shared task (Subtask 1) that involves developing a system to classify a tweet into a country-level dialect.We utilized several machine learning techniques as well as deep learning transformer-based models.For the machine learning approach, we build an ensemble classifier of various machine learning models.In our deep learning approach, we consider bidirectional LSTM model and AraBERT pretrained model.The results demonstrate that the deep learning approach performs noticeably better than the other machine learning approaches with 68.7% accuracy on the development set.'USER'.Around 10M unlabeled tweets were also provided to participating teams by the NADI shared task organizers.
CMCC: A Comprehensive and Large-Scale Human-Human Dataset for Dialogue Systems

Dialogue modeling problems severely limit the real-world deployment of neural conversational models and building a human-like dialogue agent is an extremely challenging task.Recently, data-driven models become more and more prevalent which need a huge amount of conversation data.In this paper, we release around 100,000 dialogue, which come from real-world dialogue transcripts between real users and customer-service staffs.We call this dataset as CMCC (China Mobile Customer Care) dataset, which differs from existing dialogue datasets in both size and nature significantly.The dataset reflects several characteristics of human-human conversations, e.g., task-driven, care-oriented, and long-term dependency among the context.It also covers various dialogue types including task-oriented, chitchat and conversational recommendation in real-world scenarios.To our knowledge, CMCC is the largest real human-human spoken dialogue dataset and has dozens of times the data scale of others, which shall significantly promote the training and evaluation of dialogue modeling methods.The results of extensive experiments indicate that CMCC is challenging and needs further effort.We hope that this resource will allow for more effective models across various dialogue sub-problems to be built in the future.
Cross-modal Transfer Between Vision and Language for Protest Detection

Most of today's systems for socio-political event detection are text-based, while an increasing amount of information published on the web is multi-modal.We seek to bridge this gap by proposing a method that utilizes existing annotated unimodal data to perform event detection in another data modality, zero-shot.Specifically, we focus on protest detection in text and images, and show that a pretrained vision-and-language alignment model (CLIP) can be leveraged towards this end.In particular, our results suggest that annotated protest text data can act supplementarily for detecting protests in images, but significant transfer is demonstrated in the opposite direction as well.
Curriculum Knowledge Distillation for Emoji-supervised Cross-lingual Sentiment Analysis

Existing sentiment analysis models have achieved great advances with the help of sufficient sentiment annotations.Unfortunately, many languages do not have sufficient sentiment corpus.To this end, recent studies have proposed cross-lingual sentiment analysis to transfer sentiment analysis models from resource-rich languages to low-resource languages.However, these studies either rely on external cross-lingual supervision (e.g., parallel corpora and translation model), or are limited by the cross-lingual gaps.In this work, based on the intuitive assumption that the relationships between emojis and sentiments are consistent across different languages, we investigate transferring sentiment knowledge across languages with the help of emojis.To this end, we propose a novel cross-lingual sentiment analysis approach dubbed Curriculum Knowledge Distiller (CKD).The core idea of CKD is to use emojis to bridge the source and target languages.Note that, compared with texts, emojis are more transferable, but cannot reveal the precise sentiment.Thus, we distill multiple Intermediate Sentiment Classifiers (ISC) on source language corpus with emojis to get ISCs with different attention weights of texts.To transfer them into the target language, we distill ISCs into the Target Language Sentiment Classifier (TSC) following the curriculum learning mechanism.In this way, TSC can learn delicate sentiment knowledge, meanwhile, avoid being affected by cross-lingual gaps.Experimental results on five cross-lingual benchmarks clearly verify the effectiveness of our approach.
Political Event Coding as Text-to-Text Sequence Generation

We report on the current status of an effort to produce political event data from unstructured text via a Transformer language model.Compelled by the current lack of publicly available and up-to-date event coding software, we seek to train a model that can produce structured political event records at the sentence level.Our approach differs from previous efforts in that we conceptualize this task as one of text-to-text sequence generation.We motivate this choice by outlining desirable properties of text generation models for the needs of event coding.To overcome the lack of sufficient training data, we also describe a method for generating synthetic text and event record pairs that we use to fit our model.
Is NLP Ready for Standardization?

While standardization is a well-established activity in other scientific fields such as telecommunications, networks or multimedia, in the field of AI and more specifically NLP it is still at its dawn.In this paper, we explore how various aspects of NLP (evaluation, data, tasks...) lack standards and how that can impact science, but also the society, the industry, and regulations.We argue that the numerous initiatives to rationalize the field and establish good practices are only the first step, and developing formal standards remains needed to bring further clarity to NLP research and industry, at a time where this community faces various crises regarding ethics or reproducibility.We thus encourage NLP researchers to contribute to existing and upcoming standardization projects, so that they can express their needs and concerns, while sharing their expertise.
An Unsupervised, Geometric and Syntax-aware Quantification of Polysemy

Polysemy is the phenomenon where a single word form possesses two or more related senses.It is an extremely ubiquitous part of natural language and analyzing it has sparked rich discussions in the linguistics, psychology and philosophy communities alike.With scarce attention paid to polysemy in computational linguistics, and even scarcer attention toward quantifying polysemy, in this paper, we propose a novel, unsupervised framework to compute and estimate polysemy scores for words in multiple languages.We infuse our proposed quantification with syntactic knowledge in the form of dependency structures.This informs the final polysemy scores of the lexicon motivated by recent linguistic findings that suggest there is an implicit relation between syntax and ambiguity/polysemy.We adopt a graph based approach by computing the discrete Ollivier Ricci curvature on a graph of the contextual nearest neighbors.We test our framework on curated datasets controlling for different sense distributions of words in 3 typologically diverse languages -English, French and Spanish.The effectiveness of our framework is demonstrated by significant correlations of our quantification with expert human annotated language resources like WordNet.We observe a 0.3 point increase in the correlation coefficient as compared to previous quantification studies in English.Our research leverages contextual language models and syntactic structures to empirically support the widely held theoretical linguistic notion that syntax is intricately linked to ambiguity/polysemy.
Inferring Implicit Relations in Complex Questions with Language Models

A prominent challenge for modern language understanding systems is the ability to answer implicit reasoning questions, where the required reasoning steps for answering the question are not mentioned in the text explicitly.In this work, we investigate why current models struggle with implicit reasoning question answering (QA) tasks, by decoupling inference of reasoning steps from their execution.We define a new task of implicit relation inference and construct a benchmark, IMPLICITRELATIONS, where given a question, a model should output a list of concept-relation pairs, where the relations describe the implicit reasoning steps required for answering the question.Using IM-PLICITRELATIONS, we evaluate models from the GPT-3 family and find that, while these models struggle on the implicit reasoning QA task, they often succeed at inferring implicit relations.This suggests that the challenge in implicit reasoning questions does not stem from the need to plan a reasoning strategy alone, but to do it while also retrieving and reasoning over relevant information.
Linguistic Elements of Engaging Customer Service Discourse on Social Media

Customers are rapidly turning to social media for customer support.While brand agents on these platforms are motivated and wellintentioned to help and engage with customers, their efforts are often ignored if their initial response to the customer does not match a specific tone, style, or topic the customer is aiming to receive.The length of a conversation can reflect the effort and quality of the initial response made by a brand toward collaborating and helping consumers, even when the overall sentiment of the conversation might not be very positive.Thus, through this study, we aim to bridge this critical gap in the existing literature by analyzing language's content and stylistic aspects such as expressed empathy, psycho-linguistic features, dialogue tags, and metrics for quantifying personalization of the utterances that can influence the engagement of an interaction.This paper demonstrates that we can predict engagement using initial customer and brand posts.