Reviewerly/exHarmony huggingface dataset:
ParaMac: A General Unsupervised Paraphrase Generation Framework Leveraging Semantic Constraints and Diversifying Mechanisms

Paraphrase generation reflects the ability to understand the meaning from the language surface form and rephrase it to other expressions.Recent paraphrase generation works have paid attention to unsupervised approaches based on Pre-trained Language Models (PLMs) to avoid heavy reliance on parallel data by utilizing PLMs' generation ability.However, the generated pairs of existing unsupervised methods are usually weak either in semantic equivalence or expression diversity.In this paper, we present a novel unsupervised paraphrase generation framework called Paraphrase Machine.By employing multi-aspect equivalence constraints and multi-granularity diversifying mechanisms, Paraphrase Machine is able to achieve good semantic equivalence and expressive diversity, producing a high-quality unsupervised paraphrase dataset.Based on this dataset, we train a general paraphrase model, which can be directly applied to rewrite the input sentence of various domains without any fine-tuning, and achieves substantial gains of 9.1% and 3.3% absolutely in BLEU score over previous SOTA on Quora and MSCOCO.By further fine-tuning our model with domain-specific training sets, the improvement can be increased to even 18.0% and 4.6%.Most importantly, by applying it to language understanding and generation tasks under the low-resource setting, we demonstrate that our model can serve as a universal data augmentor to boost the few-shot performance (e.g., average 2.0% gain on GLUE).Code and data can be found at https://github.com/Matthewlliu/UnsupervisedParaphrase.
IRRGN: An Implicit Relational Reasoning Graph Network for Multi-turn Response Selection

The task of response selection in multi-turn dialogue is to find the best option from all candidates.In order to improve the reasoning ability of the model, previous studies pay more attention to using explicit algorithms to model the dependencies between utterances, which are deterministic, limited and inflexible.In addition, few studies consider differences between the options before and after reasoning.In this paper, we propose an Implicit Relational Reasoning Graph Network to address these issues, which consists of the Utterance Relational Reasoner (URR) and the Option Dual Comparator (ODC).URR aims to implicitly extract dependencies between utterances, as well as utterances and options, and make reasoning with relational graph convolutional networks.ODC focuses on perceiving the difference between the options through dual comparison, which can eliminate the interference of the noise options.Experimental results on two multi-turn dialogue reasoning benchmark datasets Mu-Tual and MuTual plus show that our method significantly improves the baseline of four pretrained language models and achieves state-ofthe-art performance.The model surpasses human performance for the first time on the Mu-Tual dataset.Our code is released in the link.
WeDef: Weakly Supervised Backdoor Defense for Text Classification

Existing backdoor defense methods are only effective for limited trigger types.To defend different trigger types at once, we start from the class-irrelevant nature of the poisoning process and propose a novel weakly supervised backdoor defense framework WeDef.Recent advances in weak supervision make it possible to train a reasonably accurate text classifier using only a small number of user-provided, class-indicative seed words.Such seed words shall be considered independent of the triggers.Therefore, a weakly supervised text classifier trained by only the poisoned documents without their labels will likely have no backdoor.Inspired by this observation, in WeDef, we define the reliability of samples based on whether the predictions of the weak classifier agree with their labels in the poisoned training set.We further improve the results through a two-phase sanitization: (1) iteratively refine the weak classifier based on the reliable samples and (2) train a binary poison classifier by distinguishing the most unreliable samples from the most reliable samples.Finally, we train the sanitized model on the samples that the poison classifier predicts as benign.Extensive experiments show that WeDef is effective against popular trigger-based attacks (e.g., words, sentences, and paraphrases), outperforming existing defense methods.
FCGCL: Fine- and Coarse-Granularity Contrastive Learning for Speech Translation

It is notoriously difficult to implement endto-end speech translation (E2E-ST) model because of the task complexity and data scarcity.Existing techniques often attempt to carry out implicit knowledge transfer from machine translation (MT) to ST model by imposing various constraints.However, in this transfer scenario, a significant problem is that the performance of the MT will drop significantly and the final transfer effect is also restricted.In this article, we recommend Fine and Coarse Granularity Contrastive Learning (FCGCL), which conduct explicit knowledge transfer from MT to ST model.Specially, we ensure through multi granularity contrastive learning that inputs with similar semantic between different modalities are encoded closely in the shared semantic space while inputs with different semantics are kept apart.Experiments on the MuST-C datasets on all 8 languages and further analysis show that our method can effectively improve the E2E-ST performance and achieves an average BLEU of 29.0 1 .
A Benchmark Study of Contrastive Learning for Arabic Social Meaning

Contrastive learning (CL) brought significant progress to various NLP tasks.Despite this progress, CL has not been applied to Arabic NLP to date.Nor is it clear how much benefits it could bring to particular classes of tasks such as those involved in Arabic social meaning (e.g., sentiment analysis, dialect identification, hate speech detection).In this work, we present a comprehensive benchmark study of state-ofthe-art supervised CL methods on a wide array of Arabic social meaning tasks.Through extensive empirical analyses, we show that CL methods outperform vanilla finetuning on most tasks we consider.We also show that CL can be data efficient and quantify this efficiency.Overall, our work allows us to demonstrate the promise of CL methods, including in low-resource settings.
Can Transformers Reason in Fragments of Natural Language?

State-of-the-artdeep-learning-based approaches to Natural Language Processing (NLP) are credited with various capabilities that involve reasoning with natural language texts.In this paper we carry out a large-scale empirical study investigating the detection of formally valid inferences in controlled fragments of natural language for which the satisfiability problem becomes increasingly complex.We find that, while transformerbased language models perform surprisingly well in these scenarios, a deeper analysis reveals that they appear to overfit to superficial patterns in the data rather than acquiring the logical principles governing the reasoning in these fragments.
FlowEval: A Consensus-Based Dialogue Evaluation Framework Using Segment Act Flows

Despite recent progress in open-domain dialogue evaluation, how to develop automatic metrics remains an open problem.We explore the potential of dialogue evaluation featuring dialog act information, which was hardly explicitly modeled in previous methods.However, defined at the utterance level in general, dialog act is of coarse granularity, as an utterance can contain multiple segments possessing different functions.Hence, we propose segment act, an extension of dialog act from utterance level to segment level, and crowdsource a largescale dataset for it.To utilize segment act flows, sequences of segment acts, for evaluation, we develop the first consensus-based dialogue evaluation framework, FlowEval.This framework provides a reference-free approach for dialog evaluation by finding pseudo-references.Extensive experiments against strong baselines on three benchmark datasets demonstrate the effectiveness and other desirable characteristics of our FlowEval, pointing out a potential path for better dialogue evaluation.
Temporal Word Meaning Disambiguation using TimeLMs

Meaning of words constantly change given the events in modern civilization.Large Language Models use word embeddings, which are often static and thus cannot cope with this semantic change.Thus,it is important to resolve ambiguity in word meanings.This paper is an effort in this direction, where we explore methods for word sense disambiguation for the EvoNLP shared task.We conduct rigorous ablations for two solutions to this problem.We see that an approach using time-aware language models helps this task.Furthermore, we explore possible future directions to this problem.
Improving the Extraction of Supertags for Constituency Parsing with Linear Context-Free Rewriting Systems

In parsing phrase structures, supertagging achieves a symbiosis between the interpretability of formal grammars and the accuracy and speed of more recent neural models.The approach was only recently transferred to parsing discontinuous constituency structures with linear context-free rewriting systems (LCFRS).We reformulate and parameterize the previously fixed extraction process for LCFRS supertags with the aim to improve the overall parsing quality.These parameters are set in the context of several steps in the extraction process and are used to control the granularity of extracted grammar rules as well as the association of lexical symbols with each supertag.We evaluate the influence of the parameters on the sets of extracted supertags and the parsing quality using three treebanks in the English and German language, and we compare the best-performing configurations to recent state-of-the-art parsers in the area.Our results show that some of our configurations and the slightly modified parsing process improve the quality and speed of parsing with our supertags over the previous approach.Moreover, we achieve parsing scores that either surpass or are among the state-of-the-art in discontinuous constituent parsing.
CRIPP-VQA: Counterfactual Reasoning about Implicit Physical Properties via Video Question Answering

Videos often capture objects, their visible properties, their motion, and the interactions between different objects.Objects also have physical properties such as mass, which the imaging pipeline is unable to directly capture.However, these properties can be estimated by utilizing cues from relative object motion and the dynamics introduced by collisions.In this paper, we introduce CRIPP-VQA 1 , a new video question answering dataset for reasoning about the implicit physical properties of objects in a scene.CRIPP-VQA contains videos of objects in motion, annotated with questions that involve counterfactual reasoning about the effect of actions, questions about planning in order to reach a goal, and descriptive questions about visible properties of objects.The CRIPP-VQA test set enables evaluation under several outof-distribution settings -videos with objects with masses, coefficients of friction, and initial velocities that are not observed in the training distribution.Our experiments reveal a surprising and significant performance gap in terms of answering questions about implicit properties (the focus of this paper) and explicit properties of objects (the focus of prior work).
A Critical Reflection and Forward Perspective on Empathy and Natural Language Processing

We review the state of research on empathy in natural language processing and identify the following issues: (1) empathy definitions are absent or abstract, which (2) leads to low construct validity and reproducibility.Moreover, (3) emotional empathy is overemphasized, skewing our focus to a narrow subset of simplified tasks.We believe these issues hinder research progress and argue that current directions will benefit from a clear conceptualization that includes operationalizing cognitive empathy components.Our main objectives are to provide insight and guidance on empathy conceptualization for NLP research objectives and to encourage researchers to pursue the overlooked opportunities in this area, highly relevant, e.g., for clinical and educational sectors.
Natural Language Deduction through Search over Statement Compositions

In settings from fact-checking to question answering, we frequently want to know whether a collection of evidence (premises) entails a hypothesis.Existing methods primarily focus on the end-to-end discriminative version of this task, but less work has treated the generative version in which a model searches over the space of statements entailed by the premises to constructively derive the hypothesis.We propose a system for doing this kind of deductive reasoning in natural language by decomposing the task into separate steps coordinated by a search procedure, producing a tree of intermediate conclusions that faithfully reflects the system's reasoning process.Our experiments on the EntailmentBank dataset (Dalvi et al., 2021) demonstrate that the proposed system can successfully prove true statements while rejecting false ones.Moreover, it produces natural language explanations with a 17% absolute higher step validity than those produced by an end-to-end T5 model.
Impact of Sequence Length and Copying on Clause-Level Inflection

We present the University of British Columbia's submission to the MRL shared task on multilingual clause-level morphology.Our submission extends word-level inflectional models to the clause-level in two ways: first, by evaluating the role that BPE has on the learning of inflectional morphology, and second, by evaluating the importance of a copy bias obtained through data hallucination.Experiments demonstrate a strong preference for language-tuned BPE and a copy bias over a vanilla transformer.The methods are complementary for inflection and analysis tasks -combined models see error reductions of 38% for inflection and 15.6% for analysis; However, this synergy does not hold for reinflection, which performs best under a BPE-only setting.A deeper analysis of the errors generated by our models illustrates that the copy bias may be too strong -the combined model produces predictions more similar to the copy-influenced system, despite the success of the BPE-model.
Exploiting In-Domain Bilingual Corpora for Zero-Shot Transfer Learning in NLU of Intra-Sentential Code-Switching Chatbot Interactions

Code-switching (CS) is a very common phenomenon in regions with various co-existing languages.Since CS is such a frequent habit in informal communications, both spoken and written, it also arises naturally in Human-Machine Interactions.Therefore, in order for natural language understanding (NLU) not to be degraded, CS must be taken into account when developing chatbots.The co-existence of multiple languages in a single NLU model has become feasible with multilingual language representation models such as mBERT.In this paper, the efficacy of zero-shot cross-lingual transfer learning with mBERT for NLU is evaluated on a Basque-Spanish CS chatbot corpus, comparing the performance of NLU models trained using in-domain chatbot utterances in Basque and/or Spanish without CS.The results obtained indicate that training joint multi-intent classification and entity recognition models on both languages simultaneously achieves best performance, better capturing the CS patterns.
ProofInfer: Generating Proof via Iterative Hierarchical Inference

Proof generation focuses on deductive reasoning: given a hypothesis and a set of theories, including some supporting facts and logical rules expressed in natural language, the model generates a proof tree indicating how to deduce the hypothesis from given theories.Current models with state-of-theart performance employ the stepwise method, linking an individual node to the proof step-bystep.However, these methods actually focus on generating several proof paths rather than a whole tree.To address this problem, we propose ProofInfer, which generates the proof tree via iterative hierarchical inference.At each step, ProofInfer generates the entire layer for proof tree, where all nodes in this layer are generated simultaneously.Since the conventional autoregressive generation architecture cannot simultaneously predict multiple nodes, ProofInfer employs text-to-text paradigm to avoid it.To this end, we propose a divideand-conquer algorithm to encode the proof tree as the plain text recursively without structure information loss.Experimental results show that ProofInfer significantly outperforms the state-of-the-art (SOTA) models on several widely-used datasets.In addition, ProofInfer still performs well with data-limited, achieving comparable performance to the SOTA models with only 40% of the training data. 1
Digging Errors in NMT: Evaluating and Understanding Model Errors from Partial Hypothesis Space

Solid evaluation of neural machine translation (NMT) is key to its understanding and improvement. Current evaluation of an NMT system is usually built upon a heuristic decoding algorithm (e.g., beam search) and an evaluation metric assessing similarity between the translation and golden reference. However, this system-level evaluation framework is limited by evaluating only one best hypothesis and search errors brought by heuristic decoding algorithms. To better understand NMT models, we propose a novel evaluation protocol, which defines model errors with model’s ranking capability over hypothesis space. To tackle the problem of exponentially large space, we propose two approximation methods, top region evaluation along with an exact top-k decoding algorithm, which finds top-ranked hypotheses in the whole hypothesis space, and Monte Carlo sampling evaluation, which simulates hypothesis space from a broader perspective. To quantify errors, we define our NMT model errors by measuring distance between the hypothesis array ranked by the model and the ideally ranked hypothesis array. After confirming the strong correlation with human judgment, we apply our evaluation to various NMT benchmarks and model architectures. We show that the state-of-the-art Transformer models face serious ranking issues and only perform at the random chance level in the top region. We further analyze model errors on architectures with different depths and widths, as well as different data-augmentation techniques, showing how these factors affect model errors. Finally, we connect model errors with the search algorithms and provide interesting findings of beam search inductive bias and correlation with Minimum Bayes Risk (MBR) decoding.
ARGUABLY @ Causal News Corpus 2022: Contextually Augmented Language Models for Event Causality Identification

Causal (a cause-effect relationship between two arguments) has become integral to various NLP domains such as question answering, summarization, and event prediction.To understand causality in detail, Event Causality Identification with Causal News Corpus (CASE-2022) has organized shared tasks.This paper defines our participation in Subtask 1, which focuses on classifying event causality.We used sentence level augmentation based on contextualized word embeddings of distillBERT to construct new data.This data was then trained using two approaches.The first technique used the DeBERTa language model, and the second used the RoBERTa language model in combination with cross attention.We obtained the second-best F1 score (0.8610) in the competition with Contextually Augmented DeBERTa model.
Progressive Sentiment Analysis for Code-Switched Text Data

Multilingual transformer language models have recently attracted much attention from researchers and are used in cross-lingual transfer learning for many NLP tasks such as text classification and named entity recognition.However, similar methods for transfer learning from monolingual text to code-switched text have not been extensively explored mainly due to the following challenges: (1) Codeswitched corpus, unlike monolingual corpus, consists of more than one language and existing methods can't be applied efficiently, (2) Codeswitched corpus is usually made of resourcerich and low-resource languages and upon using multilingual pre-trained language models, the final model might bias towards resourcerich language.In this paper, we focus on code-switched sentiment analysis where we have a labelled resource-rich language dataset and unlabelled code-switched data.We propose a framework that takes the distinction between resource-rich and low-resource language into account.Instead of training on the entire code-switched corpus at once, we create buckets based on the fraction of words in the resource-rich language and progressively train from resource-rich language dominated samples to low-resource language dominated samples.Extensive experiments across multiple language pairs demonstrate that progressive training helps low-resource language dominated samples.
QaDialMoE: Question-answering Dialogue based Fact Verification with Mixture of Experts

Fact verification is an essential tool to mitigate the spread of false information online, which has gained a widespread attention recently.However, a fact verification in the questionanswering dialogue is still underexplored.In this paper, we propose a neural network based approach called question-answering dialogue based fact verification with mixture of experts (QaDialMoE).It exploits questions and evidence effectively in the verification process and can significantly improve the performance of fact verification.Specifically, we exploit the mixture of experts to focus on various interactions among responses, questions and evidence.A manager with an attention guidance module is implemented to guide the training of experts and assign a reasonable attention score to each expert.A prompt module is developed to generate synthetic questions that make our approach more generalizable.Finally, we evaluate the QaDialMoE and conduct a comparative study on three benchmark datasets.The experimental results demonstrate that our Qa-DialMoE outperforms previous approaches by a large margin and achieves new state-of-theart results on all benchmarks.This includes the accuracy improvements on the HEALTHVER as 84.26%, the FAVIQ A dev set as 78.7%, the FAVIQ R dev set as 86.1%, test set as 86.0%, and the COLLOQUIAL as 89.5%.To our best knowledge, this is the first work to investigate a question-answering dialogue based fact verification, and achieves new state-of-the-art results on various benchmark datasets.
Multimodal Knowledge Learning for Named Entity Disambiguation

With the popularity of online social media, massive-scale multimodal information has brought new challenges to traditional Named Entity Disambiguation (NED) tasks.Recently, Multimodal Named Entity Disambiguation (MNED) has been proposed to link ambiguous mentions with the textual and visual contexts to a predefined knowledge graph.Existing attempts usually perform MNED by annotating multimodal mentions and adding multimodal features to traditional NED models.However, these studies may suffer from 1) failing to model multimodal information at the knowledge level, and 2) lacking multimodal annotation data against the large-scale unlabeled corpus.In this paper, we explore a pioneer study on leveraging multimodal knowledge learning to address the MNED task.Specifically, we first harvest multimodal knowledge in the Meta-Learning way, which is much easier than collecting ambiguous mention corpus.Then we design a knowledge-guided transfer learning strategy to extract unified representation from different modalities.Finally, we propose an Interactive Multimodal Learning Network (IMN) to fully utilize the multimodal information on both the mention and knowledge sides.Extensive experiments conducted on two public MNED datasets demonstrate that the proposed method achieves improvements over the stateof-the-art multimodal methods.