Reviewerly/exHarmony huggingface dataset:
Adaptive Ordered Information Extraction with Deep Reinforcement Learning

Information extraction (IE) has been studied extensively.The existing methods always follow a fixed extraction order for complex IE tasks with multiple elements to be extracted in one instance such as event extraction.However, we conduct experiments on several complex IE datasets and observe that different extraction orders can significantly affect the extraction results for a great portion of instances, and the ratio of sentences that are sensitive to extraction orders increases dramatically with the complexity of the IE task.Therefore, this paper proposes a novel adaptive ordered IE paradigm to find the optimal element extraction order for different instances, so as to achieve the best extraction results.We also propose an reinforcement learning (RL) based framework to generate optimal extraction order for each instance dynamically.Additionally, we propose a co-training framework adapted to RL to mitigate the exposure bias during the extractor training phase.Extensive experiments conducted on several public datasets demonstrate that our proposed method can beat previous methods and effectively improve the performance of various IE tasks, especially for complex ones. 1
Training for Grammatical Error Correction Without Human-Annotated L2 Learners’ Corpora

Grammatical error correction (GEC) is a challenging task for non-native second language (L2) learners and learning machines.Datadriven GEC learning requires as much humanannotated genuine training data as possible.However, it is difficult to produce larger-scale human-annotated data, and synthetically generated large-scale parallel training data is valuable for GEC systems.In this paper, we propose a method for rebuilding a corpus of synthetic parallel data using target sentences predicted by a GEC model to improve performance.Experimental results show that our proposed pre-training outperforms that on the original synthetic datasets.Moreover, it is also shown that our proposed training without human-annotated L2 learners' corpora is as practical as conventional full pipeline training with both synthetic datasets and L2 learners' corpora in terms of accuracy.
DivHSK: Diverse Headline Generation using Self-Attention based Keyword Selection

Diverse headline generation is an NLP task where given a news article, the goal is to generate multiple headlines that are true to the content of the article, but are different among themselves.This task aims to exhibit and exploit semantically similar one-to-many relationships between a source news article and multiple target headlines.Towards this, we propose a novel model called DIVHSK.It has two components: KEYSELECT for selecting the important keywords, and SEQGEN, for finally generating the multiple diverse headlines.In KEYSELECT, we cluster the self-attention heads of the last layer of the pre-trained encoder and select the mostattentive theme and general keywords from the source article.Then, cluster-specific keyword sets guide the SEQGEN, a pre-trained encoderdecoder model, to generate diverse yet semantically similar headlines.The proposed model consistently outperformed existing literature and our strong baselines and emerged as a stateof-the-art model.Additionally, We have also created a high-quality multi-reference headline dataset from news articles 1 . Pre-Trained Encoder
WADER at SemEval-2023 Task 9: A Weak-labelling framework for Data augmentation in tExt Regression Tasks

Intimacy is an essential element of human relationships and language is a crucial means of conveying it.Textual intimacy analysis can reveal social norms in different contexts and serve as a benchmark for testing computational models' ability to understand social information.In this paper, we propose a novel weaklabeling strategy for data augmentation in text regression tasks called WADER.WADER uses data augmentation to address the problems of data imbalance and data scarcity and provides a method for data augmentation in cross-lingual, zero-shot tasks.We benchmark the performance of State-of-the-Art pre-trained multilingual language models using WADER and analyze the use of sampling techniques to mitigate bias in data and optimally select augmentation candidates.Our results show that WADER outperforms the baseline model and provides a direction for mitigating data imbalance and scarcity in text regression tasks.
NVIDIA NeMo Offline Speech Translation Systems for IWSLT 2023

This paper provides an overview of NVIDIA NeMo's speech translation systems for the IWSLT 2023 Offline Speech Translation Task.This year, we focused on end-to-end system which capitalizes on pre-trained models and synthetic data to mitigate the problem of direct speech translation data scarcity.When trained on IWSLT 2022 constrained data, our best En→De end-to-end model achieves the average score of 31 BLEU on 7 test sets from IWSLT 2010-2020 which improves over our last year cascade (28.4) and end-to-end (25.7) submissions.When trained on IWSLT 2023 constrained data, the average score drops to 29.5 BLEU.
Clemson NLP at SemEval-2023 Task 7: Applying GatorTron to Multi-Evidence Clinical NLI

This paper presents our system descriptions for SemEval 2023-Task 7: Multi-evidence Natural Language Inference for Clinical Trial Data sub-tasks one and two.Provided with a collection of Clinical Trial Reports (CTRs) and corresponding expert-annotated claim statements, sub-task one involves determining an inferential relationship between the statement and CTR premise: contradiction or entailment.Subtask two involves retrieving evidence from the CTR which is necessary to determine the entailment in sub-task one.For sub-task two we employ a recent transformer-based language model pretrained on biomedical literature, which we domain-adapt on a set of clinical trial reports.For sub-task one, we take an ensemble approach in which we leverage the evidence retrieval model from sub-task two to extract relevant sections, which are then passed to a second model of equivalent architecture to determine entailment.Our system achieves a ranking of seventh on sub-task one with an F1score of 0.705 and sixth on sub-task two with an F1-score of 0.806.In addition, we find that the high rate of success of language models on this dataset may be partially attributable to the existence of annotation artifacts. 1
Multi-CLS BERT: An Efficient Alternative to Traditional Ensembling

Ensembling BERT models often significantly improves accuracy, but at the cost of significantly more computation and memory footprint.In this work, we propose Multi-CLS BERT, a novel ensembling method for CLS-based prediction tasks that is almost as efficient as a single BERT model.Multi-CLS BERT uses multiple CLS tokens with a parameterization and objective that encourages their diversity.Thus instead of fine-tuning each BERT model in an ensemble (and running them all at test time), we need only fine-tune our single Multi-CLS BERT model (and run the one model at test time, ensembling just the multiple final CLS embeddings).To test its effectiveness, we build Multi-CLS BERT on top of a state-of-the-art pretraining method for BERT (Aroca-Ouellette and Rudzicz, 2020).In experiments on GLUE and SuperGLUE we show that our Multi-CLS BERT reliably improves both overall accuracy and confidence estimation.When only 100 training samples are available in GLUE, the Multi-CLS BERT Base model can even outperform the corresponding BERT Large model.We analyze the behavior of our Multi-CLS BERT, showing that it has many of the same characteristics and behavior as a typical BERT 5-way ensemble, but with nearly 4-times less computation and memory.
K-pop and fake facts: from texts to smart alerting for maritime security

Maritime security requires full-time monitoring of the situation, mainly based on technical data such as radar or Automatic Identification System (AIS) but also from Open Source Intelligence like inputs (e.g., newspapers).Some threats to the operational reliability of this maritime surveillance, such as malicious actors, introduce discrepancies between hard and soft data (sensors & texts), either by tweaking their AIS emitters or by emitting false information on pseudo-newspapers.Many techniques exist to identify these pieces of false information, including using knowledge base population techniques to build a structured view of the information.This paper presents a use case for suspect data identification in a maritime setting.The proposed system UMBAR ingests data from sensors and texts, processing them through an information extraction step, in order to feed a Knowledge Base (KB) and finally perform coherence checks between the extracted facts.
PropSegmEnt: A Large-Scale Corpus for Proposition-Level Segmentation and Entailment Recognition

The widely studied task of Natural Language Inference (NLI) requires a system to recognize whether one piece of text is textually entailed by another, i.e. whether the entirety of its meaning can be inferred from the other.In current NLI datasets and models, textual entailment relations are typically defined on the sentence-or paragraph-level.However, even a simple sentence often contains multiple propositions, i.e. distinct units of meaning conveyed by the sentence.As these propositions can carry different truth values in the context of a given premise, we argue for the need to recognize the textual entailment relation of each proposition in a sentence individually.We propose PROPSEGMENT, a corpus of over 45K propositions annotated by expert human raters.Our dataset structure aligns with the tasks of (1) segmenting sentences within a document to the set of propositions, and (2) classifying the entailment relation of each proposition with respect to a different yet topicallyaligned document, i.e. documents describing the same event or entity.We establish strong baselines for the segmentation and entailment tasks.Through case studies on summary hallucination detection and document-level NLI, we demonstrate that our conceptual framework is potentially useful for understanding and explaining the compositionality of NLI labels.
azaad@BND at SemEval-2023 Task 2: How to Go from a Simple Transformer Model to a Better Model to Get Better Results in Natural Language Processing

In this article, which was prepared for the sameval2023 competition (task number 2), information about the implementation techniques of the transformer model and the use of the pre-trained BERT model in order to identify the named entity (NER) in the English language, has been collected and also the implementation method is explained.Finally, it led to an F1 score of about 57% for Fine-grained and 72% for Coarse-grained in the dev data.In the final test data, F1 score reached 50%.
YNU-HPCC at SemEval-2023 Task 6: LEGAL-BERT Based Hierarchical BiLSTM with CRF for Rhetorical Roles Prediction

To understand a legal document for real-world applications, SemEval-2023 Task 6 proposes a shared Subtask A, rhetorical roles (RRs) prediction, which requires a system to automatically assign a RR label for each semantical segment in a legal text.In this paper, we propose a LEGAL-BERT based hierarchical BiLSTM model with conditional random field (CRF) for RR prediction, which primarily consists of two parts: word-level and sentence-level encoders.The word-level encoder first adopts a legaldomain pre-trained language model, LEGAL-BERT, initially word-embedding words in each sentence in a document and a word-level BiL-STM further encoding such sentence representation.The sentence-level encoder then uses an attentive pooling method for sentence embedding and a sentence-level BiLSTM for document modeling.Finally, a CRF is utilized to predict RRs for each sentence.The officially released results show that our method outperformed the baseline systems.Our team won 7th rank out of 27 participants in Subtask A.
Learning from a Friend: Improving Event Extraction via Self-Training with Feedback from Abstract Meaning Representation

Data scarcity has been the main factor that hinders the progress of event extraction.To overcome this issue, we propose a Self-Training with Feedback (STF) framework that leverages the large-scale unlabeled data and acquires feedback for each new event prediction from the unlabeled data by comparing it to the Abstract Meaning Representation (AMR) graph of the same sentence.Specifically, STF consists of (1) a base event extraction model trained on existing event annotations and then applied to large-scale unlabeled corpora to predict new event mentions as pseudo training samples, and (2) a novel scoring model that takes in each new predicted event trigger, an argument, its argument role, as well as their paths in the AMR graph to estimate a compatibility score indicating the correctness of the pseudo label.The compatibility scores further act as feedback to encourage or discourage the model learning on the pseudo labels during self-training.Experimental results on three benchmark datasets, including ACE05-E, ACE05-E + , and ERE, demonstrate the effectiveness of the STF framework on event extraction, especially event argument extraction, with significant performance gain over the base event extraction models and strong baselines.Our experimental analysis further shows that STF is a generic framework as it can be applied to improve most, if not all, event extraction models by leveraging largescale unlabeled data, even when high-quality AMR graph annotations are not available.
Rethinking Dictionaries and Glyphs for Chinese Language Pre-training

We introduce CDBERT, a new learning paradigm that enhances the semantics understanding ability of the Chinese Pretrained Language Models (PLMs) with dictionary knowledge and structure of Chinese characters.We name the two core modules of CDBERT as Shuowen and Jiezi, where Shuowen refers to the process of retrieving the most appropriate meaning from Chinese dictionaries and Jiezi refers to the process of enhancing characters' glyph representations with structure understanding.To facilitate dictionary understanding, we propose three pre-training tasks, i.e., Masked Entry Modeling, Contrastive Learning for Synonym and Antonym, and Example Learning.We evaluate our method on both modern Chinese understanding benchmark CLUE and ancient Chinese benchmark CCLUE.Moreover, we propose a new polysemy discrimination task PolyMRC based on the collected dictionary of ancient Chinese.Our paradigm demonstrates consistent improvements on previous Chinese PLMs across all tasks.Moreover, our approach yields significant boosting on few-shot setting of ancient Chinese understanding.
Indirectly Supervised Natural Language Processing

This tutorial targets researchers and practitioners who are interested in ML technologies for NLP from indirect supervision.In particular, we will present a diverse thread of indirect supervision studies that try to answer the following questions: (i) when and how can we provide supervision for a target task T , if all we have is data that corresponds to a "related" task T ′ ?(ii) humans do not use exhaustive supervision; they rely on occasional feedback, and learn from incidental signals from various sources; how can we effectively incorporate such supervision in machine learning?(iii) how can we leverage multi-modal supervision to help NLP?To the end, we will discuss several lines of research that address those challenges, including (i) indirect supervision from T ′ that handles T with outputs spanning from a moderate size to an open space, (ii) the use of sparsely occurring and incidental signals, such as partial labels, noisy labels, knowledge-based constraints, and cross-domain or cross-task annotationsall having statistical associations with the task, (iii) principled ways to measure and understand why these incidental signals can contribute to our target tasks, and (iv) indirect supervision from vision-language signals.We will conclude the tutorial by outlining directions for further investigation.
Search-Oriented Conversational Query Editing

Conversational query rewriting (CQR) realizes conversational search by reformulating the search dialogue into a standalone rewrite.However, existing CQR models either are not learned toward improving the downstream search performance or inefficiently generate the rewrite token-by-token from scratch while neglecting the fact that the search dialogue often has a large overlap with the rewrite.In this paper, we propose EDIRCS, a new text editingbased CQR model tailored for conversational search.In EDIRCS, most of the rewrite tokens are selected from the dialogue in a nonautoregressive fashion and only a few new tokens are generated to supplement the final rewrite, which makes EDIRCS highly efficient.In particular, the learning of EDIRCS is augmented with two search-oriented objectives, including contrastive ranking augmentation and contextualization knowledge transfer, which effectively improve it to select and generate more useful tokens from the view of retrieval.We show that EDIRCS outperforms state-of-theart CQR models on three conversational search benchmarks while having low rewriting latency, and is more robust to out-of-domain search dialogues and long dialogue context.
NormNet: Normalize Noun Phrases for More Robust NLP

A critical limitation of deep NLP models is their over-fitting over spurious features. Previous work has proposed several approaches to debunk such features and reduce their impact on the learned models. In this work, a normalization strategy is proposed to eliminate the false features caused by the textual surfaces of noun phrases. The motivation for this strategy is that noun phrases often play the role of slots in textual expressions and their exact forms are often not that important for performing the final task. As an intuitive example, consider the expression ”x like eating y". There are a huge number of suitable instantiations for x and y in the locale. However, humans can already infer the sentiment polarity of x toward y without knowing their exact forms.Based on this intuition, we introduce NormNet, a pretrained language model based network, to implement the normalization strategy. NormNet learns to replace as many noun phrases in the input sentence as possible with pre-defined base forms. The output of NormNet is then fed as input to a prompt-based learning model to perform label prediction. To evaluate the effectiveness of our strategy, we conducted experimental studies on several tasks, including aspect sentiment classification (ASC), semantic text similarity (STS), and natural language inference (NLI). The experimental results confirm the effectiveness of our strategy.
ACROSS: An Alignment-based Framework for Low-Resource Many-to-One Cross-Lingual Summarization

This research addresses the challenges of Cross-Lingual Summarization (CLS) in low-resource scenarios and over imbalanced multilingual data.Existing CLS studies mostly resort to pipeline frameworks or multi-task methods in bilingual settings.However, they ignore the data imbalance in multilingual scenarios and do not utilize the high-resource monolingual summarization data.In this paper, we propose the Aligned CROSs-lingual Summarization (ACROSS) model to tackle these issues.Our framework aligns lowresource cross-lingual data with high-resource monolingual data via contrastive and consistency loss, which help enrich low-resource information for high-quality summaries.In addition, we introduce a data augmentation method that can select informative monolingual sentences, which facilitates a deep exploration of high-resource information and introduce new information for low-resource languages.Experiments 1 on the CrossSum dataset show that ACROSS outperforms baseline models and obtains consistently dominant performance on 45 language pairs.
Fighting Bias With Bias: Promoting Model Robustness by Amplifying Dataset Biases

NLP models often rely on superficial cues known as dataset biases to achieve impressive performance, and can fail on examples where these biases do not hold. Recent work sought to develop robust, unbiased models by filtering biased examples from training sets. In this work, we argue that such filtering can obscure the true capabilities of models to overcome biases, which might never be removed in full from the dataset. We suggest that in order to drive the development of models robust to subtle biases, dataset biases should be amplified in the training set. We introduce an evaluation framework defined by a bias-amplified training set and an anti-biased test set, both automatically extracted from existing datasets. Experiments across three notions of bias, four datasets and two models show that our framework is substantially more challenging for models than the original data splits, and even more challenging than hand-crafted challenge sets. Our evaluation framework can use any existing dataset, even those considered obsolete, to test model robustness. We hope our work will guide the development of robust models that do not rely on superficial biases and correlations. To this end, we publicly release our code and data.
Difficulty-Controllable Neural Question Generation for Reading Comprehension using Item Response Theory

Question generation (QG) for reading comprehension, a technology for automatically generating questions related to given reading passages, has been used in various applications, including in education.Recently, QG methods based on deep neural networks have succeeded in generating fluent questions that are pertinent to given reading passages.One example of how QG can be applied in education is a reading tutor that automatically offers reading comprehension questions related to various reading materials.In such an application, QG methods should provide questions with difficulty levels appropriate for each learner's reading ability in order to improve learning efficiency.Several difficulty-controllable QG methods have been proposed for doing so.However, conventional methods focus only on generating questions and cannot generate answers to them.Furthermore, they ignore the relation between question difficulty and learner ability, making it hard to determine an appropriate difficulty for each learner.To resolve these problems, we propose a new method for generating questionanswer pairs that considers their difficulty, estimated using item response theory.The proposed difficulty-controllable generation is realized by extending two pre-trained transformer models: BERT and GPT-2.
Noise-Robust Training with Dynamic Loss and Contrastive Learning for Distantly-Supervised Named Entity Recognition

Distantly-supervised named entity recognition (NER) aims at training networks with distantlylabeled data, which is automatically obtained by matching entity mentions in the raw text with entity types in a knowledge base.Distant supervision may induce incomplete and noisy labels, so recent state-of-the-art methods employ sample selection mechanism to separate clean data from noisy data based on the model's prediction scores.However, they ignore the noise distribution change caused by data selection, and they simply excludes noisy data during training, resulting in information loss.We propose to (1) use a dynamic loss function to better adapt to the changing noise during the training process, and (2) incorporate token level contrastive learning to fully utilize the noisy data as well as facilitate feature learning without relying on labels.We conduct extensive experiments on multiple datasets and our method obtains 4.3%, 1.5%, 0.9% F1 score improvements over the current state-of-the-art on Wikigold, CoNLL03 and OntoNotes5.0.