Reviewerly/exHarmony huggingface dataset:
Sartipi-Sedighin at SemEval-2023 Task 2: Fine-grained Named Entity Recognition with Pre-trained Contextual Language Models and Data Augmentation from Wikipedia

This paper presents the system developed by the Sartipi-Sedighin team for SemEval 2023 Task 2, which is a shared task focused on multilingual complex named entity recognition (NER), or MultiCoNER II.The goal of this task is to identify and classify complex named entities (NEs) in text across multiple languages.To tackle the MultiCoNER II task, we leveraged pre-trained language models (PLMs) fine-tuned for each language included in the dataset.In addition, we also applied a data augmentation technique to increase the amount of training data available to our models.Specifically, we searched for relevant NEs that already existed in the training data within Wikipedia, and we added new instances of these entities to our training corpus.Our team achieved an overall F1 score of 61.25% in the English track and 71.79% in the multilingual track across all 13 tracks of the shared task that we submitted to.
ABCD Team at SemEval-2023 Task 12: An Ensemble Transformer-based System for African Sentiment Analysis

This paper describes the system of the ABCD team for three main tasks in the SemEval-2023 Task 12: AfriSenti-SemEval for Low-resource African Languages using Twitter Dataset.We focus on exploring the performance of ensemble architectures based on the soft voting technique and different pre-trained transformerbased language models.The experimental results show that our system has achieved competitive performance in some Tracks in Task A: Monolingual Sentiment Analysis, where we rank the Top 3, Top 2, and Top 4 for the Hause, Igbo and Moroccan languages.Besides, our model achieved competitive results and ranked 14 th place in Task B (multilingual) setting and 14 th and 8 th place in Track 17 and Track 18 of Task C (zero-shot) setting.
Discriminative Reasoning with Sparse Event Representation for Document-level Event-Event Relation Extraction

Document-level Event-Event Relation Extraction (DERE) aims to extract relations between events in a document.It challenges conventional sentence-level task (SERE) with difficult long-text understanding.In this paper, we propose a novel DERE model (SENDIR) for better document-level reasoning.Different from existing works that build an event graph via linguistic tools, SENDIR does not require any prior knowledge.The basic idea is to discriminate event pairs in the same sentence or span multiple sentences by assuming their different information density: 1) low density in the document suggests sparse attention to skip irrelevant information.Our module 1 designs various types of attention for event representation learning to capture long-distance dependence.2) High density in a sentence makes SERE relatively easy.Module 2 uses different weights to highlight the roles and contributions of intra-and intersentential reasoning, which introduces supportive event pairs for joint modeling.Extensive experiments demonstrate great improvements in SENDIR and the effectiveness of various sparse attention for document-level representations.Codes will be released later.
TALP-UPC at ProbSum 2023: Fine-tuning and Data Augmentation Strategies for NER

This paper describes the submission of the TALP-UPC team to the Problem List Summarization task from the BioNLP 2023 workshop.This task consists of automatically extracting a list of health issues from the e-health medical record of a given patient.Our submission combines additional steps of data annotation with finetuning of BERT pre-trained language models.Our experiments focus on the impact of finetuning on different datasets as well as the addition of data augmentation techniques to delay overfitting.
Enhancing Event Causality Identification with Event Causal Label and Event Pair Interaction Graph

Most existing event causality identification (ECI) methods rarely consider the event causal label information and the interaction information between event pairs.In this paper, we propose a framework to enrich the representation of event pairs by introducing the event causal label information and the event pair interaction information.In particular, 1) we design an event-causal-label-aware module to model the event causal label information, in which we design the event causal label prediction task as an auxiliary task of ECI, aiming to predict which events are involved in the causal relationship (we call them causality-related events) by mining the dependencies between events.2) We further design an event pair interaction graph module to model the interaction information between event pairs, in which we construct the interaction graph with event pairs as nodes and leverage graph attention mechanism to model the degree of dependency between event pairs.The experimental results show that our approach outperforms previous stateof-the-art methods on two benchmark datasets EventStoryLine and Causal-TimeBank.45 l ...
Compositional Mathematical Encoding for Math Word Problems

Solving math word problem (MWP) remains a challenging task, as it requires to understand both the semantic meanings of the text and the mathematical logic among quantities, i.e., for both semantics modal and quantity modal learning. Current MWP encoders work in a uni-modal setting and map the given problem description to a latent representation, then for decoding. The generalizability of these MWP encoders is thus limited because some problems are semantics-demanding and others are quantity-demanding. To address this problem, we propose a Compositional Math Word Problem Solver (C-MWP) which works in a bi-modal setting encoding in an interactive way. Extensive experiments validate the effectiveness of C-MWP and show its superiority over state-of-the-art models on public benchmarks.
TranSFormer: Slow-Fast Transformer for Machine Translation

Learning multiscale Transformer models has been evidenced as a viable approach to augmenting machine translation systems.Prior research has primarily focused on treating subwords as basic units in developing such systems.However, the incorporation of finegrained character-level features into multiscale Transformer has not yet been explored.In this work, we present a Slow-Fast two-stream learning model, referred to as TranSFormer, which utilizes a "slow" branch to deal with subword sequences and a "fast" branch to deal with longer character sequences.This model is efficient since the fast branch is very lightweight by reducing the model width, and yet provides useful fine-grained features for the slow branch.Our TranSFormer shows consistent BLEU improvements (larger than 1 BLEU point) on several machine translation benchmarks.
QA Domain Adaptation using Hidden Space Augmentation and Self-Supervised Contrastive Adaptation

Question answering (QA) has recently shown impressive results for answering questions from customized domains.Yet, a common challenge is to adapt QA models to an unseen target domain.In this paper, we propose a novel selfsupervised framework called QADA for QA domain adaptation.QADA introduces a novel data augmentation pipeline used to augment training QA samples.Different from existing methods, we enrich the samples via hidden space augmentation.For questions, we introduce multi-hop synonyms and sample augmented token embeddings with Dirichlet distributions.For contexts, we develop an augmentation method which learns to drop context spans via a custom attentive sampling strategy.Additionally, contrastive learning is integrated in the proposed self-supervised adaptation framework QADA.Unlike existing approaches, we generate pseudo labels and propose to train the model via a novel attention-based contrastive adaptation method.The attention weights are used to build informative features for discrepancy estimation that helps the QA model separate answers and generalize across source and target domains.To the best of our knowledge, our work is the first to leverage hidden space augmentation and attention-based contrastive adaptation for self-supervised domain adaptation in QA.Our evaluation shows that QADA achieves considerable improvements on multiple target datasets over state-of-the-art baselines in QA domain adaptation.
There Is No Standard Answer: Knowledge-Grounded Dialogue Generation with Adversarial Activated Multi-Reference Learning

Knowledge-grounded conversation (KGC) shows excellent potential to deliver an engaging and informative response.However, existing approaches emphasize selecting one golden knowledge given a particular dialogue context, overlooking the one-to-many phenomenon in dialogue.As a result, the existing paradigm limits the diversity of knowledge selection and generation.To this end, we establish a multireference KGC dataset and propose a series of metrics to systematically assess the one-tomany efficacy of existing KGC models.Furthermore, to extend the hypothesis space of knowledge selection to enhance the mapping relationship between multiple knowledge and multiple responses, we devise a span-based variational model and optimize the model in a wake-sleep style with an ameliorated evidence lower bound objective to learn the oneto-many generalization.Both automatic and human evaluations demonstrate the efficacy of our approach.
Rescue Implicit and Long-tail Cases: Nearest Neighbor Relation Extraction

Relation extraction (RE) has achieved remarkable progress with the help of pre-trained language models.However, existing RE models are usually incapable of handling two situations: implicit expressions and long-tail relation types, caused by language complexity and data sparsity.In this paper, we introduce a simple enhancement of RE using k nearest neighbors (kNN-RE).kNN-RE allows the model to consult training relations at test time through a nearest-neighbor search and provides a simple yet effective means to tackle the two issues above.Additionally, we observe that kNN-RE serves as an effective way to leverage distant supervision (DS) data for RE.Experimental results show that the proposed kNN-RE achieves state-of-the-art performances on a variety of supervised RE datasets, i.e., ACE05, SciERC, and Wiki80, along with outperforming the best model to date on the i2b2 and Wiki80 datasets in the setting of allowing using DS.Our code and models are available at: https://github.com/YukinoWan/kNN-RE.
A Good Neighbor, A Found Treasure: Mining Treasured Neighbors for Knowledge Graph Entity Typing

The task of knowledge graph entity typing (KGET) aims to infer the missing types for entities in knowledge graphs.Some pioneering work has proved that neighbor information is essential for the task.However, existing methods only leverage the one-hop neighbor information of the central entity, ignoring the multi-hop neighbor information that can provide valuable clues for inference.Besides, we also observe that there are co-occurrence relations between types, which is very helpful in alleviating the false-negative problem.In this paper, we propose a novel method called Mining Treasured Neighbors (MiNer) to make use of these two characteristics.Firstly, we devise a Neighbor Information Aggregation module to aggregate the neighbor information.Then, we propose an Entity Type Inference module to mitigate the adverse impact of the irrelevant neighbor information.Finally, a Type Co-occurrence Regularization module is designed to prevent the model from overfitting the false-negative examples caused by missing types.Experimental results on two widely used datasets indicate that our approach significantly outperforms previous state-of-the-art methods. 1
Visual Semantic Parsing: From Images to Abstract Meaning Representation

The success of scene graphs for visual scene understanding has brought attention to the benefits of abstracting a visual input (e.g., image) into a structured representation, where entities (people and objects) are nodes connected by edges specifying their relations.Building these representations, however, requires expensive manual annotation in the form of images paired with their scene graphs or frames.These formalisms remain limited in the nature of entities and relations they can capture.In this paper, we propose to leverage a widely-used meaning representation in the field of natural language processing, the Abstract Meaning Representation (AMR), to address these shortcomings.Compared to scene graphs, which largely emphasize spatial relationships , our visual AMR graphs are more linguistically informed, with a focus on higherlevel semantic concepts extrapolated from visual input.Moreover, they allow us to generate meta-AMR graphs to unify information contained in multiple image descriptions under one representation.Through extensive experimentation and analysis, we demonstrate that we can re-purpose an existing text-to-AMR parser to parse images into AMRs.Our findings point to important future research directions for improved scene understanding.
Unsupervised training data re-weighting for natural language understanding with local distribution approximation

One of the major challenges of training Natural Language Understanding (NLU) production models lies in the discrepancy between the distributions of the offline training data and of the online live data, due to, e.g., biased sampling scheme, cyclic seasonality shifts, annotated training data coming from a variety of different sources, and a changing pool of users. Consequently, the model trained by the offline data is biased. We often observe this problem especially in task-oriented conversational systems, where topics of interest and the characteristics of users using the system change over time. In this paper we propose an unsupervised approach to mitigate the offline training data sampling bias in multiple NLU tasks. We show that a local distribution approximation in the pre-trained embedding space enables the estimation of importance weights for training samples guiding re-sampling for an effective bias mitigation. We illustrate our novel approach using multiple NLU datasets and show improvements obtained without additional annotation, making this a general approach for mitigating effects of sampling bias.
End-to-End Speech to Intent Prediction to improve E-commerce Customer Support Voicebot in Hindi and English

Automation of on-call customer support relies heavily on accurate and efficient speech-to-intent (S2I) systems. Building such systems using multi-component pipelines can pose various challenges because they require large annotated datasets, have higher latency, and have complex deployment. These pipelines are also prone to compounding errors. To overcome these challenges, we discuss an end-to-end (E2E) S2I model for customer support voicebot task in a bilingual setting. We show how we can solve E2E intent classification by leveraging a pre-trained automatic speech recognition (ASR) model with slight modification and fine-tuning on small annotated datasets. Experimental results show that our best E2E model outperforms a conventional pipeline by a relative ~27% on the F1 score.
CamPros at CASE 2022 Task 1: Transformer-based Multilingual Protest News Detection

Socio-political protests often lead to grave consequences when they occur.The early detection of such protests is very important for taking early precautionary measures.However, the main shortcoming of protest event detection is the scarcity of sufficient training data for specific language categories, which makes it difficult to train data-hungry deep learning models effectively.Therefore, cross-lingual and zero-shot learning models are needed to detect events in various low-resource languages.This paper proposes a multi-lingual crossdocument level event detection approach using pre-trained transformer models developed for Shared Task 1 at CASE 2022.The shared task constituted four subtasks for event detection at different granularity levels, i.e., document level to token level, spread over multiple languages (English, Spanish, Portuguese, Turkish, Urdu, and Mandarin).Our system achieves an average F 1 score of 0.73 for document-level event detection tasks.Our approach secured 2 nd position for the Hindi language in subtask 1 with an F 1 score of 0.80.While for Spanish, we secure 4 th position with an F 1 score of 0.69.
Varifocal Question Generation for Fact-checking

Fact-checking requires retrieving evidence related to a claim under investigation.The task can be formulated as question generation based on a claim, followed by question answering.However, recent question generation approaches assume that the answer is known and typically contained in a passage given as input, whereas such passages are what is being sought when verifying a claim.In this paper, we present Varifocal, a method that generates questions based on different focal points within a given claim, i.e. different spans of the claim and its metadata, such as its source and date.Our method outperforms previous work on a fact-checking question generation dataset on a wide range of automatic evaluation metrics.These results are corroborated by our manual evaluation, which indicates that our method generates more relevant and informative questions.We further demonstrate the potential of focal points in generating sets of clarification questions for product descriptions.
Learning Instructions with Unlabeled Data for Zero-Shot Cross-Task Generalization

Training language models to learn from human instructions for zero-shot cross-task generalization has attracted much attention in NLP communities.Recently, instruction tuning (IT), which fine-tunes a pre-trained language model on a massive collection of tasks described via human-craft instructions, has been shown effective in instruction learning for unseen tasks.However, IT relies on a large amount of humanannotated samples, which restricts its generalization.Unlike labeled data, unlabeled data are often massive and cheap to obtain.In this work, we study how IT can be improved with unlabeled data.We first empirically explore the IT performance trends versus the number of labeled data, instructions, and training tasks.We find it critical to enlarge the number of training instructions, and the instructions can be underutilized due to the scarcity of labeled data.Then, we propose Unlabeled Data Augmented Instruction Tuning (UDIT) to take better advantage of the instructions during IT by constructing pseudo-labeled data from unlabeled plain texts.We conduct extensive experiments to show UDIT's effectiveness in various scenarios of tasks and datasets.We also comprehensively analyze the key factors of UDIT to investigate how to better improve IT with unlabeled data.The code is publicly available at https://github.com/thu-coai/UDIT.
Does Joint Training Really Help Cascaded Speech Translation?

Currently, in speech translation, the straightforward approach -cascading a recognition system with a translation system -delivers state-of-theart results.However, fundamental challenges such as error propagation from the automatic speech recognition system still remain.To mitigate these problems, recently, people turn their attention to direct data and propose various joint training methods.In this work, we seek to answer the question of whether joint training really helps cascaded speech translation.We review recent papers on the topic and also investigate a joint training criterion by marginalizing the transcription posterior probabilities.Our findings show that a strong cascaded baseline can diminish any improvements obtained using joint training, and we suggest alternatives to joint training.We hope this work can serve as a refresher of the current speech translation landscape, and motivate research in finding more efficient and creative ways to utilize the direct data for speech translation.
A Joint Learning Framework for Restaurant Survival Prediction and Explanation

The bloom of the Internet and the recent breakthroughs in deep learning techniques open a new door to AI for E-commence, with a trend evolved from using a few financial factors such as liquidity and profitability to using more advanced AI techniques to process complex and multi-modal data.In this paper, we tackle the practical problem of restaurant survival prediction.We argue that traditional methods ignore two essential aspects, which are very helpful for the task: 1) modeling customer reviews and 2) jointly considering status prediction and result explanation.Thus, we propose a novel joint learning framework for explainable restaurant survival prediction based on the multi-modal data of user-restaurant interactions and users' textual reviews.Moreover, we design a graph neural network to capture the high-order interactions and design a co-attention mechanism to capture the most informative and meaningful signal from noisy textual reviews.Our results on two datasets show a significant and consistent improvement over the SOTA techniques (average 6.8% improvement in prediction and 45.3% improvement in explanation).
FALTE: A Toolkit for Fine-grained Annotation for Long Text Evaluation

A growing swath of NLP research is tackling problems related to generating long text, including tasks such as open-ended story generation, summarization, dialogue, and more.However, we currently lack appropriate tools to evaluate these long outputs of generation models: classic automatic metrics such as ROUGE have been shown to perform poorly, and newer learned metrics do not necessarily work well for all tasks and domains of text.Human rating and error analysis remain a crucial component for any evaluation of long text generation.In this paper, we introduce FALTE, a web-based annotation toolkit designed to streamline such evaluations.Our tool allows researchers to collect fine-grained judgments of text quality from crowdworkers using an error taxonomy specific to the downstream task.Using the task interface, annotators can select and assign error labels to text span selections in an incremental paragraph-level annotation workflow.The latter functionality is designed to simplify the document-level task into smaller units and reduce cognitive load on the annotators.Our tool has previously been used to run a large-scale annotation study that evaluates the coherence of long generated summaries, demonstrating its utility.
MUSIED: A Benchmark for Event Detection from Multi-Source Heterogeneous Informal Texts

Event detection (ED) identifies and classifies event triggers from unstructured texts, serving as a fundamental task for information extraction.Despite the remarkable progress achieved in the past several years, most research efforts focus on detecting events from formal texts (e.g., news articles, Wikipedia documents, financial announcements).Moreover, the texts in each dataset are either from a single source or multiple yet relatively homogeneous sources.With massive amounts of user-generated text accumulating on the Web and inside enterprises, identifying meaningful events in these informal texts, usually from multiple heterogeneous sources, has become a problem of significant practical value.As a pioneering exploration that expands event detection to the scenarios involving informal and heterogeneous texts, we propose a new large-scale Chinese event detection dataset based on user reviews, text conversations, and phone conversations in a leading e-commerce platform for food service.We carefully investigate the proposed dataset's textual informality and multi-source heterogeneity characteristics by inspecting data samples quantitatively and qualitatively.Extensive experiments with state-of-the-art event detection methods verify the unique challenges posed by these characteristics, indicating that multisource informal event detection remains an open problem and requires further efforts.Our benchmark and code are released at https: //github.com/myeclipse/MUSIED.