Reviewerly/exHarmony huggingface dataset:
Interactive Span Recommendation for Biomedical Text

Motivated by the scarcity of high-quality labeled biomedical text, as well as the success of data programming, we introduce KRISS-Search.By leveraging the Unified Medical Language Systems (UMLS) ontology, KRISS-Search addresses an interactive few-shot span recommendation task that we propose.We first introduce unsupervised KRISS-Search and show that our method outperforms existing methods in identifying spans that are semantically similar to a given span of interest, with > 50% AUPRC improvement relative to PubMed-BERT.We then introduce supervised KRISS-Search, which leverages human interaction to improve the notion of similarity used by unsupervised KRISS-Search.Through simulated human feedback, we demonstrate an enhanced F1 score of 0.68 in classifying spans as semantically similar or different in the lowlabel setting, outperforming PubMedBERT by 2 F1 points.Finally, supervised KRISS-Search demonstrates competitive or superior performance compared to PubMedBERT in few-shot biomedical named entity recognition (NER) across five benchmark datasets, with an average improvement of 5.6 F1 points.We envision KRISS-Search increasing the efficiency of programmatic data labeling and also providing broader utility as an interactive biomedical search engine.
Parameter-efficient Weight Ensembling Facilitates Task-level Knowledge Transfer

Recent studies show that large-scale pretrained language models could be efficaciously adapted to particular tasks in a parameterefficient manner.The trained lightweight set of parameters, such as adapters, can be easily stored and shared as a capability equipped with the corresponding models.Owning many lightweight parameters, we focus on transferring them between tasks to acquire an improvement in performance of new tasks, the key point of which is to obtain the similarity between tasks.In this paper, we explore 5 parameter-efficient weight ensembling methods to achieve such transferability and verify the effectiveness of them.These methods extract the information of datasets and trained lightweight parameters from different perspectives to obtain the similarity between tasks, and weight the existing lightweight parameters according to the comparability to acquire a suitable module for the initialization of new tasks.We apply them to three parameter-efficient tuning methods and test them on a wide set of downstream tasks.Experimental results show that our methods show an improvement of 5%~8% over baselines and could largely facilitate task-level knowledge transfer. Web Search Environment ContentFind key words….. 1/9 为什么⽕⻋轨道总是停在岩 Why do train tracks always stop on 为什么普通铁路铁轨上都会铺⽯⼦?普通铁路采⽤的是有砟轨道，⽽没有" " 道砟 ")的⾼铁采⽤的是⽆砟轨道。 统有砟轨道的优点；3.传统有砟轨道 同样是铁轨，为什么⽕⻋轨道下⾯会 ⼦,⾼铁轨道却不铺?Why ordinary railroad tracks are paved The same is the railway track, why are under the train track, but not the high-s 道砟的好处之⼀是能够降低列⻋经过时 和热量… Web Search Environmen Content Find key words….. 为什么⽕⻋轨道总是停在 Why do train tracks always stop o 为什么普通铁路铁轨上都会铺⽯⼦ 普通铁路采⽤的是有砟轨道，⽽没有 " 道砟 ")的⾼铁采⽤的是⽆砟轨道。 统有砟轨道的优点；3.传统有砟轨道 同样是铁轨，为什么⽕⻋轨道下⾯ ⼦,⾼铁轨道却不铺?Why ordinary railroad tracks are pave
Content Moderation for Evolving Policies using Binary Question Answering

Content moderation on social media is governed by policies that are intricate and frequently updated with evolving world events.However, automated content moderation systems often restrict easy adaptation to policy changes and are expected to learn policy intricacies from limited amounts of labeled data, which make effective policy compliance challenging.We propose to model content moderation as a binary question answering problem where the questions validate the loosely coupled themes constituting a policy.A decision logic is applied on top to aggregate the themespecific validations.This way the questions pass theme information to a transformer network as explicit policy prompts, that in turn enables explainability.This setting further allows for faster adaptation to policy updates by leveraging zero-shot capabilities of pre-trained transformers.We showcase improved recall for our proposed method at 95% precision on two proprietary datasets of social media posts and comments respectively annotated under curated Hate Speech and Commercial Spam policies.
Probing for Hyperbole in Pre-Trained Language Models

Hyperbole is a common figure of speech, which is under-explored in NLP research.In this study, we conduct edge and minimal description length (MDL) probing experiments for three pre-trained language models (PLMs) in an attempt to explore the extent to which hyperbolic information is encoded in these models.We use both word-in-context and sentencelevel representations as model inputs as a basis for comparison.We also annotate 63 hyperbole sentences from the HYPO dataset according to an operational taxonomy to conduct an error analysis to explore the encoding of different hyperbole categories.Our results show that hyperbole is to a limited extent encoded in PLMs, and mostly in the final layers.They also indicate that hyperbolic information may be better encoded by the sentence-level representations, which, due to the pragmatic nature of hyperbole, may therefore provide a more accurate and informative representation in PLMs.Finally, the inter-annotator agreement for our annotations, a Cohen's Kappa of 0.339, suggest that the taxonomy categories may not be intuitive and need revision or simplification.
An Effective Deployment of Contrastive Learning in Multi-label Text Classification

The effectiveness of contrastive learning technology in natural language processing tasks is yet to be explored and analyzed.How to construct positive and negative samples correctly and reasonably is the core challenge of contrastive learning.It is even harder to discover contrastive objects in multi-label text classification tasks.There are very few contrastive losses proposed previously.In this paper, we investigate the problem from a different angle by proposing five novel contrastive losses for multi-label text classification tasks.These are Strict Contrastive Loss (SCL), Intra-label Contrastive Loss (ICL), Jaccard Similarity Contrastive Loss (JSCL), Jaccard Similarity Probability Contrastive Loss (JSPCL), and Stepwise Label Contrastive Loss (SLCL).We explore the effectiveness of contrastive learning for multilabel text classification tasks by the employment of these novel losses and provide a set of baseline models for deploying contrastive learning techniques on specific tasks.We further perform an interpretable analysis of our approach to show how different components of contrastive learning losses play their roles.The experimental results show that our proposed contrastive losses can bring improvement to multi-label text classification tasks.Our work also explores how contrastive learning should be adapted for multi-label text classification tasks.
Commonsense Knowledge Transfer for Pre-trained Language Models

Despite serving as the foundation models for a wide range of NLP benchmarks, pre-trained language models have shown limited capabilities of acquiring implicit commonsense knowledge from self-supervision alone, compared to learning linguistic and factual knowledge that appear more explicitly in the surface patterns in text.In this work, we introduce commonsense knowledge transfer, a framework to transfer the commonsense knowledge stored in a neural commonsense knowledge model to a general-purpose pre-trained language model.It first exploits general texts to form queries for extracting commonsense knowledge from the neural commonsense knowledge model and then refines the language model with two selfsupervised objectives: commonsense mask infilling and commonsense relation prediction, which align human language with the underlying commonsense knowledge.Empirical results show that our approach consistently improves the model's performance on downstream tasks that require commonsense reasoning.Moreover, we find that the improvement is more significant in the few-shot setting.This suggests that our approach helps language models better transfer to downstream tasks without extensive supervision by injecting commonsense knowledge into their parameters.
My side, your side and the evidence: Discovering aligned actor groups and the narratives they weave

News reports about emerging issues often include several conflicting story lines.Individual stories can be conceptualized as samples from an underlying mixture of competing narratives.The automated identification of these distinct narratives from unstructured text is a fundamental yet difficult task in Computational Linguistics since narratives are often intertwined and only implicitly conveyed in text.In this paper, we consider a more feasible proxy task: Identify the distinct sets of aligned story actors responsible for sustaining the issue-specific narratives.Discovering aligned actors, and the groups these alignments create, brings us closer to estimating the narrative that each group represents.With the help of Large Language Models (LLM), we address this task by: (i) Introducing a corpus of text segments rich in narrative content associated with six different current issues; (ii) Introducing a novel two-step graph-based framework that (a) identifies alignments between actors (INCANT) and (b) extracts aligned actor groups using the network structure (TAMPA).Amazon Mechanical Turk evaluations demonstrate the effectiveness of our framework.Across domains, alignment relationships from INCANT are accurate (macro F 1 ≥ 0.75) and actor groups from TAMPA are preferred over 2 non-trivial baseline models (ACC ≥ 0.75).
Prototype-Guided Pseudo Labeling for Semi-Supervised Text Classification

The semi-supervised text classification (SSTC) task aims at training text classification models with a few labeled data and massive unlabeled data.Recent works achieve this task by pseudo-labeling methods that assign pseudolabels to unlabeled data as additional supervision.However, these models may suffer from incorrect pseudo-labels caused by underfitting of decision boundaries and generating biased pseudo-labels on imbalanced data.We propose a prototype-guided semi-supervised model to address the above problems, which integrates a prototype-anchored contrasting strategy and a prototype-guided pseudo-labeling strategy.Particularly, the prototype-anchored constrasting constructs prototypes to cluster text representations with the same class, forcing them to be high-density distributed, thus alleviating the underfitting of decision boundaries.And the prototype-guided pseudo-labeling selects reliable pseudo-labeled data around prototypes based on data distribution, thus alleviating the bias from imbalanced data.Empirical results on 4 commonly-used datasets demonstrate that our model is effective and outperforms state-ofthe-art methods.
Do I have the Knowledge to Answer? Investigating Answerability of Knowledge Base Questions

When answering natural language questions over knowledge bases, missing facts, incomplete schema and limited scope naturally lead to many questions being unanswerable.While answerability has been explored in other QA settings, it has not been studied for QA over knowledge bases (KBQA).We create GrailQAbility, a new benchmark KBQA dataset with unanswerability, by first identifying various forms of KB incompleteness that make questions unanswerable, and then systematically adapting GrailQA (a popular KBQA dataset with only answerable questions).Experimenting with three state-of-the-art KBQA models, we find that all three models suffer a drop in performance even after suitable adaptation for unanswerable questions.In addition, these often detect unanswerability for wrong reasons and find specific forms of unanswerability particularly difficult to handle.This underscores the need for further research in making KBQA systems robust to unanswerability.
Don’t Lose Yourself! Empathetic Response Generation via Explicit Self-Other Awareness

As a critical step to achieve human-like chatbots, empathetic response generation has attained increasing interests.Previous attempts are incomplete and not sufficient enough to elicit empathy because they only stay on the initial stage of empathy to automatically sense and simulate the feelings and thoughts of others via other-awareness.However, they ignore to include self-awareness to consider the own views of the self in their responses, which is a crucial process to achieve the empathy.To this end, we propose to generate Empathetic response with explicit Self-Other Awareness (EmpSOA).Specifically, three stages, selfother differentiation, self-other modulation and self-other generation, are devised to clearly maintain, regulate and inject the self-other aware information into the process of empathetic response generation.Both automatic and human evaluations on the benchmark dataset demonstrate the superiority of Emp-SOA to generate more empathetic responses.Our source code is available at https:// github.com/circle-hit/EmpSOA.
CLIPText: A New Paradigm for Zero-shot Text Classification

While CLIP models are useful for zero-shot vision-and-language (VL) tasks or computer vision tasks, little attention has been paid to the application of CLIP for language tasks.Intuitively, CLIP model have a rich representation pre-trained with natural language supervision, in which we argue that it is useful for language tasks.Hence, this work bridge this gap by investigating a CLIP model for zero-shot text classification.Specifically, we introduce CLIP-TEXT, a novel paradigm for zero-shot text classification, which reformulates zero-shot text classification into a text-image matching problem that CLIP can be applied to.In addition, we further incorporate prompt into CLIP-TEXT (PROMPT-CLIPTEXT) to better derive knowledge from CLIP.Experimental results on seven publicly available zero-shot text classification datasets show that both CLIPTEXT and PROMPT-CLIPTEXT attain promising performance.Besides, extensive analysis further verifies that knowledge from CLIP can benefit zero-shot text classification task.We hope this work can attract more breakthroughs on applying VL pre-trained models for language tasks.
How effective is machine translation on low-resource code-switching? A case study comparing human and automatic metrics

This paper presents an investigation into the differences between processing monolingual input and code-switching (CSW) input in the context of machine translation (MT).Specifically, we compare the performance of three MT systems (Google, mBART-50 and M2M-100 big ) in terms of their ability to translate monolingual Vietnamese, a low-resource language, and Vietnamese-English CSW respectively.To our knowledge, this is the first study to systematically analyse what might happen when multilingual MT systems are exposed to CSW data using both automatic and human metrics.We find that state-of-the-art neural translation systems not only achieve higher scores on automatic metrics when processing CSW input (compared to monolingual input), but also produce translations that are consistently rated as more semantically faithful by humans.We further suggest that automatic evaluation alone is insufficient for evaluating the translation of CSW input.Our findings establish a new benchmark that offers insights into the relationship between MT and CSW.
Prompt-based Zero-shot Text Classification with Conceptual Knowledge

In recent years, pre-trained language models have garnered significant attention due to their effectiveness, which stems from the rich knowledge acquired during pre-training.To mitigate the inconsistency issues between pre-training tasks and downstream tasks and to facilitate the resolution of language-related issues, promptbased approaches have been introduced, which are particularly useful in low-resource scenarios.However, existing approaches mostly rely on verbalizers to translate the predicted vocabulary to task-specific labels.The major limitations of this approach are the ignorance of potentially relevant domain-specific words and being biased by the pre-training data.To address these limitations, we propose a framework that incorporates conceptual knowledge for text classification in the extreme zero-shot setting.The framework includes prompt-based keyword extraction, weight assignment to each prompt keyword, and final representation estimation in the knowledge graph embedding space.We evaluated the method on four widelyused datasets for sentiment analysis and topic detection, demonstrating that it consistently outperforms recently-developed prompt-based approaches in the same experimental settings.
Improving Generalization in Language Model-based Text-to-SQL Semantic Parsing: Two Simple Semantic Boundary-based Techniques

Compositional and domain generalization present significant challenges in semantic parsing, even for state-of-the-art semantic parsers based on pre-trained language models (LMs).In this study, we empirically investigate improving an LM's generalization in semantic parsing with two simple techniques: at the token level, we introduce a token preprocessing method to preserve the semantic boundaries of tokens produced by LM tokenizers; at the sequence level, we propose to use special tokens to mark the boundaries of components aligned between input and output.Our experimental results on two text-to-SQL semantic parsing datasets show that our token preprocessing, although simple, can substantially improve the LM performance on both types of generalization, and our component boundary marking method is particularly helpful for compositional generalization. 1
Echoes from Alexandria: A Large Resource for Multilingual Book Summarization

In recent years, research in text summarization has mainly focused on the news domain, where texts are typically short and have strong layout features.The task of full-book summarization presents additional challenges which are hard to tackle with current resources, due to their limited size and availability in English only.To overcome these limitations, we present "Echoes from Alexandria", or in shortened form, "Echoes", a large resource for multilingual book summarization.Echoes features three novel datasets: i) Echo-Wiki, for multilingual book summarization, ii) Echo-XSum, for extremely-compressive multilingual book summarization, and iii) Echo-FairySum, for extractive book summarization.To the best of our knowledge, Echoes -with its thousands of books and summaries -is the largest resource, and the first to be multilingual, featuring 5 languages and 25 language pairs.In addition to Echoes, we also introduce a new extractive-then-abstractive baseline, and, supported by our experimental results and manual analysis of the summaries generated, we argue that this baseline is more suitable for book summarization than purely-abstractive approaches.We release our resource and software at https://github.com/Babelscape/echoes-from-alexandria in the hope of fostering innovative research in multilingual book summarization.
Evaluating Classroom Potential for Card-it: Digital Flashcards for Studying and Learning Italian Morphology

This paper presents Card-it, a web-based application for learning Italian verb conjugation.Card-it integrates a large-scale finite-state morphological (FSM) analyzer and a flashcard application as a user-friendly way for learners to utilize the analyzer.While Card-it can be used by individual learners, to support classroom adoption, we implemented simple classroom management functionalities such as sharing flashcards to a class and tracking students' progression.We evaluated Card-it with teachers of Italian.Card-it was reported as engaging and supportive, especially by featuring two different quiz types combined with a verb form lookup feature.Teachers were optimistic about the potential of Card-it as a classroom supplementary tool for learners of Italian as L2.Future work includes sample sentences and a complete learners evaluation.
Layerwise universal adversarial attack on NLP models

In this work, we examine the vulnerability of language models to universal adversarial triggers (UATs).We propose a new whitebox approach to the construction of layerwise UATs (LUATs), which searches the triggers by perturbing hidden layers of a network.On the example of three transformer models and three datasets from the GLUE benchmark, we demonstrate that our method provides better transferability in a model-to-model setting with an average gain of 9.3% in the fooling rate over the baseline.Moreover, we investigate triggers transferability in the task-to-task setting.Using small subsets from the datasets similar to the target tasks for choosing a perturbed layer, we show that LUATs are more efficient than vanilla UATs by 7.1% in the fooling rate.
KNOW How to Make Up Your Mind! Adversarially Detecting and Alleviating Inconsistencies in Natural Language Explanations

While recent works have been considerably improving the quality of the natural language explanations (NLEs) generated by a model to justify its predictions, there is very limited research in detecting and alleviating inconsistencies among generated NLEs.In this work, we leverage external knowledge bases to significantly improve on an existing adversarial attack for detecting inconsistent NLEs.We apply our attack to high-performing NLE models and show that models with higher NLE quality do not necessarily generate fewer inconsistencies.Moreover, we propose an offthe-shelf mitigation method to alleviate inconsistencies by grounding the model into external background knowledge.Our method decreases the inconsistencies of previous highperforming NLE models as detected by our attack.
L3I++ at SemEval-2023 Task 2: Prompting for Multilingual Complex Named Entity Recognition

This paper summarizes the participation of the L3i laboratory of the University of La Rochelle in the SemEval-2023 Task 2, Multilingual Complex Named Entity Recognition (MultiCoNER II).Similar to MultiCoNER I, the task seeks to develop methods to detect semantic ambiguous and complex entities in short and low-context settings.However, MultiCoNER II adds a fine-grained entity taxonomy with over 30 entity types and corrupted data on the test partitions.We approach these complications following prompt-based learning as (1) a ranking problem using a seq2seq framework, and (2) an extractive question-answering task.Our findings show that even if prompting techniques have a similar recall to fine-tuned hierarchical language model-based encoder methods, precision tends to be more affected.
Prodicus at SemEval-2023 Task 4: Enhancing Human Value Detection with Data Augmentation and Fine-Tuned Language Models

This paper introduces a data augmentation technique for the task of detecting human values.Our approach involves generating additional examples using metadata that describes the labels in the datasets.We evaluated the effectiveness of our method by fine-tuning BERT and RoBERTa models on our augmented dataset and comparing their F 1 -scores to those of the non-augmented dataset.We obtained competitive results on both the Main test set and the Nahj al-Balagha test set, ranking 14th and 7th respectively among the participants.We also demonstrate that by incorporating our augmentation technique, the classification performance of BERT and RoBERTa is improved, resulting in an increase of up to 10.1% in their F 1 -score.
Towards Speech Dialogue Translation Mediating Speakers of Different Languages

We present a new task, speech dialogue translation mediating speakers of different languages.We construct the SpeechBSD dataset for the task and conduct baseline experiments.Furthermore, we consider context to be an important aspect that needs to be addressed in this task and propose two ways of utilizing context, namely monolingual context and bilingual context.We conduct cascaded speech translation experiments using Whisper and mBART, and show that bilingual context performs better in our settings.彼は良い考えだと言ってました He said it's a good idea MT 少し甘いと思います I think it's a bit sweet MT What do you think about it?