Reviewerly/exHarmony huggingface dataset:
Exploring and Verbalizing Academic Ideas by Concept Co-occurrence

Researchers usually come up with new ideas only after thoroughly comprehending vast quantities of literature.The difficulty of this procedure is exacerbated by the fact that the number of academic publications is growing exponentially.In this study, we devise a framework based on concept co-occurrence for academic idea inspiration, which has been integrated into a research assistant system.From our perspective, the fusion of two concepts that co-occur in an academic paper can be regarded as an important way of the emergence of a new idea.We construct evolving concept graphs according to the co-occurrence relationship of concepts from 20 disciplines or topics.Then we design a temporal link prediction method based on masked language model to explore potential connections between different concepts.To verbalize the newly discovered connections, we also utilize the pretrained language model to generate a description of an idea based on a new data structure called co-occurrence citation quintuple.We evaluate our proposed system using both automatic metrics and human assessment.The results demonstrate that our system has broad prospects and can assist researchers in expediting the process of discovering new ideas. 1
A New Aligned Simple German Corpus

Leichte Sprache", the German counterpart to Simple English, is a regulated language aiming to facilitate complex written language that would otherwise stay inaccessible to different groups of people.We present a new sentencealigned monolingual corpus for Simple German -German.It contains multiple documentaligned sources which we have aligned using automatic sentence-alignment methods.We evaluate our alignments based on a manually labelled subset of aligned documents.The quality of our sentence alignments, as measured by the F1-score, surpasses previous work.
DLAMA: A Framework for Curating Culturally Diverse Facts for Probing the Knowledge of Pretrained Language Models

A few benchmarking datasets have been released to evaluate the factual knowledge of pretrained language models.These benchmarks (e.g., LAMA, and ParaRel) are mainly developed in English and later are translated to form new multilingual versions (e.g., mLAMA, and mParaRel).Results on these multilingual benchmarks suggest that using English prompts to recall the facts from multilingual models usually yields significantly better and more consistent performance than using non-English prompts.Our analysis shows that mLAMA is biased toward facts from Western countries, which might affect the fairness of probing models.We propose a new framework for curating factual triples from Wikidata that are culturally diverse.A new benchmark DLAMA-v1 is built of factual triples from three pairs of contrasting cultures having a total of 78,259 triples from 20 relation predicates.The three pairs comprise facts representing the (Arab and Western), (Asian and Western), and (South American and Western) countries respectively.Having a more balanced benchmark (DLAMA-v1) supports that mBERT performs better on Western facts than non-Western ones, while monolingual Arabic, English, and Korean models tend to perform better on their culturally proximate facts.Moreover, both monolingual and multilingual models tend to make a prediction that is culturally or geographically relevant to the correct label, even if the prediction is wrong.
Hindi Chatbot for Supporting Maternal and Child Health Related Queries in Rural India

In developing countries like India, doctors and healthcare professionals working in public health spend significant time answering health queries that are fact-based and repetitive.Therefore, we propose an automated way to answer maternal and child health-related queries.A database of Frequently Asked Questions (FAQs) and their corresponding answers generated by experts is curated from rural health workers and young mothers.We develop a Hindi chatbot that identifies k relevant Question and Answer (QnA) pairs from the database in response to a healthcare query (q) written in Devnagri script or Hindi-English (Hinglish) code-mixed script.The curated database covers 80% of all the queries that a user of our study is likely to ask.We experimented with (i) rule-based methods, (ii) sentence embeddings, and (iii) a paraphrasing classifier, to calculate the q-Q similarity.We observed that paraphrasing classifier gives the best result when trained first on an open-domain text and then on the healthcare domain.Our chatbot uses an ensemble of all three approaches.We observed that if a given q can be answered using the database, then our chatbot can provide at least one relevant QnA pair among its top three suggestions for up to 70% of the queries.
On the Off-Target Problem of Zero-Shot Multilingual Neural Machine Translation

While multilingual neural machine translation has achieved great success, it suffers from the off-target issue, where the translation is in the wrong language.This problem is more pronounced on zero-shot translation tasks.In this work, we find that failing in encoding discriminative target language signal will lead to offtarget and a closer lexical distance (i.e., KLdivergence) between two languages' vocabularies is related with a higher off-target rate.We also find that solely isolating the vocab of different languages in the decoder can alleviate the problem.Motivated by the findings, we propose Language Aware Vocabulary Sharing (LAVS), a simple and effective algorithm to construct the multilingual vocabulary, that greatly alleviates the off-target problem of the translation model by increasing the KLdivergence between languages.We conduct experiments on a multilingual machine translation benchmark in 11 languages.Experiments show that the off-target rate for 90 translation tasks is reduced from 29% to 8%, while the overall BLEU score is improved by an average of 1.9 points without extra training cost or sacrificing the supervised directions' performance.We release the code at https://github.com/PKUnlpicler/Off-Target-MNMTfor reproduction.
DSHacker at SemEval-2023 Task 3: Genres and Persuasion Techniques Detection with Multilingual Data Augmentation through Machine Translation and Text Generation

In our article, we present the systems developed for SemEval-2023 Task 3, which aimed to evaluate the ability of Natural Language Processing (NLP) systems to detect genres and persuasion techniques in multiple languages.We experimented with several data augmentation techniques, including machine translation (MT) and text generation.For genre detection, synthetic texts for each class were created using the Ope-nAI GPT-3 Davinci language model.In contrast, to detect persuasion techniques, we relied on augmenting the dataset through text translation using the DeepL translator.Fine-tuning the models using augmented data resulted in a top-ten ranking across all languages, indicating the effectiveness of the approach.The models for genre detection demonstrated excellent performance, securing the first, second, and third positions in Spanish, German, and Italian, respectively.Moreover, one of the models for persuasion techniques' detection secured the third position in Polish.Our contribution constitutes the system architecture that utilizes DeepL and GPT-3 for data augmentation for the purpose of detecting both genre and persuasion techniques.
Learning In-context Learning for Named Entity Recognition

Named entity recognition in real-world applications suffers from the diversity of entity types, the emergence of new entity types, and the lack of high-quality annotations.To address the above problems, this paper proposes an in-context learning-based NER approach, which can effectively inject in-context NER ability into PLMs and recognize entities of novel types on-the-fly using only a few demonstrative instances.Specifically, we model PLMs as a meta-function λ instruction, demonstrations, text .M 1 , and a new entity extractor can be implicitly constructed by applying new instruction and demonstrations to PLMs, i.e., (λ.M)(instruction, demonstrations) → F where F will be a new entity extractor, i.e., F: text → entities.To inject the above in-context NER ability into PLMs, we propose a meta-function pre-training algorithm, which pre-trains PLMs by comparing the (instruction, demonstration)-initialized extractor with a surrogate golden extractor.Experimental results on 4 few-shot NER datasets show that our method can effectively inject in-context NER ability into PLMs and significantly outperforms the PLMs+fine-tuning counterparts.* This work was partially done when Jiawei Chen interned at Baidu.† Corresponding authors. 1 This paper represents functions using lambdacalculus (Barendregt, 1992), and each function is represented as λx,y,z.M , where x, y, z are variables and M is function definition/abstraction.The function can apply to arguments such as (λ x,y,z .M )(x = A, y = B, z = C) (fully applied) or (λ x,y,z .M )(x = A, y = B) (partially applied).
Enhancing Continual Relation Extraction via Classifier Decomposition

Continual relation extraction (CRE) models aim at handling emerging new relations while avoiding catastrophically forgetting old ones in the streaming data.Though improvements have been shown by previous CRE studies, most of them only adopt a vanilla strategy when models first learn representations of new relations.In this work, we point out that there exist two typical biases after training of this vanilla strategy: classifier bias and representation bias, which causes the previous knowledge that the model learned to be shaded.To alleviate those biases, we propose a simple yet effective classifier decomposition framework that splits the last FFN layer into separated previous and current classifiers, so as to maintain previous knowledge and encourage the model to learn more robust representations at this training stage.Experimental results on two standard benchmarks show that our proposed framework consistently outperforms the state-of-the-art CRE models, which indicates that the importance of the first training stage to CRE models may be underestimated.Our code is available at https://github.com/hemingkx/CDec.
Multimodal Neural Machine Translation Using Synthetic Images Transformed by Latent Diffusion Model

This study proposes a new multimodal neural machine translation (MNMT) model using synthetic images transformed by a latent diffusion model.MNMT translates a source language sentence based on its related image, but the image usually contains noisy information that are not relevant to the source language sentence.Our proposed method first generates a synthetic image corresponding to the content of the source language sentence by using a latent diffusion model and then performs translation based on the synthetic image.The experiments on the English-German translation tasks using the Multi30k dataset demonstrate the effectiveness of the proposed method.
Learning Multi-Step Reasoning by Solving Arithmetic Tasks

Mathematical reasoning is regarded as a necessary ability for Language Models (LMs).Recent works demonstrate large LMs' impressive performance in solving math problems.The success is attributed to their Chain-of-Thought (CoT) reasoning abilities, i.e., the ability to decompose complex questions into step-by-step reasoning chains, but such ability seems only to emerge from models with abundant parameters.This work investigates how to incorporate relatively small LMs with the capabilities of multi-step reasoning.We propose to inject such abilities by continually pre-training LMs on a synthetic dataset MSAT which is composed of Multi-step Arithmetic Tasks.Our experiments on four math word problem datasets show the effectiveness of the proposed method in enhancing LMs' math reasoning abilities. 1
Focusing, Bridging and Prompting for Few-shot Nested Named Entity Recognition

Few-shot named entity recognition (NER), identifying named entities with a small number of labeled data, has attracted much attention.Frequently, entities are nested within each other.However, most of the existing work on few-shot NER addresses flat entities instead of nested entities.To tackle nested NER in a fewshot setting, it is crucial to utilize the limited labeled data to mine unique features of nested entities, such as the relationship between inner and outer entities and contextual position information.Therefore, in this work, we propose a novel method based on focusing, bridging and prompting for few-shot nested NER without using source domain data.Both focusing and bridging components provide accurate candidate spans for the prompting component.The prompting component leverages the unique features of nested entities to classify spans based on soft prompts and contrastive learning.Experimental results show that the proposed approach achieves state-of-the-art performance consistently on the four benchmark datasets (ACE2004, ACE2005, GENIA and KBP2017) and outperforms several competing baseline models on F1-score by 9.33% on ACE2004, 6.17% on ACE2005, 9.40% on GENIA and 5.12% on KBP2017 on the 5-shot setting.
nancy-hicks-gribble at SemEval-2023 Task 5: Classifying and generating clickbait spoilers with RoBERTa

Clickbait spoiling and spoiler type classification in the setting of the SemEval2023 shared task five was used to explore transformer based text classification in comparison to conventional, shallow learned classifying models.Additionally, an initial model for spoiler creation was explored.The task was to classify or create spoilers for clickbait social media posts.The classification task was addressed by comparing different classifiers trained on hand crafted features to pre-trained and fine-tuned RoBERTa transformer models.The spoiler generation task was formulated as a question answering task, using the clickbait posts as questions and the articles as foundation to retrieve the answer from.The results show that even of the shelve transformer models outperform shallow learned models in the classification task.The spoiler generation task is more complex and needs an advanced system.
Adam-Smith at SemEval-2023 Task 4: Discovering Human Values in Arguments with Ensembles of Transformer-based Models

This paper presents the best-performing approach alias "Adam Smith" for the SemEval-2023 Task 4: "Identification of Human Values behind Arguments".The goal of the task was to create systems that automatically identify the values within textual arguments.We train transformer-based models until they reach their loss minimum or f1-score maximum.Ensembling the models by selecting one global decision threshold that maximizes the f1-score leads to the best-performing system in the competition.Ensembling based on stacking with logistic regressions shows the best performance on an additional dataset provided to evaluate the robustness ("Nahj al-Balagha").Apart from outlining the submitted system, we demonstrate that the use of the large ensemble model is not necessary and that the system size can be significantly reduced.
Transferring General Multimodal Pretrained Models to Text Recognition

This paper proposes a new method, OFA-OCR, to transfer multimodal pretrained models to text recognition.Specifically, we recast text recognition as image captioning and directly transfer a unified vision-language pretrained model to the end task.Without pretraining on large-scale annotated or synthetic text recognition data, OFA-OCR outperforms the baselines and achieves state-of-the-art performance in the Chinese text recognition benchmark.Additionally, we construct an OCR pipeline with OFA-OCR, and we demonstrate that it can achieve competitive performance with the product-level API.The code 1 and demo 2 are publicly available.
What In-Context Learning “Learns” In-Context: Disentangling Task Recognition and Task Learning

Large language models (LLMs) exploit incontext learning (ICL) to solve tasks with only a few demonstrations, but its mechanisms are not yet well-understood.Some works suggest that LLMs only recall already learned concepts from pre-training, while others hint that ICL performs implicit learning over demonstrations.We characterize two ways through which ICL leverages demonstrations.Task recognition (TR) captures the extent to which LLMs can recognize a task through demonstrations -even without ground-truth labels -and apply their pre-trained priors, whereas task learning (TL) is the ability to capture new input-label mappings unseen in pre-training.Using a wide range of classification datasets and three LLM families (GPT-3, LLaMA and OPT), we design controlled experiments to disentangle the roles of TR and TL in ICL.We show that (1) models can achieve non-trivial performance with only TR, and TR does not further improve with larger models or more demonstrations; (2) LLMs acquire TL as the model scales, and TL's performance consistently improves with more demonstrations in context.Our findings unravel two different forces behind ICL and we advocate for discriminating them in future ICL research due to their distinct nature. 1
Few-shot Event Detection: An Empirical Study and a Unified View

Few-shot event detection (ED) has been widely studied, while this brings noticeable discrepancies, e.g., various motivations, tasks, and experimental settings, that hinder the understanding of models for future progress.This paper presents a thorough empirical study, a unified view of ED models, and a better unified baseline.For fair evaluation, we compare 12 representative methods on three datasets, which are roughly grouped into prompt-based and prototype-based models for detailed analysis.Experiments consistently demonstrate that prompt-based methods, including Chat-GPT, still significantly trail prototype-based methods in terms of overall performance.To investigate their superior performance, we break down their design elements along several dimensions and build a unified framework on prototype-based methods.Under such unified view, each prototype-method can be viewed a combination of different modules from these design elements.We further combine all advantageous modules and propose a simple yet effective baseline, which outperforms existing methods by a large margin (e.g., 2.7% F 1 gains under low-resource setting). 1 MethodTask setting Experimental setting LR EL CT TT
COBRA Frames: Contextual Reasoning about Effects and Harms of Offensive Statements

Warning: This paper contains content that may be offensive or upsetting.Understanding the harms and offensiveness of statements requires reasoning about the social and situational context in which statements are made.For example, the utterance "your English is very good" may implicitly signal an insult when uttered by a white man to a nonwhite colleague, but uttered by an ESL teacher to their student would be interpreted as a genuine compliment.Such contextual factors have been largely ignored by previous approaches to toxic language detection.We introduce COBRA frames, the first context-aware formalism for explaining the intents, reactions, and harms of offensive or biased statements grounded in their social and situational context.We create COBRACORPUS, a dataset of 33k potentially offensive statements paired with machine-generated contexts and free-text explanations of offensiveness, implied biases, speaker intents, and listener reactions.To study the contextual dynamics of offensiveness, we train models to generate COBRA explanations, with and without access to the context.We find that explanations by context-agnostic models are significantly worse than by contextaware ones, especially in situations where the context inverts the statement's offensiveness (29% accuracy drop).Our work highlights the importance and feasibility of contextualized NLP by modeling social factors. Cobra FrameHow does offensiveness change with a different context?
ClarifyDelphi: Reinforced Clarification Questions with Defeasibility Rewards for Social and Moral Situations

Context is everything, even in commonsense moral reasoning.Changing contexts can flip the moral judgment of an action; Lying to a friend is wrong in general, but may be morally acceptable if it is intended to protect their life.We present CLARIFYDELPHI, an interactive system that learns to ask clarification questions (e.g., "why did you lie to your friend?") in order to elicit additional salient contexts of a social or moral situation.We posit that questions whose potential answers lead to diverging moral judgments are the most informative.Thus, we propose a reinforcement learning framework with a defeasibility reward that aims to maximize the divergence between moral judgments of hypothetical answers to a question.Human evaluation demonstrates that our system generates more relevant, informative and defeasible questions compared to competitive baselines.Our work is ultimately inspired by studies in cognitive science that have investigated the flexibility in moral cognition (i.e., the diverse contexts in which moral rules can be bent), and we hope that research in this direction can assist both cognitive and computational investigations of moral judgments.
Logical Transformers: Infusing Logical Structures into Pre-Trained Language Models

Natural language contains rich logical structures and logical information, and correctly detecting and accurately understanding these logical structures and information underlying natural language texts is very crucial for NLP models' performance on many important NLU and NLG tasks.Existing pre-trained language models based on the transformer architecture mostly adopt a classical design for constructing their input embeddings that ignores the logical structures underlying natural language texts, thus limiting their ability to better capture and encode key logical information in the input sequences.To overcome such limitations, in this paper we first propose a novel approach to construct logic-aware input embeddings for transformer language models through a combination of logic detection, logic mapping and hierarchical logical projections, and then develop a corresponding new modeling paradigm that can upgrade existing transformer language models into logical transformers to boost their performance on different NLU and NLG tasks.Our empirical experiments on four important and challenging NLU and NLG tasks demonstrate that our proposed logical transformer language models can achieve superior performance over their baseline transformer models through a deeper understanding of the logical structures of texts.
To Copy Rather Than Memorize: A Vertical Learning Paradigm for Knowledge Graph Completion

Embedding models have shown great power in knowledge graph completion (KGC) task.By learning structural constraints for each training triple, these methods implicitly memorize intrinsic relation rules to infer missing links.However, this paper points out that the multihop relation rules are hard to be reliably memorized due to the inherent deficiencies of such implicit memorization strategy, making embedding models underperform in predicting links between distant entity pairs.To alleviate this problem, we present Vertical Learning Paradigm (VLP), which extends embedding models by allowing to explicitly copy target information from related factual triples for more accurate prediction.Rather than solely relying on the implicit memory, VLP directly provides additional cues to improve the generalization ability of embedding models, especially making the distant link prediction significantly easier.Moreover, we also propose a novel relative distance based negative sampling technique (ReD) for more effective optimization.Experiments demonstrate the validity and generality of our proposals on two standard benchmarks.Our code is available at https://github.com/rui9812/VLP.